@suffolklitlab/alkiln 4.10.3 → 4.10.4-cli-playground-1

package/.github/CODEOWNERS

@@ -0,0 +1 @@
+*       @plocket @BryceStevenWilley

package/CHANGELOG.md

@@ -38,9 +38,27 @@ Format:
 ### Security
 - 
 -->
-<!-- ## [Unreleased] -->
+## [Unreleased]
+
+### Added
+- A new script: `alkiln-run`, which acts like `npm run cucumber`, but can
+  be run in any directory, not just in an npm package.
+- Ability to run ALKiln from an interview by a logged-in user
 
+### Changed
+- upgraded cucumber v8.6.0
+- using cucumber's JS API to run tests. For more details on how it works,
+    see [the cucumber-js docs](https://github.com/cucumber/cucumber-js/blob/main/docs/javascript_api.md).
+- the github action no longer runs `npm run XYZ`; it directly calls scripts,
+    e.g. `alkiln-setup`, `alkiln-run`, `alkiln-takedown`
+- don't print the ["publish this cucumber report" message](https://github.com/cucumber/cucumber-js/blob/main/docs/configuration.md#options)
 
+## [4.10.4] - 2023-02-15
+### Added
+- Contribution docs
+
+### Fixed
+- #511, couldn't take screenshots of signature pages https://github.com/SuffolkLITLab/ALKiln/issues/511
 
 ## [4.10.3] - 2023-01-11
 ### Removed
@@ -563,4 +581,4 @@ Format:
 - `example.feature` - an example test to get a project started
 - `github/workflows/run_form_tests.yml` - Automated github workflow to run tests when code is pushed and a manual trigger for testing without needing to push
 - `.gitignore` - ignores files that are generated by the tests and shouldn't be added to the repo
-- `package.json` - keep up-to-date version of this test repository and hold scripts that the automated tests run
+- `package.json` - keep up-to-date version of this test repository and hold scripts that the automated tests run

package/CONTRIBUTING.md

@@ -0,0 +1,205 @@
+# Contributing
+
+For documentation changes, you can just edit and make a pull request. Ignore the instructions below.
+
+## Setup
+
+If you are not already in contact with us, you can join the #assembly-line-test channel in the [docassemble Slack](https://docassemble.org/docs/support.html#tocAnchor-1-1) or [make a new issue](https://github.com/SuffolkLITLab/ALKiln/issues/new) and mention @niharikasingh to let us know you're interested.
+
+1. Asking to join as a contributor is simplest. You can fork, but you will have to change lines in your `.env` file and may have to set GitHub secrets in your repository. You have to talk to us either way.
+1. Clone the repository locally: `git clone git@github.com:SuffolkLITLab/ALKiln.git`
+1. Run `npm install`
+1. Create [your `.env` file](#env) at the root of the project.
+1. Contact @niharikasingh to get important information for your `.env` file.
+1. Run `npm run setup`
+
+Advanced contributors: If you're going to work on our docassemble internal testing interviews, do so as a contributor on this repository. Avoid forking if you can. The reasons are complex. If you feel a need to fork, reach out to us.
+
+## PRs
+
+Before making a pull request
+
+1. Search issues to see if what you are thinking of has been discussed yet.
+1. Make an issue about the change you are thinking of.
+1. Assign yourself to an issue.
+1. Make a new branch.
+1. Make your edits. Feel free to [run tests](#running-internal-tests) as you go.
+1. When you're ready to push
+   1. Run `npm run test` to make sure your code passes all our tests.
+   1. Make an entry in the `CHANGELOG.md` describing your change.
+   1. AVOID changing the version number. We'll do that after merging your changes.
+   1. If needed, update the `README.md` (e.g. the cheat sheet section) and/or `CONTRIBUTING.md` docs. Changes to our architecture might require editing `CONTRIBUTING.md`.
+1. Push to GitHub.
+1. Wait for the GitHub tests to pass. You may have to re-run failing tests—server restarts or race conditions may interfere with them. We have reduced them greatly, but some are impossible to avoid.
+1. Make your pull request. Work in progress pull requests are great too. A pull request must be reviewed before it can be merged.
+
+## .env
+
+You need to make your own `.env` file. It contains sensitive variables, like passwords. We use them for our internal tests. The file should look something like the below:
+
+```bash
+# For everyone
+
+DOCASSEMBLE_DEVELOPER_API_KEY=<ask @niharikasingh>
+SERVER_URL=https://apps-dev.suffolklitlab.org
+REPO_URL=https://github.com/SuffolkLITLab/alkiln
+SERVER_RELOAD_TIMEOUT_SECONDS=
+MAX_SECONDS_FOR_SETUP=
+
+# For internal tests
+
+# Set DEBUG to 1 to run in non-headless mode and see the test run live
+DEBUG=
+# If you edit interview files (.yml files), you should push to a new 
+# branch and update BRANCH_PATH to that branch name
+BRANCH_PATH=v4
+USER1_EMAIL=<ask @niharikasingh>
+USER1_PASSWORD=<ask @niharikasingh>
+SECRET_VAR1=secret-var1-value
+SECRET_VAR2=secret-var2-value
+SECRET_FOR_MISSING_FIELD=secret for missing field
+SECRET_INVALID_THERE_IS_ANOTHER=invalid value for there_is_another
+```
+
+## Running internal tests
+
+Only run this the first time you run tests in this repository to make the code available to test on the docassemble server:
+
+```bash
+npm run setup
+```
+
+If you run `takedown`, you will then have to run `setup` again.
+
+Use the syntax below to trigger all tests (cucumber and unit tests):
+
+```bash
+npm run test
+```
+
+Run only cucumber tests:
+
+```bash
+npm run cucumber
+
+# To test Features or Scenarios specific tags:
+npm run cucumber -- "--tags" "@tagname"
+```
+
+To run the unit tests in isolation:
+
+```bash
+npm run unit
+```
+
+If you or someone else changes the interview code in `./docassemble`, you have to clean up the old data on the server before running `setup` again:
+
+```bash
+npm run takedown
+```
+
+## How the tests run
+
+Our tests require the user/developer to have a docassemble server on which they host these interviews (online forms) and at least one developer account. With our help, our users set up the tests to run when they commit new code or to run manually. When GitHub runs the tests, it does the following:
+
+1. Sets up on the docassemble server. In a given developer account:
+   1. Creates a new interview for this particular branch of the repository.
+   1. Pulls in the code from that branch into the docassemble server.
+1. Runs the tests
+   1. Pretends to be an anonymous account that comes to the form and inputs answers.
+   1. Writes up reports and possibly takes screenshots to show to the developer.
+1. Cleans up by deleting the interview it created
+
+When the developer commits code to GitHub, their account triggers our code, passing along variables that our code needs, like their GitHub secret of the API key for their docassemble server developer account.
+
+## Very general architecture of files and folders
+
+An honest look at our current project architecture—some of our files and folders, what they do, and how they interact.
+
+### Logic architecture
+
+ALKiln uses [cucumber](https://cucumber.io/docs/installation/javascript/)[^1] with [Gherkin](https://cucumber.io/docs/gherkin/reference/) syntax, [puppeteer](https://pptr.dev/), and [chai](https://www.chaijs.com/).
+
+#### .feature files
+
+The `.feature` files are written in Gherkin, a syntax cucumber uses. The "code" in there relies on the functions set up in `./lib/steps.js`.
+
+```js
+// The test_something.feature file step
+Then I sign
+
+// relies on the code in steps.js
+Then('I sign', { timeout: -1 }, async () => {
+  return wrapPromiseWithTimeout(
+    scope.steps.sign( scope ),
+    scope.timeout
+  );
+});
+```
+
+#### steps.js
+
+`./lib/steps.js` defines all the Gherkin sentences that our users (the developers) can write. See [cucumber documentation on defining steps](https://cucumber.io/docs/cucumber/step-definitions/?lang=javascript). We sometimes use "cucumber expressions" and, but often use regular expressions as we often want to be more flexible about the user's input.
+
+The file handles things like:
+
+* Creating state
+* Defining steps for the users/developers
+* Defining steps for internal testing, like testing for appropriate errors and error messages
+* Finishing a Scenario
+* Finishing all tests
+
+#### scope.js
+
+`./lib/steps.js` makes heavy use of `./lib/scope.js`, where most of the action happens. We have decided to mostly keep it in one file to avoid bouncing back and forth between files.
+
+Many `./lib/steps.js` steps make use of the `scope.steps` object. It is there to avoid duplicating code between steps.
+
+The file handles things like:
+
+* Finding form fields on a page
+* Filling out form fields
+* Generating random data for form fields
+* Waiting for pages to load
+* Checking for server restarts
+* Generating test report content
+
+### Logging
+
+`./lib/utils/log.js` takes care of logging information of various kinds (normal, debug, error) to the command line in ways we hope are useful. We should probably find a logging library instead, but this is what we have for now. An important feature is that it ensures the messages clearly belong to ALKiln and not to some other process.
+
+### session_vars.js
+
+The `./lib/utils/session_var.js` file keeps track of what you might otherwise think of as environment variables. We like to think of them as constants, but some of them do need to go through functions, so for consistency we get them all through functions. Also because it was easier to test them when they are functions.
+
+### Setup and takedown logic
+
+All of the setup and takedown logic is in the `./lib/docassemble` directory. It is named after the docassemble API code that is in there. Also, setup and takedown interact very closely with docassemble and the docassemble server.
+
+Other files also use the docassemble API functions, so this folder is in a bit of a messy position.
+
+### Bash commands
+
+When you `npm run` the scripts in our `./package.json` you can setup, run, or takedown tests. We are working on converting this to a command line tool to avoid some complexity in GitHub actions, increase security, and align more with what this framework has grown into. (TODO: Define this better)
+
+### GitHub composite action
+
+Our `./action.yml` is a [composite action](https://docs.github.com/en/actions/creating-actions/creating-a-composite-action). The users/developers let it do most of the work for them. It installs npm, installs our repo, then uses our repo to set up the tests, run the tests, generate and save reports, errors, and screenshots, and clean up the tests.
+
+### Internal tests
+
+We have our own docassemble package in our repo to test our own end-to-end tests. Most of what the package needs is in `./docassemble`. That docassemble package is what the test setup pulls into the docassemble server. Our internal end-to-end cucumber tests are `.feature` files that are stored in `./docassemble/ALKilnTests/data/sources`. The interview (online form) `.yml` files that those `.feature` files use are also deep in the `./docassemble` package. You probably won't be touching the interview files. If you get curious, feel free to ask us.
+
+There are other files the docassemble package needs, like `./setup.py`, so if you see them around, don't worry about them.
+
+Our unit tests are in `./tests/unit_tests` and their filenames end in `.test.js`. Like the cucumber tests, they use chai for assertions. The fixtures for the tests are also contained in that folder and end in `.fixtures.js`.
+
+If you want to know the commands, you can go to the [instructions for running tests](#running-internal-tests).
+
+### Files that you don't need to look at
+
+`index.js` and `world.js` are short files that cucumber needs. `./lib/utils/langs.js` used to work, but we're not sure it does anymore. No one has used it yet, so we're waiting to experiment until someone expresses a need or we have some extra time.
+
+## Footnotes
+
+[1] To be clear, our framework is a misuse of cucumberjs. cucumberjs is geared towards behavior driven development. We try to make BDD available to our developers, but it's not always possible and not necessarily our goal.

package/LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2023 plocket
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,26 +1,66 @@
 # ALKiln
-Integrated automated end-to-end testing with docassemble, puppeteer, cucumber, and chai.
 
-Works well with https://github.com/suffolklitlab/docassemble-AssemblyLine but isn't
-dependent on it.
+ALKiln, or Assembly Line Kiln, is an open source project that helps [docassemble](https://docassemble.org/) developers create automated tests for their interviews (online forms). It works well with the [Document Assembly Line Project](https://suffolklitlab.org/docassemble-AssemblyLine-documentation) but isn't dependent on it.
 
-Formerly named "docassemble-cucumber"
+<!-- 
+https://github.com/18F/open-source-guide/blob/18f-pages/pages/making-readmes-readable.md
+√ What is this repo or project? (You can reuse the repo description you used earlier because this section doesn’t have to be long.)
+√ How does it work?
+√ Who will use this repo or project?
+√ What is the goal of this project?
+-->
 
-## Documentation
-Read about how to use this framework at https://suffolklitlab.org/docassemble-AssemblyLine-documentation/docs/automated_integrated_testing/.
+## Use ALKiln
 
-## Integration tests
-To setup for the integration tests, create the project on the server:
-```
+Read about how to use this framework in [Assembly Line's documentation](https://suffolklitlab.org/docassemble-AssemblyLine-documentation/docs/automated_integrated_testing/).
+
+If you have questions or need to report a bug, [start a new issue](https://github.com/SuffolkLITLab/ALKiln/issues/new).
+
+The following documentation is for internal developers—those working on ALKiln itself.
+
+## Our code
+
+Our code pretends to be a human interacting with online forms (interviews) that have been made with the [docassemble](https://docassemble.org) open source platform. It looks at the HTML DOM, manipulates it, and interacts with it.
+
+It uses [cucumber](https://cucumber.io/docs/installation/javascript/), [puppeteer](https://pptr.dev/), and [chai](https://www.chaijs.com/).
+
+## Our users
+
+Our users are developers of online forms (interviews). Usually they're very new to coding and are unfamiliar with GitHub and with a lot of coding concepts. If they know how to code, it tends to be in Python because that's what docassemble uses. They usually don't have the resource or funding to put into learning these skills.
+
+For example, a pro bono law firm is trying to write a form that people can use to create a restraining order. They're coding it using docassemble. Their users are already going through a hard time, and the forms make them outline their heart wrenching experiences in detail. The developers need to make sure their users don't get system errors right in the middle of that. They use our framework to run the end-to-end tests.
+
+They do good work and we hope to give them a less code-focused way to automate their testing. They avoid having to learn Javascript, GitHub actions, and other such things.
+
+To do that, we use [cucumberjs](https://cucumber.io/docs/installation/javascript/), which uses [Gherkin](https://cucumber.io/docs/gherkin/reference/) syntax to give a user/developer a more human-like syntax to instruct alkiln on what answers to give to the user's online form.
+
+## Contributing
+
+Read about contributing in our [CONTRIBUTING.md document](CONTRIBUTING.md). Here's a quick cheat sheet for some commands once you're up and running:
+
+## Cheat sheet
+
+*Once you've already read the contributing documentation, you can use these as quick reminders for running our internal tests.*
+To set up for the integration tests, create the project on the server:
+
+```bash
 npm run setup
 ```
 
 Use the syntax below to trigger specific tags:
-```
+
+```bash
 npm run cucumber -- "--tags" "@tagname"
 ```
 
 To run the unit tests in isolation:
-```
+
+```bash
 npm run unit
 ```
+
+If you or someone else changes the interview code in `./docassemble`, you have to clean up the old data on the server before running `setup` again:
+
+```bash
+npm run takedown
+```

package/action.yml

@@ -5,15 +5,6 @@ inputs:
   SERVER_URL:
     description: 'The url of your docassemble server. This should be in your GitHub SECRETS or the SECRETS of your org.'
     required: true
-  PLAYGROUND_EMAIL:
-    description: 'The email of your docassemble testing account. This should be in your GitHub SECRETS or the SECRETS of your org.'
-    required: true
-  PLAYGROUND_PASSWORD:
-    description: 'The password of your docassemble testing account. This should be in your GitHub SECRETS or the SECRETS of your org.'
-    required: true
-  PLAYGROUND_ID:
-    description: 'The id of your docassemble testing account. This should be in your GitHub SECRETS or the SECRETS of your org.'
-    required: true
   DOCASSEMBLE_DEVELOPER_API_KEY:
     description: 'API key of the testing account with developer permissions on your docassemble server.'
     required: true
@@ -33,6 +24,16 @@ inputs:
     required: false
     default: '120'
 
+  ## Testing by hand on the server. Should only be created by interview code.
+  #_USER_ID:
+  #  description: 'AVOID SETTING THIS YOURSELF. The id of your docassemble playground account. This is created and used by ALKiln when testing manually on your server.'
+  #  required: false
+  #_PROJECT_NAME:
+  #  description: 'AVOID SETTING THIS YOURSELF. The name of the Playground Project to test manually on your server. ALKiln will ask you for this information.'
+  #  required: false
+  #_ORIGIN:
+  #  description: 'AVOID SETTING THIS YOURSELF. The origin that's triggering this round of tests.'
+
 runs:
   using: "composite"
   steps:
@@ -55,12 +56,12 @@ runs:
         echo "REPO_URL=${{ github.server_url }}/${{ github.repository }}" >> $GITHUB_ENV
         echo "BRANCH_PATH=${{ github.ref }}" >> $GITHUB_ENV
         echo "BASE_URL=${{ inputs.SERVER_URL }}" >> $GITHUB_ENV
-        echo "PLAYGROUND_EMAIL=${{ inputs.PLAYGROUND_EMAIL }}" >> $GITHUB_ENV
-        echo "PLAYGROUND_PASSWORD=${{ inputs.PLAYGROUND_PASSWORD }}" >> $GITHUB_ENV
-        echo "PLAYGROUND_ID=${{ inputs.PLAYGROUND_ID }}" >> $GITHUB_ENV
         echo "DOCASSEMBLE_DEVELOPER_API_KEY=${{ inputs.DOCASSEMBLE_DEVELOPER_API_KEY }}" >> $GITHUB_ENV
         echo "ALKILN_VERSION=${{ inputs.ALKILN_VERSION }}" >> $GITHUB_ENV
         echo "MAX_SECONDS_FOR_SETUP=${{ inputs.MAX_SECONDS_FOR_SETUP }}" >> $GITHUB_ENV
+
+      #  echo "_USER_ID=${{ inputs._USER_ID }}" >> $GITHUB_ENV
+      #  echo "_PROJECT_NAME=${{ inputs._PROJECT_NAME }}" >> $GITHUB_ENV
       shell: bash
     - name: "ALKiln: confirm info"
       run: |
@@ -68,58 +69,31 @@ runs:
         echo -e "\nALKiln version is $ALKILN_VERSION\nRepo is $REPO_URL\nBranch ref is $BRANCH_PATH\nMAX_SECONDS_FOR_SETUP is $MAX_SECONDS_FOR_SETUP\nManually added languages are $EXTRA_LANGUAGES\n"
       shell: bash
 
-    # This doesn't use an already established package-lock.json and that
-    # comes with some types of test breaking risks. Same problem as before.
-    - name: Temporarily overwrite or create package.json
-      run: |
-        cat << EOF > "package.json"
-        {
-          "scripts": {
-            "test": "npm run langs_setup && npm run cucumber -- $@",
-            "cucumber_base": "./node_modules/.bin/cucumber-js --require ./node_modules/@suffolklitlab/alkiln/lib/index.js",
-            "cucumber": "run_cucumber(){ npm run cucumber_base -- \"$@\" docassemble/*/data/sources/*.feature --retry 1; }; run_cucumber",
-            "langs": "npm run langs_setup && ./node_modules/.bin/cucumber-js --require ./node_modules/.bin/alkiln-cucumber docassemble/*/data/sources/*_lang*.feature",
-            "langs_setup": "node_modules/.bin/alkiln-langs 'docassemble/*/data/sources/*.feature'",
-            "setup": "node_modules/.bin/alkiln-setup",
-            "takedown": "node_modules/.bin/alkiln-takedown"
-          },
-          "dependencies": {
-            "colors": "1.4.0",
-            "@cucumber/cucumber": "7.3.2",
-            "@suffolklitlab/alkiln": "$ALKILN_VERSION"
-          }
-        }
-        EOF
-        # Keep in sync with the cucumber.js in root
-        cat << EOF > "cucumber.js"
-        module.exports = {
-          default: "--require ./lib/index.js --format progress --format summary:cucumber-report.txt"
-        }
-        EOF
-      shell: bash
-
     # Install node
     - name: "ALKiln: Install node packages"
       uses: actions/setup-node@v1
       with:
         node-version: "17"
-    - run: npm install
+    # npm install -g "@suffolklitlab/alkiln@$ALKILN_VERSION"
+    - name: Install ALKiln directly from npm
+      run: |
+        npm install -g github:SuffolkLITLab/ALKiln#lighter_package_json
       shell: bash
 
     # Run tests
     - name: "ALKiln: Create a Project and pull the package from GitHub"
-      run: npm run setup
+      run: alkiln-setup
       shell: bash
     - name: "ALKiln: Run tests"
       if: ${{ success() }}
-      run: npm run test -- ${{ github.event.inputs.tags && format('"--tags" "{0}"', github.event.inputs.tags) }}
+      run: alkiln-run ${{ github.event.inputs.tags && format('"{0}"', github.event.inputs.tags) }}
       shell: bash
     - run: echo -e "\n\n====ALKiln could not create a project on your server's testing account or pull your package into it. Check the messages above this line.\n\n"
       if: ${{ failure() }}
       shell: bash
     - name: "ALKiln: Delete project from docassemble test account"
       if: ${{ always() }}
-      run: npm run takedown
+      run: alkiln-takedown
       shell: bash
     
     # Upload artifacts that subscribers can download on the Actions summary page

package/cucumber.js

@@ -1,3 +1,3 @@
 module.exports = {
-  default: "--require ./lib/index.js --format progress --format summary:cucumber-report.txt"
+  default: "--publish-quiet  --require ./lib/index.js --format progress --format summary:cucumber-report.txt"
 }

package/lib/run_cucumber.js

@@ -0,0 +1,92 @@
+#!/usr/bin/env node
+
+const fs = require('fs');
+const log = require('./utils/log');
+const session_vars = require('./utils/session_vars');
+
+// import {loadConfiguration, loadSupport, runCucumber } from '@cucumber/cucumber/api';
+const cucumber_api = require('@cucumber/cucumber/api');
+
+(async function main() {
+  let argv = process.argv;
+
+  let just_args = argv.slice(2);
+
+  const environment = { cwd: process.cwd() };
+  const provided_config = {
+    "publishQuiet": true,
+    "require": "./lib/index.js", 
+    "format": ["progress", "summary:cucumber-report.txt"],
+    "paths": ["docassemble/*/data/sources/*.feature"],
+    "retry": 1,
+  }
+
+  // If they're running it from an interview on their server instead of from GitHub
+  if ( session_vars.get_origin() === `interview` ) {
+    provided_config.paths.append(`/usr/share/docassemble/files/playgroundsources/${session_vars.get_user_id()}/${session_vars.get_user_project_name()}/*.feature`);
+  }
+
+  let tag_vals = just_args;
+  if ( session_vars.get_origin() === `interview` ) {
+    // Should tags be given as a string or a list? String would give option to
+    // give a boolean expression. TODO: offer a dropdown with available tags.
+    tag_vals = session_vars.get_tags().split(` `);
+  }
+
+  if (tag_vals.length > 0) {
+    // Cucumber expects tags in boolean expressions, like (@a0 or @a1) and @random.
+    // If we get a list of only tags, then "or" them all together
+    if (tag_vals.every((val) => val.startsWith(`@`))) {
+      provided_config.tags = tag_vals.join(` or `);
+    }
+    // If we get a list of seemingly no tags, assume they meant to use tags
+    else if (tag_vals.every((val) => !val.startsWith(`@`))) {
+      provided_config.tags = tag_vals.map((val) => `@${ val }`).join(` or `);
+    }
+    // If it's a mix of tags and no tags, it's probably a boolean expression!
+    else {
+      provided_config.tags = tag_vals.join(` `);
+    }
+  }
+
+
+  const { runConfiguration } = await cucumber_api.loadConfiguration({
+    provided: provided_config
+  }, environment);
+  var path = require('path');
+  var codeDir = path.dirname(require.main.filename);
+  const support = await cucumber_api.loadSupport({ support: {
+    requireModules: [`${ codeDir }/index.js`],
+    requirePaths: [ `${ codeDir }/index.js`],
+    importPaths: [],
+  }, sources: {paths: []}}, environment);
+  
+  // TODO: Maybe only log these for debug?
+  console.log(`ALKiln run_cucumber.js configuration:`);
+  console.log(`%o`, runConfiguration);
+
+  const { success } = await cucumber_api.runCucumber({...runConfiguration, support});
+  console.log(`ALKiln run_cucumber.js succeeded:`);
+  console.log(`%o`, success);
+  const exitCode = success ? 0 : 1;
+  process.exitCode = exitCode;
+
+  // This is hardcoded information from `cucumber.js`. Now we on cucumber 8, we should
+  // change that the cucumber.json and actually parse the info from it.
+  // This code isn't used for anything other than detecting and removing color codes 
+  let summary_file_name = 'cucumber-report.txt';
+  fs.readFile(summary_file_name, 'utf8', (err, data) => {
+    if (err) {
+      console.error(err);
+      return;
+    }
+
+    // Replace terminal color codes, if present
+    // https://gist.github.com/fnky/458719343aabd01cfb17a3a4f7296797#colors--graphics-mode
+    data = data.replace(/\x1b\[[0-9][0-9]?[0-9]?m/g, '');
+
+    // Put it in the correct file in the correct folder
+    let artifacts_path = session_vars.get_artifacts_path_name();
+    fs.appendFileSync( `${ artifacts_path}/${ log.debug_log_file }`, data);
+  });
+})();

package/lib/scope.js

@@ -52,7 +52,7 @@ let is_secret_var = function(row) {
   /* `row` might be a row, a source row, or an original row. */
   let original = get_original_row(row);
   return original.flags !== undefined && original.flags.secret_name !== undefined;
-}
+};
 
 let fromBase64 = function(base64_str) {
   /* Takes a base64 encoded string and returns it decoded.
@@ -636,54 +636,6 @@ module.exports = {
 
     return false;
   },
-  
-  setVar: async function ( scope, var_name, value ) {
-    /* Use variable names and values to interact with buttons that
-    *    have values, text input elements, textareas, and dropdowns.
-    * Returns the element that was found.
-    * 
-    * TODO:
-    * - Birthdate fields that will be showing up relatively soon
-    * - Time fields?
-    */
-    let base64_var_name = toBase64( var_name );
-
-    // Will catch some radio and checkbox inputs too, which will then be rejected
-    // Note: I think `showif`s with `code` don't use `data-saveas`
-    let to_manipulate = await Promise.race([
-      scope.page.waitForSelector(`button[name="${base64_var_name}"][value="${value}"]`),  // button
-      scope.page.waitForSelector(`:not(button)[name="${base64_var_name}"]`),  // others (not showif)
-      scope.page.waitForSelector(`*[data-saveas="${base64_var_name}"] input`),  // text input showif
-      scope.page.waitForSelector(`*[data-saveas="${base64_var_name}"] textarea`),  // textarea showif
-      scope.page.waitForSelector(`*[data-saveas="${base64_var_name}"] select`),  // select showif/hideif
-    ]);
-
-    // Throw error for checkboxes and radios
-    let type = await to_manipulate.evaluate( elem => { return elem.getAttribute('type') });
-    if ( type === `checkbox` || type === `radio` ) {
-      throw new TypeError(`You need the ${type} step, e.g. 'I tap the ${type} variable "some_dict" with the value "choice_value"'`);
-    }
-
-    // Actions based on element
-    let tagName = await to_manipulate.evaluate( elem => { return elem.tagName });
-    tagName = tagName.toLowerCase();
-
-    if ( tagName === `button` ) {  // Tap buttons
-      await scope.tapElement( scope, to_manipulate );
-
-    } else if ( tagName === `select` ) {  // Choose drop-down option
-      await to_manipulate.select(value);
-
-    } else if ( tagName === `input` || tagName === `textarea` ) {
-      // Set text field values (includes any other input fields)
-      await to_manipulate.evaluate( el => { el.value = '' });
-      await to_manipulate.type( value );
-      // TODO: Add hideif to this too
-    }
-
-    // Do your own waiting if you need to in the step
-    return to_manipulate;
-  },  // Ends scope.setVar()
 
   continue_button_selector: `fieldset.da-field-buttons button[type="submit"].btn-primary`,
   signature_selector: `fieldset .dasigsave`,
@@ -1736,7 +1688,7 @@ module.exports = {
       await scope.afterStep(scope, { waitForShowIf: true });
     },
     canvas: async function ( scope, { handle, answer }) {
-      await scope.sign( scope );
+      await scope.draw_signature( scope );
       await scope.afterStep( scope );
     },
     input: async function ( scope, { handle, answer }) {
@@ -1919,7 +1871,7 @@ module.exports = {
     }, 500)
   },
 
-  sign: async function ( scope ) {
+  draw_signature: async function ( scope ) {
     /* On a signature page, there should only be a single signature canvas
     * and a continue button. Draw in the #dasigcanvas element. `<canvas>`
     * takes a while to load. */
@@ -2327,6 +2279,23 @@ module.exports = {
     return scope;
   },  // Ends scope.throwPageError()
 
+  take_a_screenshot: async ( scope,{ path }) => {
+    /* Takes a jpeg screenshot. Avoids destroying signatures. */
+
+    let fullPage = true;
+    let signature_elem = await scope.page.$(scope.signature_selector);
+    if ( signature_elem !== null ) {
+      fullPage = false;
+    }
+
+    await scope.page.screenshot({
+      path: path,
+      type: 'jpeg',
+      fullPage: fullPage,
+    });
+
+  },  // Ends scope.take_a_screenshot()
+
 
   //#####################################
   //#####################################
@@ -2349,14 +2318,10 @@ module.exports = {
         name = `screenshot_on-${ id }-${ date }`;
       }
 
-      await scope.page.screenshot({
-        path: `./${ scope.paths.scenario }/${ name }.jpg`,
-        type: 'jpeg',
-        fullPage: true
-      });
+      await scope.take_a_screenshot( scope, { path: `./${ scope.paths.scenario }/${ name }.jpg` });
 
       await scope.afterStep( scope );
-    },  // Ends screenshot()
+    },  // Ends scope.steps.screenshot()
 
     add_json_vars_to_report: async ( scope ) => {
       /** Uses  https://docassemble.org/docs/functions.html#js_get_interview_variables
@@ -2371,7 +2336,7 @@ module.exports = {
         type: `json`,
         value: JSON.stringify( data, null, 2 )
       });
-    },  // Ends add_json_vars_to_report()
+    },  // Ends scope.steps.add_json_vars_to_report()
 
     link_works: async ( scope, linkText ) => {
       let [link] = await scope.page.$x(`//a[contains(text(), "${ linkText }")]`);
@@ -2390,7 +2355,7 @@ module.exports = {
       linkPage.close()
 
       await scope.afterStep(scope);
-    },  // Ends link_works()
+    },  // Ends scope.steps.link_works()
 
     set_regular_var: async ( scope, var_name, answer ) => {
       /** Set a non-story table variable (with or without a choice associated with it) to a value. Set:
@@ -2408,7 +2373,7 @@ module.exports = {
       // Don't continue if the variable doesn't cause navigation itself. Also,
       // if this variable cannot be set on this page, trigger an error
       await scope.setFields( scope, { var_data, ensure_navigation: false, ensure_all_vars_used: true });
-    },  // Ends set_regular_var()
+    },  // Ends scope.steps.set_regular_var()
 
     set_secret_var: async ( scope, var_name, answer_env_name ) => {
       /** Sets a non-story table variable to a "secret" value (i.e. an env var) */
@@ -2425,7 +2390,7 @@ module.exports = {
       // Don't continue if the variable doesn't cause navigation itself. Also,
       // if this variable cannot be set on this page, trigger an error
       await scope.setFields( scope, { var_data, ensure_navigation: false, ensure_all_vars_used: true });
-    },
+    },  // Ends scope.steps.set_secret_var
 
     tap_term: async ( scope, phrase ) => {
       /** Tap a link that doesn't navigate. Depends on the language of the text. */
@@ -2438,13 +2403,13 @@ module.exports = {
       await scope.tapElement( scope, link );
 
       await scope.afterStep(scope, {waitForShowIf: true});
-    },  // Ends tap_term()
+    },  // Ends scope.steps.tap_term()
 
     sign: async ( scope ) => {
       /** Sign on a signature page. */
-      await scope.sign( scope );
+      await scope.draw_signature( scope );
       await scope.afterStep(scope);
-    },  // Ends sign()
+    },  // Ends scope.steps.sign()
 
     download: async ( scope, filename ) => {
       /* Taps the link that leads to the given filename to trigger downloading.
@@ -2466,7 +2431,7 @@ module.exports = {
       // Should this be using `scope.tapElement()`?
       await elem.evaluate( elem => { return elem.click(); });
       await scope.detectDownloadComplete( scope );
-    },  // Ends download()
+    },  // Ends scope.steps.download()
 
     set_al_name: async ( scope, var_base, name_str ) => {
       /** Given a variable name and a space-separated string, fills the
@@ -2503,15 +2468,20 @@ module.exports = {
 
       let desired_pattern = name_patterns[( name_parts.length - 1 )];
 
+      let raw_data = [];
       for ( let parti = 0; parti < name_parts.length; parti++ ) {
         let var_name = `${ var_base }.name.${ desired_pattern[ parti ] }`
         let value = name_parts[ parti ];
-        await scope.setVar( scope, var_name, value );
+        raw_data.push({ var: var_name, value: value, trigger: scope.trigger_not_needed_flag });
       }
 
+      let var_data = await scope.normalizeTable( scope, { var_data: raw_data, });
+      // Avoid continuing automatically. If this var cannot be set on this page, error
+      await scope.setFields( scope, { var_data, ensure_navigation: false, ensure_all_vars_used: true });
+
       // Continue to next page...?
       await scope.afterStep(scope, {waitForShowIf: true});
-    },  // Ends set_al_name()
+    },  // Ends scope.steps.set_al_name()
 
     set_al_address: async ( scope, var_base, address_str ) => {
       /** Given a variable name and a four-part address formatted as below,
@@ -2530,16 +2500,21 @@ module.exports = {
       let address_parts = split.concat( last_parts );
       let start = `${ var_base }.address.`;
 
-      let index = 0;
-      await scope.setVar( scope, `${start}address`, address_parts[0] );
-      // Ensure it's without 'Unit '/'unit '
-      await scope.setVar( scope, `${start}unit`, (address_parts[1].toLowerCase()).replace(/unit\s*/, '') );
-      await scope.setVar( scope, `${start}city`, address_parts[2] );
-      await scope.setVar( scope, `${start}state`, address_parts[3] );
-      await scope.setVar( scope, `${start}zip`, address_parts[4] );
+      let no_trigger = scope.trigger_not_needed_flag;  // just making the lines shorter
+      let raw_data = [
+        { var: `${start}address`, value: address_parts[0], trigger: no_trigger },
+        // Remove accidental 'Unit' or 'unit ' text in case dev included it accidentally
+        { var: `${start}unit`, value: (address_parts[1].toLowerCase()).replace(/unit\s*/, ''), trigger: no_trigger },
+        { var: `${start}city`, value: address_parts[2], trigger: no_trigger },
+        { var: `${start}state`, value: address_parts[3], trigger: no_trigger },
+        { var: `${start}zip`, value: address_parts[4], trigger: no_trigger },
+      ];
+      let var_data =  await scope.normalizeTable( scope, { var_data: raw_data, });
+      // Avoid continuing automatically. If this var cannot be set on this page, error
+      await scope.setFields( scope, { var_data, ensure_navigation: false, ensure_all_vars_used: true });
 
       await scope.afterStep(scope, {waitForShowIf: true});
-    },  // Ends set_al_address()
+    },  // Ends scope.steps.set_al_address()
 
     sign_in: async function ( scope, { email_secret_name, password_secret_name }) {
       /** Allow the developer to log a user into their server using GitHub

package/lib/steps.js

@@ -92,7 +92,7 @@ BeforeAll(async () => {
   // Make the folder to store all the files for this run of the tests
   scope.paths.artifacts = files.make_artifacts_folder();
   // Save its name in a file. `session_vars` can get it back later.
-  // This is used by at least clean_reports.js.
+  // This is used by at least run_cucumber.js.
   session_vars.save_artifacts_path_name( scope.paths.artifacts );
 
   scope.paths.report = `${ scope.paths.artifacts }/report.txt`;
@@ -103,7 +103,7 @@ BeforeAll(async () => {
 Before(async (scenario) => {
   // Create browser, which can't be created in BeforeAll() where .driver doesn't exist for some reason
   if (!scope.browser) {
-    scope.browser = await scope.driver.launch({ headless: !session_vars.get_debug(), devtools: session_vars.get_debug() });
+    scope.browser = await scope.driver.launch({ args: ['--no-sandbox'], headless: !session_vars.get_debug(), devtools: session_vars.get_debug() });
   }
   // Clean up all previously existing pages
   for (const page of await scope.browser.pages()) { 
@@ -846,11 +846,8 @@ When(/I answer randomly for at most ([0-9]+) (?:page(?:s)?|screen(?:s)?)/, { tim
           let buttons = await scope.steps.set_random_page_vars( scope );
 
           // Take a screenshot of the page before continuing or ending
-          await scope.page.screenshot({
-            path: `${ scope.paths.random_screenshots }/random-${ Date.now() }.jpg`,
-            type: `jpeg`,
-            fullPage: true
-          });
+          let path = `${ scope.paths.random_screenshots }/random-${ Date.now() }.jpg`;
+          await scope.take_a_screenshot( scope, { path: path });
 
           // Prep for the next loop to possibly stop. Signature page buttons
           // don't count as buttons in the way our field detects them
@@ -1119,27 +1116,23 @@ After(async function(scenario) {
       if ( !!scope.disable_error_screenshot ) {
         scope.addToReport(scope, {
           type: `row`,
-          value: `For security, ALKiln will not create a screenshot for this error. It's possible a secret is being used on this screen.`
+          value: `For security, ALKiln will avoid creating a screenshot for this error. It's possible a secret is being used on this screen.`
         });
       } else {
 
         // Save/download a picture of the screen that's showing during the unexpected status
         // Save one copy in the outer-most artifact folder
         let scenario_filename = await scope.getSafeScenarioFilename( scope, { prefix: `error_on` });
-        await scope.page.screenshot({
-          path: `${ scope.paths.artifacts }/${ scenario_filename }.jpg`,
-          type: `jpeg`,
-          fullPage: true
-        });
+        let path_outer = `${ scope.paths.artifacts }/${ scenario_filename }.jpg`;
+        await scope.take_a_screenshot( scope, { path: path_outer });
+
         // Save another copy in the artifact's Scenario folder
         let screenshot_name = `error_on`;
         let { id } = await scope.examinePageID( scope, 'none to match' );
         screenshot_name += `-${ id }`;
-        await scope.page.screenshot({
-          path: `${ scope.paths.scenario }/${ screenshot_name }.jpg`,
-          type: `jpeg`,
-          fullPage: true
-        });
+        let path_scenario = `${ scope.paths.scenario }/${ screenshot_name }.jpg`;
+        await scope.take_a_screenshot( scope, { path: path_scenario });
+
       }  // ends if scope.disable_error_screenshot
     }  // ends if scope.page exists
 

package/lib/utils/log.js

@@ -52,7 +52,8 @@ log.debug = function ({ type='', pre='', data='', post='' }) {
 };
 
 log.add_to_debug_log = function(value) {
-  fs.appendFileSync(log.debug_log_file, value);
+  let artifacts_path = session_vars.get_artifacts_path_name();
+  fs.appendFileSync(`${ artifacts_path }/${ log.debug_log_file}`, value);
 };
 
 log.debug_log_file = 'debug_log.txt'

package/lib/utils/session_vars.js

@@ -23,9 +23,19 @@ module.exports = session_vars;
 // These default to null
 // TODO: How to isolate debug elsewhere so this file can use the logger?
 session_vars.get_debug = function () { return process.env.DEBUG || null; };
+session_vars.is_dev_env = function () { return process.env.DEV || null; }
+
 session_vars.get_dev_api_key = function () { return process.env.DOCASSEMBLE_DEVELOPER_API_KEY || null; };
 session_vars.get_repo_url = function () { return process.env.REPO_URL || null; };
-session_vars.is_dev_env = function () { return process.env.DEV || null; }
+
+// Created by interview code when tests through an interview manually on the server
+session_vars.get_origin = function () {
+  /** Possible values: 'interview', 'github'. In future, maybe 'playground' or 'dashboard'. */
+  return process.env._ORIGIN || null;
+};
+session_vars.get_user_project_name = function () { return process.env._PROJECT_NAME || null; };
+session_vars.get_user_id = function () { return process.env._USER_ID || null; };
+session_vars.get_tags = function () { return process.env._TAGS || null; };
 
 // More complex logic
 session_vars.get_server_reload_timeout = function () {
@@ -129,8 +139,12 @@ session_vars.save_project_name = function ( project_name ) {
   return file;
 };  // Ends save_project_name()
 
+// TODO next: Add use of env var in here in case not through GitHub cli_playground
 session_vars.get_project_name = function () {
-  /* Get the name of the current docassemble Project  */
+  /* Get the name of the current docassemble Project */
+  if ( session_vars.get_origin() === 'interview' ) {
+    return session_vars.get_user_project_name();
+  }
   let json = JSON.parse( fs.readFileSync( runtime_config_path ));
   let project_name = json[ project_name_key ] || null;
   if ( session_vars.get_debug() ) { console.log( `ALKiln debug: Project name from file is "${ project_name }".` ); }
@@ -174,7 +188,7 @@ session_vars.save_artifacts_path_name = function ( path_name ) {
     console.log( `ALKiln debug: Stored name of artifacts directory "${ path_name }" locally.` );
   }
   return file;
-};  // Ends get_project_name()
+};  // Ends save_artifacts_path_name()
 
 session_vars.get_artifacts_path_name = function () {
   /* Get the name of the current artifacts path. */
@@ -187,7 +201,7 @@ session_vars.get_artifacts_path_name = function () {
   } catch ( error ) {
     return null;
   }
-};  // Ends get_project_name()
+};  // Ends get_artifacts_path_name()
 
 /**
  * Ensure all required environment variables exist. Add each missing variable

package/package.json

@@ -1,14 +1,12 @@
 {
   "name": "@suffolklitlab/alkiln",
-  "version": "4.10.3",
+  "version": "4.10.4-cli-playground-1",
   "description": "Integrated automated end-to-end testing with docassemble, puppeteer, and cucumber.",
   "main": "lib/index.js",
   "scripts": {
     "test": "npm run unit && npm run langs_setup && npm run cucumber -- $@",
-    "cucumber_base": "run_cucumber(){ ./node_modules/.bin/cucumber-js \"$@\"; status=$?; ./lib/utils/clean_reports.js; return $status; }; run_cucumber",
-    "cucumber": "run_base(){ npm run cucumber_base -- \"$@\" docassemble/*/data/sources/*.feature --retry 1; }; run_base",
-    "cucumber_external": "run_cucumber(){ tmp_path=`pwd`; ( new_path=\"$1\"; shift; cd \"$new_path\"; npm run --prefix \"${tmp_path}\" cucumber_base -- \"$@\" \"$new_path\"/docassemble/*/data/sources/*.feature;) }; run_cucumber",
-    "langs": "npm run langs_setup && ./node_modules/.bin/cucumber-js --require ./lib/index.js docassemble/*/data/sources/*_lang*.feature",
+    "cucumber": "./lib/run_cucumber.js",
+    "langs": "npm run langs_setup && npm run cucumber",
     "langs_setup": "./lib/utils/langs.js 'docassemble/*/data/sources/*.feature'",
     "langs_local": "npm run langs_setup && npm run local",
     "local": "npm run setup && npm run cucumber; npm run takedown",
@@ -36,13 +34,13 @@
   "homepage": "https://suffolklitlab.org/docassemble-AssemblyLine-documentation/docs/automated_integrated_testing/",
   "dependencies": {
     "@axe-core/puppeteer": "4.4.2",
-    "@cucumber/cucumber": "7.3.2",
+    "@cucumber/cucumber": "8.6.0",
     "@faker-js/faker": "^7.3.0",
     "axios": "0.24.0",
     "chai": "4.2.0",
     "cheerio": "1.0.0-rc.10",
-    "dotenv": "8.2.0",
     "deep-equal-in-any-order": "^2.0.0",
+    "dotenv": "8.2.0",
     "mocha": "9.2.0",
     "puppeteer": "13.1.3",
     "qs": "6.10.2",
@@ -52,6 +50,7 @@
   "bin": {
     "alkiln-setup": "./lib/docassemble/setup.js",
     "alkiln-takedown": "./lib/docassemble/takedown.js",
-    "alkiln-langs": "./lib/utils/langs.js"
+    "alkiln-langs": "./lib/utils/langs.js",
+    "alkiln-run": "./lib/run_cucumber.js"
   }
 }

package/lib/utils/clean_reports.js

@@ -1,29 +0,0 @@
-#!/usr/bin/env node
-
-const fs = require('fs');
-const log = require('./log');
-const session_vars = require('./session_vars');
-
-// This is hardcoded information from `cucumber.js`. Once we move to cucumber 8, we should
-// change that the cucumber.json and actually parse the info from it.
-// The file won't be used for anything other than detecting and removing color codes 
-let summary_file_name = 'cucumber-report.txt';
-fs.readFile(summary_file_name, 'utf8', (err, data) => {
-  if (err) {
-    console.error(err);
-    return;
-  }
-
-  // Replace terminal color codes, if present
-  // https://gist.github.com/fnky/458719343aabd01cfb17a3a4f7296797#colors--graphics-mode
-  data = data.replace(/\x1b\[[0-9][0-9]?[0-9]?m/g, '');
-
-  // Put it in the correct file in the correct folder
-  let artifacts_path = session_vars.get_artifacts_path_name();
-  fs.appendFile( `${ artifacts_path }/${ log.debug_log_file }`, data, err => {
-    if (err) {
-      console.error(err);
-      return;
-    }
-  });  // Ends appendFileSync
-});