@suffolklitlab/alkiln 5.12.0 → 5.13.0-logs2

package/docs/decisions/detect_sign_in.md

@@ -0,0 +1,124 @@
+# Use response data to detect sign-in success
+
+## Context and Problem Statement
+
+What is a robust way to detect whether signing in worked or not?
+
+## Considered Options
+
+- Elements in the DOM.
+  - Use some class or ID that only the sign-in page has, so we can detect that a user is on a new page and succeeded.
+  - User-facing invalidation message elements in the DOM on failure.
+- URL change.
+- Response data.
+
+## Decision Outcome
+
+Use a change in the URL to detect sign-in navigation.
+
+## Pros and Cons of the Options
+
+### DOM
+
+Pros:
+
+- It can be easy to do with the tools we are using.
+
+Cons:
+
+- The DOM can, and has been, changed by docassemble and these kinds of changes are made pretty quietly so we find it hard to keep track of them.
+- It is pretty easy for site developers to change the HTML of standard pages like the sign-in page.
+
+### URL change
+
+Pros:
+
+- It seems very likely that developers will keep an unsuccessful sign-in on the same page, even if they change some other functionality.
+- It seems very likely that developers will change the url when the sign-in succeeds.
+- Sticking with checking the url would mesh with how we currently detecting page navigation. See the cons for issues with that.
+
+Cons:
+
+- It seems precarious, though I haven't yet been able to articulate why. I know it hasn't been a sufficient full solution in the past for interview pages. For example, in a `show if` infinite page loop.
+- Developers can manipulate a page's url, though it seems unlikely in this situation.
+
+### Response data
+
+Use the response data from the sign in request.
+
+Pros:
+
+- It seems reliable - it seems very likely that developers will re-direct their users to a different page, even if they change some other functionality.
+- Less likely to change than the DOM.
+- Has some standard values we can expect, like status codes.
+
+Cons:
+
+- DA's responses are not typical. There's no 400's in unsuccessful responses.
+- Regular interview page navigation doesn't get a 302, so this method would be a special case for signing in.
+
+Options are:
+
+- 302 redirection status. It seems fairly reliable that a developer will send a user to a different page after they sign in, and also likely that they will keep the user on the same page if the sign-in failed.
+- Cache-Control header value that contains "must-revalidate" (See [MDN cache control directives](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#directives)). It makes sure this data isn't cached. I'm not familiar with this and I'm not sure what might or might not trigger this.
+- Cookie with the name "secret". This seems promising, but I don't know much about cookies.
+
+Based on our current knowledge, we think the 302 status response will be most reliable.
+
+Notes for response data:
+
+I have isolated the differences between the sign-in success and failure responses.
+
+Key for following entries:
+
+```
+<key of object containing the data>
+  🌈 <sign in success data>
+vs
+  🤕 <sign in failure data>
+```
+
+Findings:
+
+```
+response
+  🌈 "redirectURL": "/path", "headersSize": 552, "bodySize": 0, "_transferSize": 552,
+vs
+  🤕 "redirectURL": "", "headersSize": 570, "bodySize": 1998, "_transferSize": 2568,
+---
+response
+  🌈 "status": 302, "statusText": "FOUND",
+vs
+  🤕 "status": 200, "statusText": "OK",
+
+---
+headers
+  🌈 no cache-control
+vs
+  🤕 { "name": "Cache-Control", "value": "no-store, no-cache, must-revalidate, post-check=0, pre-check=0, max-age=0" },
+---
+headers
+  🌈 { "name": "Content-Length", "value": "209" },
+vs
+  🤕 { "name": "Content-Encoding", "value": "gzip" },
+---
+headers
+  🌈 { "name": "Location", "value": "/interviews" },
+vs
+  🤕 no location
+---
+headers
+  🌈 { "name": "Set-Cookie", "value": "secret=secretchars; Secure; HttpOnly; Path=/; SameSite=Lax" },
+vs
+  🤕 no secret
+---
+headers
+  🌈 no transfer-encoding
+vs
+  🤕 { "name": "Transfer-Encoding", "value": "chunked" },
+---
+cookies
+  🌈 { "name": "secret", "value": "secretchars", "path": "/", "domain": "site.com", "expires": null, "httpOnly": true, "secure": true, "sameSite": "Lax" },
+vs
+  🤕 no secret
+```

package/docs/decisions/pdfjs_version_2024_08_08.md

@@ -0,0 +1,18 @@
+# Leave pdfjs at its current version
+
+## Context and Problem Statement
+
+pdfjs has a security vulnerability (that does not affect us) where malicious PDFs can inject code into an unsuspecting project and cause havoc.
+
+Our users are downloading and manipulating their own PDFs. We will assume they're not putting malicious code into their PDFs.
+
+The update changes pdfjs .js files to be .mjs files. It might take some bother to configure our repo to use .mjs files.
+
+## Considered Options
+
+- Update pdfjs
+- Leave pdfjs at its current version, 3.2.146
+
+## Decision Outcome
+
+Leave pdfjs at its current version, 3.2.146. The vulnerability is irrelevant for our situation and we don't have bandwidth to set up the configurations right now.

package/docs/decisions/reports_and_logs_2024_08_11.md

@@ -0,0 +1,141 @@
+# Shape and behavior of the report and log objects
+
+## Context and scope
+
+We have reports and logs we show to the user in the console and as artifacts. We have logs we use for development. cucumber also has logs we want to include in the above until we are confident we are handling all cases ourselves.
+
+Right now, the system for logging is fractured and scattered - reports log to the console and save completely separately from other logs. Logs that are not from the report don't get saved anywhere. Some logs only get shown in debug mode, which can be frustrating and unhelpful.
+
+## Goals
+
+Overview:
+
+- Unify the logging and report systems
+- Clarify their flow
+- Make log information more visible
+
+Details:
+
+- Log objects have statuses, icons, log codes (e.g. ALK0105), contextual info (e.g. `setup`, `takedown`, `invalid answer`), timestamps, descriptive text, and data (like full error traces).
+- These are the broad purposes of various logs:
+1. Setup and takedown messages.
+2. Test progress (a compact way to see what's happening/happened in each test):
+
+   Test1: .............F--
+   
+   Test 2: ..
+3. A pretty report at the end with details about each test. It has sections, like "failed" and "passed", headings, etc.
+4. cucumber's various console logs
+5. Lots of other informational logs for internal development and debugging purposes.
+I want to do different things with different logs:
+- During the tests, always log items 1, 2, 3, and 4 to the console.
+- During the tests, save items 2, 3, and 4 to a log as they come with all the info in the log objects - `report_log.txt`/`running_report.txt`/`report_run.txt` maybe. If we have to stop the tests early because of an infinite loop or something, this will at least show some information.
+- At the end of the tests, format items 2, 3, and 4, often with only part of the log object info, in a file called `report.txt`.
+- During the tests, save all of the items to a `verbose_log.txt` as they happen.
+- In debug mode, log all that stuff to the console while the tests are running.
+Right now, a few of the logs, like for the test progress (item 2), are printed with `stdout` inline while all others are logged with `console.log`.
+- Logs should accept multiple arguments to match console.log/etc. behavior.
+
+Note: The progress logs printed to the console have color codes. We also may want a plain text version without any colors to save in the report log and show in the Playground[^1] "console" and/or a version converted to HTML that does have styles. The latter may be for something easier to digest in a browser and/or to show visually in the Playground version of the tests.
+
+Questions:
+- Should `log` throw errors? Otherwise everywhere we need to error, we need to use `log` and `throw`, but I am unable to find a way to remove the current function from an error stack/trace.
+- How do we capture "errors" that are more subtle? Keep the word "error" in the "context"/"types" list? For example, expected errors:
+  - Invalid answers on the form that are then assessed as valid behavior?
+  - `AbortError`s of promises that should indeed have been aborted.
+- Should we allow "in-between"s for all parts of the log? All together: pre-everything (new lines, dividers, etc), icon/context/etc., pre-logs, logs (the actual messages to print), pre-data, data, post-data? We can default to nothing for all of those.
+- How do we differentiate between methods that log to the console vs. methods that just store data?
+
+[^1]: Don't go down this rabbit hole. Broadly - with the "Playground" version, users see the results as a web page instead of in the GitHub job console or in the artifacts. The users run the tests on their own server. This has to do with the platform we built ALKiln for - [docassemble](https://docassemble.org/). It gives users a faster iteration cycle.
+
+## Non/anti-goals
+
+- Spend a month on this
+- Create a complex and heavy system to try to abstract absolutely everything
+
+## The actual design
+
+This design still seems a little complex and potentially too coupled with reports.
+
+### System-context diagram
+
+```
+logic code -> logger -> report -----------> console
+              ^     |-> console         |-> stdout/progress
+  cucumber ---|     |-> running report  |-> final report.txt
+                    |-> verbose log
+```
+
+```
+logic code -> logger ------> report ------> final report.txt
+              ^     |-> console
+  cucumber ---|     |-> stdout/inline/progress
+                    |-> running report
+                    |-> verbose log
+```
+
+```
+              |------------------------------------<|
+              v                                     |
+logic code -> logger -> report -----------> format prettily
+              ^     |-> console
+  cucumber ---|     |-> running report
+                    |-> verbose log
+                    |-> stdout/progress
+                    |-> final report.txt
+```
+
+### APIs
+
+Everything saves to the "verbose"/"debug" log. Aside from that:
+
+`log.info/warning/success/error` methods log to console. This includes, e.g, setup, takedown which show up in GitHub workflow job logs.
+
+`log.report` saves to the running report then uses a report obj focused on cosmetics that has `.heading/table/header/error/etc` methods. Maybe a `.stdout/inlne` method, which then prints to the console immediately.
+
+`log.verbose/debug` (name?) - Only save to verbose/debug log with log codes, etc.
+
+### Data storage
+
+- `report.txt` - final pretty output for the user with headings, etc.
+- Each Scenario's `report.txt` - pretty output in each test's folder.
+- `running_report_log.txt` (name?) - ugly, but still sparse, giving the user some info about what happened during the tests without overwhelming them.
+- `verbose.txt` (name? `debug`?) - very ugly and full of all the information from everywhere else and more. Useful for internal development and troubleshooting. Includes setup and takedown.
+
+It may be complex to store setup and takedown output in the verbose/debug file. `console.Console` cannot append to a file[^2], so we would have to figure out a work-around. One approach could be to always first copy from the existing verbose file and paste those contents back into the file as the first log. That may take setting up some fallbacks to avoid errors.
+
+[^2]: `fs.appendFile()` only takes strings and doesn't log exactly like the console. I want to log exactly like the console to make it easy to search for words I and others see in the console output.
+
+### Code and pseudo-code
+
+For capturing console output, look into how to get the `stdin` for capturing various console output. https://stackoverflow.com/a/54202970:
+
+```js
+const stdin = process.openStdin()
+
+process.stdout.write('Enter name: ')
+
+stdin.addListener('data', text => {
+  const name = text.toString().trim()
+  console.log('Your name is: ' + name)
+
+  stdin.pause() // stop reading
+})
+```
+
+### Degree of constraint
+
+I think this can basically be greenfield (designed from scratch). We do have a current system, but I think we can easily convert our existing code to a new system.
+
+## Alternatives considered
+
+- The `report` object saves to the running report and then sends that same information to the `log` object. Also prints the final report. This includes `stdout` logs. `log` remains otherwise the same as described above.
+- `log` handles `stdout` logs and leaves them out of the report.
+- For report content, `log` would call to `report`, which would do pretty formatting and then hand the format text back to `log` to be saved and/or printed to the console. I'm not sure how `log` could then know what to do with the output - leave out the codes/etc. when it logs to the console and/or saves to the final report.txt.
+- Avoid the complexity of storing setup and takedown output in the verbose/debug file. In GitHub, they're in the GitHub console. Locally, we see the results immediately. On the other hand, storing them ourselves lets us look for information in just one place.
+
+## Cross-cutting concerns
+
+_Security, privacy, observability, etc._
+
+The same ones we've already dealt with for our current logs.

package/docs/decisions/update_puppeteer_2024_07_28.md

@@ -0,0 +1,62 @@
+# Use response data to detect sign-in success
+
+## Context and Problem Statement
+
+Should we update to the newest version of puppeteer, [22.15.0](https://github.com/puppeteer/puppeteer/blob/main/packages/puppeteer-core/CHANGELOG.md#22150-2024-07-31)?
+
+See https://github.com/SuffolkLITLab/ALKiln/issues/930.
+
+## Considered Options
+
+- Update to 22.15.0
+- Update to 22.12.0, which is the lowest version that would fix our current problems
+- Stay with 20.8.2
+
+See pros and cons
+
+## Decision Outcome
+
+Update puppeteer from 20.8.2 to 22.15.0
+
+## Pros and Cons of the Options
+
+### Update at all {#update}
+
+**Pros:**
+
+- Makes the current fix possible
+
+**Cons:**
+
+- Change in the middle of a complex PR
+- [A few breaking changes](https://github.com/puppeteer/puppeteer/blob/main/packages/puppeteer-core/CHANGELOG.md#2200-2024-02-05), which includes for us at the very least:
+   - "Removes the deprecated `$x` (replace with `$$`) and `waitForXpath` (replace with `waitForSelector`)." We need to add [extra syntax to the start of our selector strings](https://github.com/puppeteer/puppeteer/pull/11782): "xpath//."
+   - Replace any `page.waitForTimeout` with cucumber's version or ours.
+
+### Update to to 22.15.0
+
+**Pros:**
+
+- Has all the most recent bug fixes and features
+
+**Cons:**
+
+- Exposes us to possible unknown new bugs
+
+
+### Update to to 22.12.0
+
+This is the lowest version that would fix our problem
+
+**Pros:**
+
+- Avoids possibly unknown new bugs
+
+**Cons:**
+
+- Doesn't have fixes to already known bugs
+
+
+### Stay with 20.8.2
+
+Opposite of first entry ["Update at all"](#update).

package/docs/story_tables.md

@@ -4,7 +4,7 @@ The Story Table is a very flexible tool. It lets the author answer interview (fo
 
 When the robot arrives on each interview page, it finds all the fields on the page and loops through the whole Story Table trying to match rows with fields.
 
-Here's some [documentation on the use of Story Table rows](https://suffolklitlab.org/docassemble-AssemblyLine-documentation/docs/alkiln/#story-tables).
+Here's some [documentation on the use of Story Table rows](https://suffolklitlab.org/docassemble-AssemblyLine-documentation/docs/alkiln/writing#story-table).
 
 ## Tracking use of rows
 

package/lib/docassemble/docassemble_api_REST.js

@@ -31,7 +31,28 @@ da.get_dev_id = async function ( timeout ) {
 };  // Ends da.get_dev_id()
 
 
-// node -e 'require("./lib/docassemble/docassemble_api_REST.js").delete_project( "create1" )'
+da.get_api_key_info = async function ( timeout, { api_key = null, server_url = null }) {
+  /** Get information about an API key owned by the user of the given API key. */
+  let options = {
+    method: 'GET',
+    headers: {'X-API-Key': api_key},
+    url: `${ server_url }/api/user/api`,
+    params: {
+      api_key: api_key,
+    },
+    timeout: timeout,
+  };
+
+  try {
+    return await axios.request( options );
+  } catch (error) {
+    let response = error.response || { data: null }
+    throw {...error.toJSON(), data: response.data};
+  }
+};  // Ends da.get_api_key_info()
+
+
+// node -e 'require("./lib/docassemble/docassemble_api_REST.js").delete_project()'
 da.delete_project = async function ( timeout ) {
   /* Delete a project on the user's da server. */
   let options = {
@@ -53,6 +74,51 @@ da.delete_project = async function ( timeout ) {
 };  // Ends da.delete_project()
 
 
+// node -e 'require("./lib/docassemble/docassemble_api_REST.js").delete_interview(...)'
+da.delete_interview = async function ( timeout, { session = null, api_key = null }) {
+  /* Delete an interview with the given session id. */
+  let options = {
+    method: 'DELETE',
+    headers: { 'X-API-Key': api_key },
+    // Can't use `api/session` that uses the API key to identify the user's
+    // interview sessions. The test may have signed in with an account
+    // that's different than the ALKiln dev account that sends this request.
+    url: `${ session_vars.get_da_server_url() }/api/user/interviews`,
+    params: { session },
+    timeout: timeout,
+  };
+
+  try {
+    return await axios.request( options );
+  } catch (error) {
+    let response = error.response || {data: null}
+    throw {...error.toJSON(), data: response.data};
+  }
+};  // Ends da.delete_interview()
+
+// node -e 'require("./lib/docassemble/docassemble_api_REST.js").get_interview(...)'
+da.get_interview = async function ( timeout, { session = null, api_key = null }) {
+  /* Get a specific interview. */
+  let options = {
+    method: 'GET',
+    headers: { 'X-API-Key': api_key },
+    // Can't use `/api/user/interviews` that uses the API key to identify the user's
+    // interview sessions. The test may have signed in with an account
+    // that's different than the ALKiln dev account that sends this request.
+    url: `${ session_vars.get_da_server_url() }/api/user/interviews`,
+    params: { session },
+    timeout: timeout,
+  };
+
+  try {
+    return await axios.request( options );
+  } catch ( error ) {
+    let response = error.response || { data: null }
+    throw { ...error.toJSON(), data: response.data };
+  }
+};  // Ends da.get_interview()
+
+
 // node -e 'require("./lib/docassemble/docassemble_api_REST.js").__delete_projects_starting_with( "testingdocassembleALAutomatedTestingTestsmain", 1, 1 )'
 da.__delete_projects_starting_with = async function ( base_name, starting_num=1, ending_num ) {
   /** FOR DEVELOPMENT ONLY. Deletes a set of Projects that start with a shared