@suffolklitlab/alkiln 5.12.0 → 5.13.0-logs1

package/docs/decisions/pdfjs_version_2024_08_08.md

@@ -0,0 +1,18 @@
+# Leave pdfjs at its current version
+
+## Context and Problem Statement
+
+pdfjs has a security vulnerability (that does not affect us) where malicious PDFs can inject code into an unsuspecting project and cause havoc.
+
+Our users are downloading and manipulating their own PDFs. We will assume they're not putting malicious code into their PDFs.
+
+The update changes pdfjs .js files to be .mjs files. It might take some bother to configure our repo to use .mjs files.
+
+## Considered Options
+
+- Update pdfjs
+- Leave pdfjs at its current version, 3.2.146
+
+## Decision Outcome
+
+Leave pdfjs at its current version, 3.2.146. The vulnerability is irrelevant for our situation and we don't have bandwidth to set up the configurations right now.

package/docs/decisions/reports_and_logs_2024_08_11.md

@@ -0,0 +1,141 @@
+# Shape and behavior of the report and log objects
+
+## Context and scope
+
+We have reports and logs we show to the user in the console and as artifacts. We have logs we use for development. cucumber also has logs we want to include in the above until we are confident we are handling all cases ourselves.
+
+Right now, the system for logging is fractured and scattered - reports log to the console and save completely separately from other logs. Logs that are not from the report don't get saved anywhere. Some logs only get shown in debug mode, which can be frustrating and unhelpful.
+
+## Goals
+
+Overview:
+
+- Unify the logging and report systems
+- Clarify their flow
+- Make log information more visible
+
+Details:
+
+- Log objects have statuses, icons, log codes (e.g. ALK0105), contextual info (e.g. `setup`, `takedown`, `invalid answer`), timestamps, descriptive text, and data (like full error traces).
+- These are the broad purposes of various logs:
+1. Setup and takedown messages.
+2. Test progress (a compact way to see what's happening/happened in each test):
+
+   Test1: .............F--
+   
+   Test 2: ..
+3. A pretty report at the end with details about each test. It has sections, like "failed" and "passed", headings, etc.
+4. cucumber's various console logs
+5. Lots of other informational logs for internal development and debugging purposes.
+I want to do different things with different logs:
+- During the tests, always log items 1, 2, 3, and 4 to the console.
+- During the tests, save items 2, 3, and 4 to a log as they come with all the info in the log objects - `report_log.txt`/`running_report.txt`/`report_run.txt` maybe. If we have to stop the tests early because of an infinite loop or something, this will at least show some information.
+- At the end of the tests, format items 2, 3, and 4, often with only part of the log object info, in a file called `report.txt`.
+- During the tests, save all of the items to a `verbose_log.txt` as they happen.
+- In debug mode, log all that stuff to the console while the tests are running.
+Right now, a few of the logs, like for the test progress (item 2), are printed with `stdout` inline while all others are logged with `console.log`.
+- Logs should accept multiple arguments to match console.log/etc. behavior.
+
+Note: The progress logs printed to the console have color codes. We also may want a plain text version without any colors to save in the report log and show in the Playground[^1] "console" and/or a version converted to HTML that does have styles. The latter may be for something easier to digest in a browser and/or to show visually in the Playground version of the tests.
+
+Questions:
+- Should `log` throw errors? Otherwise everywhere we need to error, we need to use `log` and `throw`, but I am unable to find a way to remove the current function from an error stack/trace.
+- How do we capture "errors" that are more subtle? Keep the word "error" in the "context"/"types" list? For example, expected errors:
+  - Invalid answers on the form that are then assessed as valid behavior?
+  - `AbortError`s of promises that should indeed have been aborted.
+- Should we allow "in-between"s for all parts of the log? All together: pre-everything (new lines, dividers, etc), icon/context/etc., pre-logs, logs (the actual messages to print), pre-data, data, post-data? We can default to nothing for all of those.
+- How do we differentiate between methods that log to the console vs. methods that just store data?
+
+[^1]: Don't go down this rabbit hole. Broadly - with the "Playground" version, users see the results as a web page instead of in the GitHub job console or in the artifacts. The users run the tests on their own server. This has to do with the platform we built ALKiln for - [docassemble](https://docassemble.org/). It gives users a faster iteration cycle.
+
+## Non/anti-goals
+
+- Spend a month on this
+- Create a complex and heavy system to try to abstract absolutely everything
+
+## The actual design
+
+This design still seems a little complex and potentially too coupled with reports.
+
+### System-context diagram
+
+```
+logic code -> logger -> report -----------> console
+              ^     |-> console         |-> stdout/progress
+  cucumber ---|     |-> running report  |-> final report.txt
+                    |-> verbose log
+```
+
+```
+logic code -> logger ------> report ------> final report.txt
+              ^     |-> console
+  cucumber ---|     |-> stdout/inline/progress
+                    |-> running report
+                    |-> verbose log
+```
+
+```
+              |------------------------------------<|
+              v                                     |
+logic code -> logger -> report -----------> format prettily
+              ^     |-> console
+  cucumber ---|     |-> running report
+                    |-> verbose log
+                    |-> stdout/progress
+                    |-> final report.txt
+```
+
+### APIs
+
+Everything saves to the "verbose"/"debug" log. Aside from that:
+
+`log.info/warning/success/error` methods log to console. This includes, e.g, setup, takedown which show up in GitHub workflow job logs.
+
+`log.report` saves to the running report then uses a report obj focused on cosmetics that has `.heading/table/header/error/etc` methods. Maybe a `.stdout/inlne` method, which then prints to the console immediately.
+
+`log.verbose/debug` (name?) - Only save to verbose/debug log with log codes, etc.
+
+### Data storage
+
+- `report.txt` - final pretty output for the user with headings, etc.
+- Each Scenario's `report.txt` - pretty output in each test's folder.
+- `running_report_log.txt` (name?) - ugly, but still sparse, giving the user some info about what happened during the tests without overwhelming them.
+- `verbose.txt` (name? `debug`?) - very ugly and full of all the information from everywhere else and more. Useful for internal development and troubleshooting. Includes setup and takedown.
+
+It may be complex to store setup and takedown output in the verbose/debug file. `console.Console` cannot append to a file[^2], so we would have to figure out a work-around. One approach could be to always first copy from the existing verbose file and paste those contents back into the file as the first log. That may take setting up some fallbacks to avoid errors.
+
+[^2]: `fs.appendFile()` only takes strings and doesn't log exactly like the console. I want to log exactly like the console to make it easy to search for words I and others see in the console output.
+
+### Code and pseudo-code
+
+For capturing console output, look into how to get the `stdin` for capturing various console output. https://stackoverflow.com/a/54202970:
+
+```js
+const stdin = process.openStdin()
+
+process.stdout.write('Enter name: ')
+
+stdin.addListener('data', text => {
+  const name = text.toString().trim()
+  console.log('Your name is: ' + name)
+
+  stdin.pause() // stop reading
+})
+```
+
+### Degree of constraint
+
+I think this can basically be greenfield (designed from scratch). We do have a current system, but I think we can easily convert our existing code to a new system.
+
+## Alternatives considered
+
+- The `report` object saves to the running report and then sends that same information to the `log` object. Also prints the final report. This includes `stdout` logs. `log` remains otherwise the same as described above.
+- `log` handles `stdout` logs and leaves them out of the report.
+- For report content, `log` would call to `report`, which would do pretty formatting and then hand the format text back to `log` to be saved and/or printed to the console. I'm not sure how `log` could then know what to do with the output - leave out the codes/etc. when it logs to the console and/or saves to the final report.txt.
+- Avoid the complexity of storing setup and takedown output in the verbose/debug file. In GitHub, they're in the GitHub console. Locally, we see the results immediately. On the other hand, storing them ourselves lets us look for information in just one place.
+
+## Cross-cutting concerns
+
+_Security, privacy, observability, etc._
+
+The same ones we've already dealt with for our current logs.

package/docs/decisions/update_puppeteer_2024_07_28.md

@@ -0,0 +1,62 @@
+# Use response data to detect sign-in success
+
+## Context and Problem Statement
+
+Should we update to the newest version of puppeteer, [22.15.0](https://github.com/puppeteer/puppeteer/blob/main/packages/puppeteer-core/CHANGELOG.md#22150-2024-07-31)?
+
+See https://github.com/SuffolkLITLab/ALKiln/issues/930.
+
+## Considered Options
+
+- Update to 22.15.0
+- Update to 22.12.0, which is the lowest version that would fix our current problems
+- Stay with 20.8.2
+
+See pros and cons
+
+## Decision Outcome
+
+Update puppeteer from 20.8.2 to 22.15.0
+
+## Pros and Cons of the Options
+
+### Update at all {#update}
+
+**Pros:**
+
+- Makes the current fix possible
+
+**Cons:**
+
+- Change in the middle of a complex PR
+- [A few breaking changes](https://github.com/puppeteer/puppeteer/blob/main/packages/puppeteer-core/CHANGELOG.md#2200-2024-02-05), which includes for us at the very least:
+   - "Removes the deprecated `$x` (replace with `$$`) and `waitForXpath` (replace with `waitForSelector`)." We need to add [extra syntax to the start of our selector strings](https://github.com/puppeteer/puppeteer/pull/11782): "xpath//."
+   - Replace any `page.waitForTimeout` with cucumber's version or ours.
+
+### Update to to 22.15.0
+
+**Pros:**
+
+- Has all the most recent bug fixes and features
+
+**Cons:**
+
+- Exposes us to possible unknown new bugs
+
+
+### Update to to 22.12.0
+
+This is the lowest version that would fix our problem
+
+**Pros:**
+
+- Avoids possibly unknown new bugs
+
+**Cons:**
+
+- Doesn't have fixes to already known bugs
+
+
+### Stay with 20.8.2
+
+Opposite of first entry ["Update at all"](#update).

package/docs/story_tables.md

@@ -4,7 +4,7 @@ The Story Table is a very flexible tool. It lets the author answer interview (fo
 
 When the robot arrives on each interview page, it finds all the fields on the page and loops through the whole Story Table trying to match rows with fields.
 
-Here's some [documentation on the use of Story Table rows](https://suffolklitlab.org/docassemble-AssemblyLine-documentation/docs/alkiln/#story-tables).
+Here's some [documentation on the use of Story Table rows](https://suffolklitlab.org/docassemble-AssemblyLine-documentation/docs/alkiln/writing#story-table).
 
 ## Tracking use of rows
 

package/lib/docassemble/docassemble_api_REST.js

@@ -31,7 +31,28 @@ da.get_dev_id = async function ( timeout ) {
 };  // Ends da.get_dev_id()
 
 
-// node -e 'require("./lib/docassemble/docassemble_api_REST.js").delete_project( "create1" )'
+da.get_api_key_info = async function ( timeout, { api_key = null, server_url = null }) {
+  /** Get information about an API key owned by the user of the given API key. */
+  let options = {
+    method: 'GET',
+    headers: {'X-API-Key': api_key},
+    url: `${ server_url }/api/user/api`,
+    params: {
+      api_key: api_key,
+    },
+    timeout: timeout,
+  };
+
+  try {
+    return await axios.request( options );
+  } catch (error) {
+    let response = error.response || { data: null }
+    throw {...error.toJSON(), data: response.data};
+  }
+};  // Ends da.get_api_key_info()
+
+
+// node -e 'require("./lib/docassemble/docassemble_api_REST.js").delete_project()'
 da.delete_project = async function ( timeout ) {
   /* Delete a project on the user's da server. */
   let options = {
@@ -53,6 +74,51 @@ da.delete_project = async function ( timeout ) {
 };  // Ends da.delete_project()
 
 
+// node -e 'require("./lib/docassemble/docassemble_api_REST.js").delete_interview(...)'
+da.delete_interview = async function ( timeout, { session = null, api_key = null }) {
+  /* Delete an interview with the given session id. */
+  let options = {
+    method: 'DELETE',
+    headers: { 'X-API-Key': api_key },
+    // Can't use `api/session` that uses the API key to identify the user's
+    // interview sessions. The test may have signed in with an account
+    // that's different than the ALKiln dev account that sends this request.
+    url: `${ session_vars.get_da_server_url() }/api/user/interviews`,
+    params: { session },
+    timeout: timeout,
+  };
+
+  try {
+    return await axios.request( options );
+  } catch (error) {
+    let response = error.response || {data: null}
+    throw {...error.toJSON(), data: response.data};
+  }
+};  // Ends da.delete_interview()
+
+// node -e 'require("./lib/docassemble/docassemble_api_REST.js").get_interview(...)'
+da.get_interview = async function ( timeout, { session = null, api_key = null }) {
+  /* Get a specific interview. */
+  let options = {
+    method: 'GET',
+    headers: { 'X-API-Key': api_key },
+    // Can't use `/api/user/interviews` that uses the API key to identify the user's
+    // interview sessions. The test may have signed in with an account
+    // that's different than the ALKiln dev account that sends this request.
+    url: `${ session_vars.get_da_server_url() }/api/user/interviews`,
+    params: { session },
+    timeout: timeout,
+  };
+
+  try {
+    return await axios.request( options );
+  } catch ( error ) {
+    let response = error.response || { data: null }
+    throw { ...error.toJSON(), data: response.data };
+  }
+};  // Ends da.get_interview()
+
+
 // node -e 'require("./lib/docassemble/docassemble_api_REST.js").__delete_projects_starting_with( "testingdocassembleALAutomatedTestingTestsmain", 1, 1 )'
 da.__delete_projects_starting_with = async function ( base_name, starting_num=1, ending_num ) {
   /** FOR DEVELOPMENT ONLY. Deletes a set of Projects that start with a shared

package/lib/docassemble/docassemble_api_interface.js

@@ -1,5 +1,5 @@
 const _da_REST = require('./docassemble_api_REST');
-const log = require('../utils/log');
+const Log = require('../utils/log');
 const session_vars = require('../utils/session_vars' );
 const time = require('../utils/time');
 
@@ -15,6 +15,9 @@ const time = require('../utils/time');
 // to specific functions, which can run in a very small
 // window.
 
+// TODO: Move get_dev_id() and such into here instead of in
+// docassemble_REST - args should be the same vals as the request expects
+
 /** Notes:
 * - Checking for a busy server is vulnerable to race conditions, but
 *    there's nothing we can do about that.
@@ -58,6 +61,10 @@ const time = require('../utils/time');
 let da_i = {};
 module.exports = da_i;
 
+const artifacts_path = session_vars.get_artifacts_path_name();
+const log = new Log({ path: artifacts_path, context: `api` });
+const context = `da api`;
+
 
 // Constants
 const DEFAULT_MAX_REQUEST_MS = session_vars.get_setup_timeout_ms();
@@ -89,7 +96,9 @@ da_i.throw_an_error_if_server_is_not_responding = async function ( options ) {
 
   // Defaults
   let { timeout } = options || { timeout: DEFAULT_MAX_REQUEST_MS };
-  log.info({ type: `da api`, code: `ALK0166`, pre: `Waiting for the docassemble server to respond. Will wait for ${ timeout/1000 } seconds at most.` });
+  log.info({ code: `ALK0166`, context: `da api`, },
+    `Waiting for the docassemble server to respond. Will wait for ${ timeout/1000 } seconds at most.`
+  );
   
   try {
     let start_time = Date.now()
@@ -97,7 +106,9 @@ da_i.throw_an_error_if_server_is_not_responding = async function ( options ) {
     let response = await _da_REST.get_dev_id( timeout );
     let end_time = Date.now()
     if ( response && response.data && response.data.id ) {
-      log.info({ type: `da api`, code: `ALK0167`, pre: `The docassemble server responded after ${ (end_time - start_time)/1000 } seconds.` });
+      log.info({ code: `ALK0167`, context: `da api`, },
+        `The docassemble server responded after ${ (end_time - start_time)/1000 } seconds.`
+      );
       return false;
     }
 
@@ -106,17 +117,16 @@ da_i.throw_an_error_if_server_is_not_responding = async function ( options ) {
     if ( is_timeout_error( error ) ) {
       // Anything that's waiting for the server to not be busy does not want
       // to move forward without a free server.
-      log.info({ type: `da api`, code: `ALK0168`, pre: `The docassemble server was still busy after ${ timeout/1000 } seconds.` });
+      log.info({ code: `ALK0168`, context: `da api`, },
+        `The docassemble server was still busy after ${ timeout/1000 } seconds.`
+      );
     } else {
       // Not sure what to do for other errors. It could be a server
       // error, like an 'Access Denied' error, which should be handled.
-      log.error({
-        type: `da api`,
-        code: `ALK0169`,
-        pre: `Tried to check if the docassemble server was busy, but `
-        + `ran into an unexpected error. The error didn't indicate a busy server, though. Error message:\n`
-        + error
-      });
+      log.warning({ code: `ALK0169`, context: `da api` },
+        `Checked if the server was busy and ran into a different error`,
+        error
+      );
     }
 
     throw error;
@@ -174,25 +184,222 @@ da_i.get_dev_id = async function ( dev_id_options ) {
   return dev_id;
 };  // Ends da_i.get_dev_id();
 
+da_i.get_api_key_access_status = async function ({ timeout = null, api_key = null }) {
+  /** Check if the API key has minimal authorization on the server.
+  *    See https://docassemble.org/docs/api.html#api_user_api_get.
+  *
+  * @param options {obj} Options for getting authorization of the API key
+  * @param options.api_key {str | null} Required. The API key to test.
+  * @param options.timeout {int | null} Optional. Max MS
+  *    to allow for server request.
+  */
+
+  let auth_data = {
+    codes: [ 1 ],
+    has_auth: null,
+    messages: [`API key authorization is not yet determined.`],
+    error: null
+  }
+
+  // Skip this for tests run from ALKilnInThePlayground
+  if ( session_vars.get_origin() === 'playground' ) {
+    auth_data.codes.push( 2 );
+    auth_data.messages.push( `API keys are unnecessary for tests run in the Playground.` );
+  }
+
+  if ( typeof api_key !== `string` ) {
+    auth_data.codes.push( 3 );
+    auth_data.messages.push( `The given API key is not a string. Type: ${ typeof api_key }`);
+  }
+
+  if ( auth_data.codes.length === 1 ) {
+    // Default timeouts
+    timeout = timeout || DEFAULT_MAX_REQUEST_MS;
+    try {
+      let response = await _da_REST.get_api_key_info( timeout, {
+        api_key,
+        server_url: session_vars.get_da_server_url(),
+      });
+      auth_data.has_auth = true;
+      auth_data.codes = [ 6 ];
+      auth_data.messages = [ `This API key has at least basic authorization on this server.` ];
+
+    } catch ( error ) {
+      auth_data.error = error;
+      if ( error.status === 403 ) {
+        auth_data.has_auth = false;
+        auth_data.codes = [ 4 ];
+        auth_data.messages = [ `This API is unverified on this server. It is probably missing or has been deleted.` ];
+      } else {
+        // Note on 404, non-existent key: 404 == 403 - Since we're using the
+        // key itself to get its own info, if it doesn't exist, we'll get 403
+        auth_data.codes = [ 5 ];
+        auth_data.messages = [ `This API key threw a non-403 error.` ];
+      }  // ends error status
+    }  // ends try
+  }  // ends auth_data.codes only has the first code
+
+  // log.debug({ type: `da api info`, code: `ALKx0202`,
+  //   pre: `API key authentication info`, data: auth_data });
+  log.debug({ code: `ALK0202`, context: `da api`, },
+    `API key authentication info`,
+    auth_data
+  );
+
+  return auth_data;
+};  // Ends da_i.get_api_key_access_status()
+
 // node -e 'require("./lib/docassemble/docassemble_api_interface.js").delete_project( "create1" )'
 da_i.delete_project = async function ( delete_options ) {
   /** Delete a project on the user's da server. Do we need to watch
   *   for a timeout? */
 
   // Defaults
-  let { timeout } = delete_options ||  { timeout: DEFAULT_MAX_REQUEST_MS };
+  let { timeout } = delete_options || { timeout: DEFAULT_MAX_REQUEST_MS };
 
   let project_name = session_vars.get_project_name()
-  log.info({ type: `da api`, code: `ALK0170`, pre: `Trying to delete Project ${ project_name }` });
+  log.info({ code: `ALK0170`, context: `da api`, },
+    `Trying to delete Project ${ project_name }`
+  );
 
   try {
     let response = await _da_REST.delete_project( timeout );
-    log.info({ type: `da api`, code: `ALK0171`, pre: `Deleted Project "${ project_name }"` });
+    log.info({ code: `ALK0171`, context: `da api`, },
+      `Deleted Project "${ project_name }"`
+    );
   } catch ( error ) {
-    let msg = `Ran into an error when deleting Project "${ project_name }". See https://docassemble.org/docs/api.html#playground_delete_project.`
-    log.error({ type: `da api`, code: `ALK0172`, pre: msg });
-    log.debug({ type: `da api`, code: `ALK0173`, pre: msg });
-    throw error;
+    log.throw({ code: `ALK0172`, context: `da api`, error: error },
+      `Ran into an error when deleting Project "${ project_name }". See https://docassemble.org/docs/api.html#playground_delete_project.`
+    );
+    // throw error;
   }
 
 };  // Ends da_i.delete_project()
+
+// node -e 'require("./lib/docassemble/docassemble_api_interface.js").quietly_try_to_delete_interviews(...)'
+da_i.quietly_try_to_delete_interviews = async function ({ interviews = {}, api_keys = [], options = {} }) {
+  /** Try to delete the given interviews. Fail silently. Example failure: the test
+   *    did not log in any users and the testing account's role prevents
+   *    access to other user's interviews. Do we need to watch for a timeout? */
+
+  // Defaults
+  let { timeout } = options || { timeout: DEFAULT_MAX_REQUEST_MS };
+
+  let responses = [];
+  for ( let id in interviews ) {
+    let interview = interviews[ id ];
+    if ( interview && interview.uid ) {
+      let sub_responses = await da_i.quietly_delete_one_interview( timeout, {
+        interview,
+        api_keys
+      });
+      responses.push( sub_responses );
+    } else {
+      log.debug({ code: `ALK0191`, context: `da api` },
+        `There is no interview to delete.`
+      );
+    }
+  }
+  return responses;
+};  // Ends da_i.quietly_try_to_delete_interviews()
+
+// node -e 'require("./lib/docassemble/docassemble_api_interface.js").quietly_delete_one_interview(...)'
+da_i.quietly_delete_one_interview = async function ( timeout, { interview = {}, api_keys = [] }) {
+  /** Try to delete an interview using all available API keys. Fail silently.
+   *    Example failure: the test did not log in any users and the testing
+   *    account's role prevents access to other user's interviews. */
+
+  let responses = [];
+  for ( let api_key of api_keys ) {
+    try {
+      log.debug({ code: `ALK0188`, context: `da api` },
+        `Trying to delete the interview with the id ${ interview.uid } and an API key.`
+      );
+
+      // Actual work:
+      let response = await _da_REST.delete_interview( timeout, {
+        session: interview.uid,
+        api_key
+      });
+      responses.push( response );
+
+      log.debug({ code: `ALK0189`, context: `da api` },
+        `No error when trying to delete interview ${ interview.uid } at ${ interview.i }.`
+      );
+    } catch ( error ) {
+      error.interview = interview;
+      log.debug({ code: `ALK0190`, context: `da api` },
+        `Error when deleting the interview. See https://docassemble.org/docs/api.html#session_delete.`,
+        error
+      );
+    }
+  }  // ends for api keys
+  return responses;
+};  // Ends da_i.quietly_delete_one_interview()
+
+// node -e 'require("./lib/docassemble/docassemble_api_interface.js").get_interviews(...)'
+da_i.get_interviews = async function ({ interviews = {}, api_keys = [], options = {} }) {
+  /** Return an object of interviews that exist on the server
+   *    that match the given interviews. Keys are the session ids. */
+
+  let fetched_interviews = {};
+  if ( interviews ) {
+    for ( let id in interviews ) {
+      let interview = interviews[ id ];
+      let fetched = await da_i.get_interview({ interview, api_keys, options });
+      if ( fetched && fetched.session ) {
+        fetched_interviews[ fetched.session ] = fetched;
+      }
+    }
+  }  // ends if interviews
+
+  return fetched_interviews;
+};  // Ends da_i.get_interviews()
+
+// node -e 'require("./lib/docassemble/docassemble_api_interface.js").get_interview(...)'
+da_i.get_interview = async function ({ interview = {}, api_keys = [], options = {} }) {
+  /** Try to get an interview from the server that matches the given interview data. */
+
+  // Defaults
+  let { timeout } = options || { timeout: DEFAULT_MAX_REQUEST_MS };
+
+  if ( interview && interview.uid ) {
+    for ( let api_key of api_keys ) {
+      try {
+        log.debug({ code: `ALK0192`, context: `da api` },
+          `Trying to get the interview with the id ${ interview.uid } and an API key.`
+        );
+
+        // Actual work:
+        let response = await _da_REST.get_interview( timeout, {
+          session: interview.uid,
+          api_key
+        });
+
+        if ( response && response.data && response.data.items && response.data.items[0] && response.data.items[0].session ) {
+          log.debug({ code: `ALK0193`, context: `da api` },
+            `Status ${ response.status }. Got ${ response.data.items[0].session }.`
+          );
+          return response.data.items[0];
+        } else {
+          log.debug({ code: `ALK0194`, context: `da api` },
+            `Status ${ response.status }. Cannot find ${ interview.uid }`
+          );
+        }
+
+      } catch ( error ) {
+        error.interview = interview;
+        log.debug({ code: `ALK0195`, context: `da api` },
+          `${ error.status || `No error.status.` } ${ error.statusText || error.data || `No error.statusText.` } Ran into an error when getting the interview ${ interview.uid }. See https://docassemble.org/docs/api.html#interviews.`,
+          error
+        );
+      }
+    }  // ends for api_keys
+  } else {
+    log.debug({ code: `ALK0196`, context: `da api` },
+      `There is no interview to get.`
+    );
+  }
+
+  return null;
+};  // Ends da_i.get_interview()

package/lib/docassemble/server_install.js

@@ -1,10 +1,11 @@
 #!/usr/bin/env node
 
 const session_vars = require('../utils/session_vars');
-const log = require('../utils/log');
+const Log = require('../utils/log');
 const fg = require('fast-glob');
 
 const server_install = async () => {
+  /** Finds out where in this directory the package's folders are. */
   session_vars.save_install_method(`server`);
 
   // Get path to folders inside repo main "docassemble" folder
@@ -23,20 +24,33 @@ const server_install = async () => {
   // Get names of all folders in repo "docassemble" folder
   let chosen_folder = folders[0];
   if ( folders.length === 0 ) {
-    let msg = `There are 0 folders at ${path}. Stopping tests.`;
-    log.error({ type: `setup`, code: `ALK0044`, data: msg })
-    throw new ReferenceError( msg );
+    let msg = `ALKiln cannot find the package's files. There are 0 folders at "${ path }". Stopping tests.`;
+    log.throw({ code: `ALK0044`, context: `server_install`,
+      error: new ReferenceError( msg );  // no logs
+    });
   }
 
   if ( folders.length > 1 ) {
-    log.warn({
-      type: `setup`,
-      code: `ALK0045`,
-      data: `There are ${folders.length} folders at ${path}: "${folders.join('", "')}". ALKiln chose to use the folder "${chosen_folder}".`
-    })
+    log.warn({ code: `ALK0045`, context: `setup`, },
+      `There are ${ folders.length } folders at "${ path }": "${ folders.join('", "') }". ALKiln chose to use the folder "${ chosen_folder }".`
+    );
+  } else {
+    log.info({ code: `ALK0218`, context: `setup`, },
+      `ALKiln found the package folder at "${ chosen_folder }".`
+    );
   }
   
   session_vars.save_repo_folder_name( chosen_folder );
 };  // Ends server_install();
 
+// 1. session_vars - make sure all the required env vars exist
+session_vars.validateEnvironment();
+
+// 2. Prepare logs
+// Local devs: This will be separate from your test paths
+// unless you pass a `path` argument into the cucumber run command
+const argv = require(`minimist`)(process.argv.slice(2));
+const log = new Log({ path: argv.path, context: `server_install` });
+
+// 3. run
 server_install();