@suffolklitlab/alkiln 5.13.3 → 5.13.4

package/docs/decisions/reports_and_logs_2024_08_11.md

@@ -0,0 +1,161 @@
+# Shape and behavior of the report and log objects
+
+<!-- TODO: turn this into a architecture design doc? -->
+<!-- TODO: Rename to just "logs" -->
+
+## Context and scope
+
+We show logs to the user in the console and as test suite artifacts. We have logs we use for development. cucumberjs also has logs we want to include in the above until we are confident we are handling all cases ourselves.
+
+Right now, the system for logging is fractured, scattered, and duplicated. Logs that are not from the report don't get saved anywhere. Some logs only get shown in debug mode in the console, which can be frustrating and unhelpful. We lack verbose output information for tests that authors run in the wild.
+
+This document excludes discussion of report architecture, but it is useful to note that at the moment they are completely separate from logs and yet save to some of the same files, so they influence our current choices.
+
+## Goals
+
+- Make log information persistent and visible
+- Add additional meaningful information to logs (e.g. metadata)
+- Make sure logs are clear without needing to read the surrounding code
+- Unify and clarify log flow
+- Make log behavior consistent
+
+## Non/anti-goals
+
+- Disrupt the signals we're giving authors about their tests with the noise of internal ALKiln details
+- Create a complex and heavy system to try to abstract absolutely everything
+- Let the complexity of the relationship with report behavior too heavily influencing our log design goals
+
+## The actual design
+
+### `Log` internal errors
+
+Logs are informational and not related to the behavior of authors' tests. Because of that, these logs should avoid adding noise to what an author sees. That is, internal warnings or errors should look different than authors' test errors. To help with this:
+
+- All log methods` arguments have default values.
+- We use try/catch very aggressively in the `Log` class.
+
+Read more of the thoughts behind this choice in the decision doc on which log level to pick and the design doc on types of errors.
+
+### Purposes of methods
+
+Some of these notes have to do with reports because logs and reports are using some of the same files at the moment and yet are not integrated. We'll write a doc for reports when we integrate reports more fully with logs.
+
+1. `Log`: Messages or errors from wrapping functionality, like setup and takedown messages.
+2. `Log`: Test progress (a compact way to see what's happening/happened in each test):
+
+   Test1: .............F--.
+
+   Test 2: ....
+3. Report: A pretty report at the end with details about each test. It has sections, like "failed" and "passed", headings, etc.
+4. Report: cucumberjs's various console logs.
+5. `Log`: Lots of other informational logs for internal development and debugging purposes.
+
+Different logs belong in different places:
+
+- ALKiln always logs items 1, 2, 3, and 4 to the console.
+- Ideally save all items in a verbose debug file sequentially as the tests run.
+- During the tests, save items 2, 3, and 4 to a running report file sequentially as they come. Include metadata. Suggested filenames: `report_log.txt`/`running_report.txt`/`report_run.txt`. If we have to stop the tests early because of an infinite loop or something, this will at least show the test author some information.
+- At the end of the tests, format items 1, 2, 3, and 4 nicely in a file called `report.txt`. This will mostly exclude the metadata.
+- When there is a warning-level or error-level log, save 1, 3, 4, and 5 in an unexpected results file too.
+- In future, discuss the usefulness of having a debug mode where ALKiln logs all items to the console while the tests are running.
+
+We may also want copies of various files that have different formats for different environments (see docs on the different environments in which ALKiln runs).
+
+- Console logs with colors.
+- Plain text versions that exclude those color codes.
+- HTML versions to show with, for example ALKilnInThePlayground. Maybe in GitHub as well in the future.
+
+### Anatomy and behavior of logging methods
+
+Most of a `Log`'s logging methods add additional metadata to a log:
+
+1. The level of the log (success, info, warning, error, and custom values)
+2. An icon showing the level of a log
+3. A log code, like (e.g. ALK0105)
+4. Contextual info about where the log originated (e.g. "setup", "takedown", "invalid answer")
+9. A timestamp
+
+To be able to show that metadata, most methods will accept custom values for parts of that metadata. Most methods offer:
+
+- Log code
+- Contextual info
+- The error throwing information
+
+For the debug method, everything other than the timestamp can be customized.
+
+Callers can influence the content and formatting of most parts of the log message. For example, they can call `.warn` to show a warning icon and level metadata. The one exception would usually be the part before the metadata. Customizing that could be useful for things like creating a visual separator before ALKiln starts evaluating the fields of a web page. For that reason, the caller can add a `before` property to the metadata that a `Log` object will add before the metadata.
+
+A `Log` object should be able to throw errors so that it can log metadata and any additional styling and context along with the error. It can also make sure that the error will be an actual `Error` object (as opposed to a string or other object). Otherwise everywhere we needed to throw an error, we would need duplicate handling bad data (like objects with circular references), using `Log` to save to files (etc.), throwing the error, and other possible complications. For that, `Logs` also need to accept this additional information:
+
+- Whether to throw an error
+- The error to throw
+
+Those methods should also accept multiple individual log arguments the same way `console.log()` does. This lets the caller include strings separately from objects they want to log, including `Error` objects, so that a `Log` object can handle them safely and style them consistently. We could have required a list of logs instead, but the extra syntax was frustrating.
+
+Right now, the only way we differentiate methods that just store data vs. methods that also log to the console is with documentation. This seems less than ideal. Using the name `record` could be confusing because developers might think they need to log to the console separately from saving the information to debug files.
+
+## APIs
+
+Proposed external methods: `success`, `info`, `warn`, `throw`, `stdout`, `debug`. See the `Log` class and its tests for the most up-to-date API.
+
+<!-- Discuss: better name for `stdout`? -->
+
+## Data storage
+
+- `report.txt` - final pretty output for the user with headings, etc.
+- Each Scenario's `report.txt` - pretty output in each test's folder.
+- `report_log.txt` (name?) - ugly, but still sparse, giving the user some info about what happened during the tests without overwhelming them.
+- `debug_log.txt` (name? `debug`?) - very ugly and full of all the information from everywhere else and more. Useful for internal development and troubleshooting. Includes setup, takedown, etc.
+- `temp_unexpected_results_debug_log.txt` - A file containing messages about unexpected behavior, like test warnings and errors, as well as internal ALKiln warnings. We don't output this in our GitHub actions - the file itself is temporary, we just add its contents to the final unexpected results file. All its information exists in the debug file too.
+- `unexpected_results.txt` - Logs for failures and other unexpected behavior. It has information about test warnings and errors, as well as internal ALKiln warnings. At first we only save report information there. When the tests finish, we add the contents of the specific debug file that contains only unexpected results.
+
+## Degree of constraint
+
+- Right now we're a bit limited by report functionality staying separate. Other than that, we can design as we want. We should avoid the current behavior of reports substantially influencing our design decisions here. We do have to keep in mind how reports might integrate with this in the future, but we will also need to change the design of the report behavior significantly in the future, so we should keep an open mind.
+- We have not yet approached saving logs from GitHub actions, though this should be fairly simple. It may add some complication.
+- We have not yet solved getting all logs from the console (e.g. cucumberjs logs).
+
+## Alternatives considered
+
+- Avoid the complexity of storing setup and takedown output in the verbose/debug file. In GitHub, they're in the GitHub console. Locally, we see the results immediately. On the other hand, storing them ourselves lets us look for information in just one place.
+- Include GitHub action output in the debug files. Out of scope for the current iteration.
+- Catch cucumberjs's actual console output. Out of scope for the current iteration.
+
+### System-context diagram
+
+Current design before integration of reports, GitHub action logs, and other non-ALKiln console logs.
+
+```
+## `Log`
+
+console methods -> ._console() -> .debug() -> log to the console
+.stdout() -> .debug() -> sometimes write to stdout
+
+.debug() -> debug_log.txt with metadata
+         -> sometimes temp_unexpected_results_debug_log.txt with metadata
+
+## `Report`
+report and its methods -> report_log.txt (plain text)
+                       -> debug_log.txt (plain text)
+                       -> sometimes unexpected_results.txt (plain text)
+                       -> report summary at end of tests
+                          -> final report.txt (plain text summary)
+                          -> final console log (summary with console colors)
+                       -> individual Scenario reports at end of tests
+                          -> final individual report summary file in each Scenario's folder
+
+## takedown
+Add temp_unexpected_results_debug_log.txt to the end of  unexpected_results.txt
+```
+
+## Cross-cutting concerns
+
+_Security, privacy, observability, etc._
+
+Code that calls `Log` methods should avoid passing sensitive information. A `Log` object doesn't protect against exposing information.
+
+## Related documentation
+
+Decision doc about which log level to use.
+Architecture design doc about types of unexpected and expected "errors".
+Decision doc about getting and storing logs.

package/docs/kinds_of_errors.md

@@ -0,0 +1,26 @@
+# Kinds of Errors
+
+An error, or even an `Error` instance, can mean a lot of different things in the context of this testing framework. Throwing errors for failing tests is what we are most familiar with, but it is just one kind of error.
+
+## Interview `Error`s
+
+1. **Interview bugs**: For example, the interview says ALKiln's answer is invalid. The author expected that answer to be valid. ALKiln saves an error message to the report and throws an error to tell cucumber to mark the test as failed. Currently, these logs errors come with the 🤕 icon. When all the tests are done, the test suite will show as failing in the console, the report, and the various logs. In future, ALKiln will use a method like `report.throw()` to save to the various log files with the `error` level and then throw the error to let cucumber know the test failed.
+1. **Interview expected errors**: For example, the interview says ALKiln's answer is invalid. The author was testing their interview's field validation rules. The test passes.
+
+## Test file `Error`s
+
+1. **Test file syntax error**: For example, a row in a table in a test file has too many columns. cucumberjs throws an error and no tests are run. ALKiln should show the tests as failed. ALKiln is unable to get the output of that error into a log, but there should be an error in the console.
+1. **Test Step typo**: For example, `And I ta p to contcontinue`. cucumberjs will cause this test to fail, giving an "undefined" Step as the reason. The test suite will show as failing in the console, the report, and the various logs.
+1. **Test bugs**: For example, a test has old values that need to be updated. The test fails. This will look like an interview bug, as this document describes.
+
+## ALKiln `Error`s
+
+1. **ALKiln un-handled bugs**: For example, we have forgotten to define a variable and none of our tests has yet triggered that code. These are unexpected errors that force processes to abort or force individual tests to fail in uninformative ways. To an author that runs into that error, it looks like their test has failed, but the fault is ours and we need to troubleshoot it. We should make that clear.
+1. **ALKiln silent bugs**: For example, a debug log has an object that causes an error when ALKiln tries to turn it into a string. ALKiln should record the failure somewhere, but should avoid throwing it. We can troubleshoot those if we see them and add new internal tests. See decisions docs on when to use which kind of log level.
+1. **ALKiln expected errors**: For example, one promise won a `Promise.race()`, so ALKiln aborted the other promises. That causes those aborted promises to error. That is why ALKiln saves the "informational" Error to the debug files and it has a level of "info" or "debug".
+
+## ALKiln's internal tests
+
+1. **ALKiln internal success-handling test failures**: For example, the test fails to fill out radio buttons correctly.
+1. **ALKiln internal failure-handling test successes**: For example, we test that the summary report shows a specific log code when an interview fails to load and the log code shows up correctly. The "test" fails - the interview fails to load - but *our* test passes - the right log code was in the summary report.
+1. **ALKiln internal failure-handling test failures**: For example, we test that the report shows a specific log code when an interview fails to load, but the log code is missing. The "test" fails and *our* test fails.

package/lib/docassemble/docassemble_api_interface.js

@@ -1,5 +1,5 @@
 const _da_REST = require('./docassemble_api_REST');
-const log = require('../utils/log');
+const Log = require('../utils/Log');
 const session_vars = require('../utils/session_vars' );
 const time = require('../utils/time');
 
@@ -61,6 +61,10 @@ const time = require('../utils/time');
 let da_i = {};
 module.exports = da_i;
 
+const context = `da api`;
+const artifacts_path = session_vars.get_artifacts_path_name();
+const log = new Log({ path: artifacts_path, context });
+
 
 // Constants
 const DEFAULT_MAX_REQUEST_MS = session_vars.get_setup_timeout_ms();
@@ -92,7 +96,9 @@ da_i.throw_an_error_if_server_is_not_responding = async function ( options ) {
 
   // Defaults
   let { timeout } = options || { timeout: DEFAULT_MAX_REQUEST_MS };
-  log.info({ type: `da api`, code: `ALK0166`, pre: `Waiting for the docassemble server to respond. Will wait for ${ timeout/1000 } seconds at most.` });
+  log.info({ code: `ALK0166`, context, },
+    `Waiting for the docassemble server to respond. Will wait for ${ timeout/1000 } seconds at most.`
+  );
   
   try {
     let start_time = Date.now()
@@ -100,7 +106,9 @@ da_i.throw_an_error_if_server_is_not_responding = async function ( options ) {
     let response = await _da_REST.get_dev_id( timeout );
     let end_time = Date.now()
     if ( response && response.data && response.data.id ) {
-      log.info({ type: `da api`, code: `ALK0167`, pre: `The docassemble server responded after ${ (end_time - start_time)/1000 } seconds.` });
+      log.info({ code: `ALK0167`, context, },
+        `The docassemble server responded after ${ (end_time - start_time)/1000 } seconds.`
+      );
       return false;
     }
 
@@ -109,17 +117,16 @@ da_i.throw_an_error_if_server_is_not_responding = async function ( options ) {
     if ( is_timeout_error( error ) ) {
       // Anything that's waiting for the server to not be busy does not want
       // to move forward without a free server.
-      log.info({ type: `da api`, code: `ALK0168`, pre: `The docassemble server was still busy after ${ timeout/1000 } seconds.` });
+      log.info({ code: `ALK0168`, context, },
+        `The docassemble server was still busy after ${ timeout/1000 } seconds.`
+      );
     } else {
       // Not sure what to do for other errors. It could be a server
       // error, like an 'Access Denied' error, which should be handled.
-      log.error({
-        type: `da api`,
-        code: `ALK0169`,
-        pre: `Tried to check if the docassemble server was busy, but `
-        + `ran into an unexpected error. The error didn't indicate a busy server, though. Error message:\n`
-        + error
-      });
+      log.warning({ code: `ALK0169`, context, },
+        `Checked if the server was busy and ran into a different error`,
+        error
+      );
     }
 
     throw error;
@@ -187,7 +194,7 @@ da_i.get_api_key_access_status = async function ({ timeout = null, api_key = nul
   *    to allow for server request.
   */
 
-  let auth_data = {
+  let auth_feedback = {
     codes: [ 1 ],
     has_auth: null,
     messages: [`API key authorization is not yet determined.`],
@@ -196,16 +203,16 @@ da_i.get_api_key_access_status = async function ({ timeout = null, api_key = nul
 
   // Skip this for tests run from ALKilnInThePlayground
   if ( session_vars.get_origin() === 'playground' ) {
-    auth_data.codes.push( 2 );
-    auth_data.messages.push( `API keys are unnecessary for tests run in the Playground.` );
+    auth_feedback.codes.push( 2 );
+    auth_feedback.messages.push( `API keys are unnecessary for tests run in the Playground.` );
   }
 
   if ( typeof api_key !== `string` ) {
-    auth_data.codes.push( 3 );
-    auth_data.messages.push( `The given API key is not a string. Type: ${ typeof api_key }`);
+    auth_feedback.codes.push( 3 );
+    auth_feedback.messages.push( `The given API key is not a string. Type: ${ typeof api_key }`);
   }
 
-  if ( auth_data.codes.length === 1 ) {
+  if ( auth_feedback.codes.length === 1 ) {
     // Default timeouts
     timeout = timeout || DEFAULT_MAX_REQUEST_MS;
     try {
@@ -213,29 +220,31 @@ da_i.get_api_key_access_status = async function ({ timeout = null, api_key = nul
         api_key,
         server_url: session_vars.get_da_server_url(),
       });
-      auth_data.has_auth = true;
-      auth_data.codes = [ 6 ];
-      auth_data.messages = [ `This API key has at least basic authorization on this server.` ];
+      auth_feedback.has_auth = true;
+      auth_feedback.codes = [ 6 ];
+      auth_feedback.messages = [ `This API key has at least basic authorization on this server.` ];
 
     } catch ( error ) {
-      auth_data.error = error;
+      auth_feedback.error = error;
       if ( error.status === 403 ) {
-        auth_data.has_auth = false;
-        auth_data.codes = [ 4 ];
-        auth_data.messages = [ `This API is unverified on this server. It is probably missing or has been deleted.` ];
+        auth_feedback.has_auth = false;
+        auth_feedback.codes = [ 4 ];
+        auth_feedback.messages = [ `This API is unverified on this server. It is probably missing or has been deleted.` ];
       } else {
         // Note on 404, non-existent key: 404 == 403 - Since we're using the
         // key itself to get its own info, if it doesn't exist, we'll get 403
-        auth_data.codes = [ 5 ];
-        auth_data.messages = [ `This API key threw a non-403 error.` ];
+        auth_feedback.codes = [ 5 ];
+        auth_feedback.messages = [ `This API key threw a non-403 error.` ];
       }  // ends error status
     }  // ends try
-  }  // ends auth_data.codes only has the first code
+  }  // ends auth_feedback.codes only has the first code
 
-  log.debug({ type: `da api info`, code: `ALK0202`,
-    pre: `API key authentication info`, data: auth_data });
+  log.debug({ code: `ALK0202`, context, },
+    `API key - what happened during authentication:`,
+    auth_feedback
+  );
 
-  return auth_data;
+  return auth_feedback;
 };  // Ends da_i.get_api_key_access_status()
 
 // node -e 'require("./lib/docassemble/docassemble_api_interface.js").delete_project( "create1" )'
@@ -247,16 +256,19 @@ da_i.delete_project = async function ( delete_options ) {
   let { timeout } = delete_options || { timeout: DEFAULT_MAX_REQUEST_MS };
 
   let project_name = session_vars.get_project_name()
-  log.info({ type: `da api`, code: `ALK0170`, pre: `Trying to delete Project ${ project_name }` });
+  log.info({ code: `ALK0170`, context, },
+    `Trying to delete Project ${ project_name }`
+  );
 
   try {
     let response = await _da_REST.delete_project( timeout );
-    log.info({ type: `da api`, code: `ALK0171`, pre: `Deleted Project "${ project_name }"` });
+    log.info({ code: `ALK0171`, context, },
+      `Deleted Project "${ project_name }"`
+    );
   } catch ( error ) {
-    let msg = `Ran into an error when deleting Project "${ project_name }". See https://docassemble.org/docs/api.html#playground_delete_project.`
-    log.error({ type: `da api`, code: `ALK0172`, pre: msg });
-    log.debug({ type: `da api`, code: `ALK0173`, pre: msg });
-    throw error;
+    log.throw({ code: `ALK0172`, context, error: error },
+      `Ran into an error when deleting Project "${ project_name }". See https://docassemble.org/docs/api.html#playground_delete_project.`
+    );
   }
 
 };  // Ends da_i.delete_project()
@@ -280,7 +292,9 @@ da_i.quietly_try_to_delete_interviews = async function ({ interviews = {}, api_k
       });
       responses.push( sub_responses );
     } else {
-      log.debug({ type: `da api info`, code: `ALK0191`, pre: `There is no interview to delete.` });
+      log.debug({ code: `ALK0191`, context, },
+        `There is no interview to delete.`
+      );
     }
   }
   return responses;
@@ -295,9 +309,9 @@ da_i.quietly_delete_one_interview = async function ( timeout, { interview = {},
   let responses = [];
   for ( let api_key of api_keys ) {
     try {
-      log.debug({ type: `da api info`, code: `ALK0188`,
-        pre: `Trying to delete the interview with the id ${ interview.uid } and an API key.`
-      });
+      log.debug({ code: `ALK0188`, context, },
+        `Trying to delete the interview with the id ${ interview.uid } and an API key.`
+      );
 
       // Actual work:
       let response = await _da_REST.delete_interview( timeout, {
@@ -306,13 +320,15 @@ da_i.quietly_delete_one_interview = async function ( timeout, { interview = {},
       });
       responses.push( response );
 
-      log.debug({ type: `da api info`, code: `ALK0189`,
-        pre: `No error when trying to delete interview ${ interview.uid } at ${ interview.i }.`
-      });
+      log.debug({ code: `ALK0189`, context, },
+        `No error when trying to delete interview ${ interview.uid } at ${ interview.i }.`
+      );
     } catch ( error ) {
       error.interview = interview;
-      let msg = `Error when deleting the interview. See https://docassemble.org/docs/api.html#session_delete.`
-      log.debug({ type: `da api info`, code: `ALK0190`, pre: msg, data: error });
+      log.debug({ code: `ALK0190`, context, },
+        `Error when deleting the interview. See https://docassemble.org/docs/api.html#session_delete.`,
+        error
+      );
     }
   }  // ends for api keys
   return responses;
@@ -347,7 +363,9 @@ da_i.get_interview = async function ({ interview = {}, api_keys = [], options =
   if ( interview && interview.uid ) {
     for ( let api_key of api_keys ) {
       try {
-        log.debug({ type: `da api info`, code: `ALK0192`, pre: `Trying to get the interview with the id ${ interview.uid } and an API key.` });
+        log.debug({ code: `ALK0192`, context, },
+          `Trying to get the interview with the id ${ interview.uid } and an API key.`
+        );
 
         // Actual work:
         let response = await _da_REST.get_interview( timeout, {
@@ -356,20 +374,28 @@ da_i.get_interview = async function ({ interview = {}, api_keys = [], options =
         });
 
         if ( response && response.data && response.data.items && response.data.items[0] && response.data.items[0].session ) {
-          log.debug({ type: `da api info`, code: `ALK0193`, pre: `Status ${ response.status }. Got ${ response.data.items[0].session }.`});
+          log.debug({ code: `ALK0193`, context, },
+            `Status ${ response.status }. Got ${ response.data.items[0].session }.`
+          );
           return response.data.items[0];
         } else {
-          log.debug({ type: `da api info`, code: `ALK0194`, pre: `Status ${ response.status }. Cannot find ${ interview.uid }`});
+          log.debug({ code: `ALK0194`, context, },
+            `Status ${ response.status }. Cannot find ${ interview.uid }`
+          );
         }
 
       } catch ( error ) {
         error.interview = interview;
-        let msg = `${ error.status || `No error.status.` } ${ error.statusText || error.data || `No error.statusText.` } Ran into an error when getting the interview ${ interview.uid }. See https://docassemble.org/docs/api.html#interviews.`
-        log.debug({ type: `da api info`, code: `ALK0195`, pre: msg, data: error });
+        log.debug({ code: `ALK0195`, context, },
+          `${ error.status || `No error.status.` } ${ error.statusText || error.data || `No error.statusText.` } Ran into an error when getting the interview ${ interview.uid }. See https://docassemble.org/docs/api.html#interviews.`,
+          error
+        );
       }
     }  // ends for api_keys
   } else {
-    log.debug({ type: `da api info`, code: `ALK0196`, pre: `There is no interview to get.` });
+    log.debug({ code: `ALK0196`, context, },
+      `There is no interview to get.`
+    );
   }
 
   return null;

package/lib/docassemble/server_install.js

@@ -1,10 +1,20 @@
 #!/usr/bin/env node
 
 const session_vars = require('../utils/session_vars');
-const log = require('../utils/log');
+const Log = require('../utils/Log');
 const fg = require('fast-glob');
 
+const context = `server_install`;
+
 const server_install = async () => {
+  /** For packages we are installing on a temporary GitHub docassemble server.
+   *
+   *  - Store that the package is being installed globally on a server as
+   *    opposed to on a specific author's account (their Playground).
+   *  - In the repo, find and store the path to directory that stores the
+   *    package itself instead of the whole repo. For example, EvictionDefense
+   *    in the docassemble-EvictionDefense repo.
+   * */
   session_vars.save_install_method(`server`);
 
   // Get path to folders inside repo main "docassemble" folder
@@ -23,20 +33,33 @@ const server_install = async () => {
   // Get names of all folders in repo "docassemble" folder
   let chosen_folder = folders[0];
   if ( folders.length === 0 ) {
-    let msg = `There are 0 folders at ${path}. Stopping tests.`;
-    log.error({ type: `setup`, code: `ALK0044`, data: msg })
-    throw new ReferenceError( msg );
+    let msg = `ALKiln cannot find the package's files. There are 0 folders at "${ path }". Stopping tests.`;
+    log.throw({ code: `ALK0044`, context,
+      error: new ReferenceError( msg )  // no logs
+    });
   }
 
   if ( folders.length > 1 ) {
-    log.warn({
-      type: `setup`,
-      code: `ALK0045`,
-      data: `There are ${folders.length} folders at ${path}: "${folders.join('", "')}". ALKiln chose to use the folder "${chosen_folder}".`
-    })
+    log.warn({ code: `ALK0045`, context, },
+      `There are ${ folders.length } folders at "${ path }": "${ folders.join('", "') }". ALKiln chose to use the folder "${ chosen_folder }".`
+    );
+  } else {
+    log.info({ code: `ALK0218`, context, },
+      `ALKiln found the package folder at "${ chosen_folder }".`
+    );
   }
   
   session_vars.save_repo_folder_name( chosen_folder );
 };  // Ends server_install();
 
+// 1. session_vars - make sure all the required env vars exist
+session_vars.validateEnvironment();
+
+// 2. Prepare logs
+// Local devs: This will be separate from your test paths
+// unless you pass a `path` argument into the cucumber run command
+const argv = require(`minimist`)(process.argv.slice(2));
+const log = new Log({ path: argv.path, context, });
+
+// 3. run
 server_install();

package/lib/docassemble/setup.js

@@ -1,26 +1,33 @@
 #!/usr/bin/env node
 
-// TODO: Rename to playground_install?
-
 const child_process = require(`child_process`);
 
-const log = require(`../utils/log`);
+const Log = require(`../utils/Log`);
 const session_vars = require(`../utils/session_vars` );
+const files = require(`../utils/files` );
+const time = require(`../utils/time` );
+
+const context = `setup`;
 
 const setup = async () => {
-  /** In the docassemble testing account identified by the
-  *   DOCASSEMBLE_DEVELOPER_API_KEY secret, create a new Project,
-  *   pull code into that project, and, if needed, wait for the
-  *   server to reload.
-  */
+  /** This is for a package being installed in an author's account (Playground)
+   *  on their docassemble server. Using the author's
+   *   DOCASSEMBLE_DEVELOPER_API_KEY secret, create a new Project, pull code
+   *   into that project, and, if needed, wait for the server to reload.
+   */
   
   // Make and save project name
   let project_name = session_vars.get_project_base_name().replace(/[^A-Za-z0-9]/g, '') + `${Date.now()}`;
   session_vars.save_project_name( project_name );
 
-  log.info({ type: `setup`, code: `ALK0046`, pre: `Starting to upload this package to `
+  const starting_msg = `Starting to upload this package to `
     + `a new Project called ${project_name} in the Playground `
-    + `of your server's testing account. It may take a couple of minutes.` });
+    + `of your server's testing account. It may take a couple of minutes.`;
+  log.info({ code: `ALK0046`, context, },
+    starting_msg
+  );
+
+  // TODO: 1. Start listening to the normal console here (for debug logging)
 
   // dainstall . --apiurl https://site.org --apikey $API_KEY --playground --project=name
   // I think this won't run on windows? Why do I think this? Where on the
@@ -34,29 +41,37 @@ const setup = async () => {
       `--playground`, `--project=${ project_name }`
     ],
     // Prints messages from the process to the console while it's running
+    // result.stderr, result.stdout, etc. will be `null`
     { stdio: 'inherit', }
   );
 
-  // 403 is invalid API key, but I'm not sure how to output
-  // the response to find out the error code
+  // TODO: 2. Stop listening to the normal console here
 
+  // 403 is invalid API key, but I'm not sure how to output
+  // the response to find out the error code. stderr is `null`.
   if ( result.status !== 0 ) {
-    log.error({
-      type: `setup`,
-      code: `ALK0047`,
-      pre: `Status code ${ result.status }. Could not dainstall this package on your server. See above for more details.`
-    });
-    throw(result.stderr);
+    const install_err_msg = `Status code ${ result.status }. Could not `
+      + `dainstall this package on your server. See above for more details.`;
+    log.throw({ code: `ALK0047`, context,
+      before: `☝️ ☝️ ☝️  Error data ☝️ ☝️ ☝️\n`,
+      error: `Error while installing project` },
+      install_err_msg
+    );
   }
 
-  log.success({
-    type: `setup`,
-    code: `ALK0048`,
-    pre: `ALKiln created a Project in your Playground named "${ project_name }"!`
-  });
+  log.success({ code: `ALK0048`, context, },
+    `ALKiln created a Project in your Playground named "${ project_name }"!`
+  );
 };  // Ends setup();
 
-
-// Before everything, make sure all the required env vars exist
+// 1. session_vars - make sure all the required env vars exist
 session_vars.validateEnvironment();
+
+// 2. Prepare logs
+// Local devs: This will be separate from your test paths
+// unless you pass a `path` argument into the cucumber run command
+const argv = require(`minimist`)(process.argv.slice(2));
+const log = new Log({ path: argv.path, context, });
+
+// 3. run
 setup();