@suffolklitlab/alkiln 4.0.0-pre-api.9 → 4.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/test_tests.yml +3 -0
- package/CHANGELOG.md +67 -12
- package/README.md +10 -2
- package/action.yml +3 -2
- package/lib/docassemble/docassemble_api_interface.js +1 -1
- package/lib/scope.js +78 -21
- package/lib/steps.js +21 -14
- package/lib/utils/langs.js +1 -1
- package/lib/utils/session_vars.js +2 -2
- package/package.json +2 -2
- package/tests/features/interactive_steps.feature +18 -0
- package/tests/features/reports.feature +23 -0
|
@@ -31,6 +31,9 @@ jobs:
|
|
|
31
31
|
REPO_URL: https://github.com/plocket/docassemble-ALAutomatedTestingTests
|
|
32
32
|
USER1_EMAIL: ${{ secrets.USER1_EMAIL }}
|
|
33
33
|
USER1_PASSWORD: ${{ secrets.USER1_PASSWORD }}
|
|
34
|
+
SECRET_VAR1: secret-var1-value
|
|
35
|
+
SECRET_VAR2: secret-var2-value
|
|
36
|
+
SECRET_FOR_MISSING_FIELD: secret for missing field
|
|
34
37
|
|
|
35
38
|
name: Run Puppeteer tests
|
|
36
39
|
steps:
|
package/CHANGELOG.md
CHANGED
|
@@ -38,12 +38,15 @@ Format:
|
|
|
38
38
|
### Security
|
|
39
39
|
-
|
|
40
40
|
-->
|
|
41
|
-
<!--
|
|
41
|
+
<!-- ## [Unreleased] -->
|
|
42
|
+
## [4.1.1] - 2022-04-01
|
|
43
|
+
### Added
|
|
44
|
+
- Step to set vars to secret variables. That is, using environment variables while hiding the names of those variables in the report, errors, and console logs.
|
|
42
45
|
|
|
43
|
-
## [
|
|
46
|
+
## [4.0.0] - 2022-03-30
|
|
44
47
|
### Added
|
|
45
|
-
- Add
|
|
46
|
-
- Add
|
|
48
|
+
- BREAKING: Add package.json creation/overwriting to action.yml. Simplify package.json.
|
|
49
|
+
- BREAKING: Add action.yml that runs most of what users' workflows run now, along with notes for a new user workflow that will take less maintainance. See #420. We need to add documentation on how to write a workflow file as it currently is if they want to take back control. Setup interview has not yet been updated with this workflow.
|
|
47
50
|
- Step: `the text in the JSON variable "variable_with_text_value" should be`. Compare JSON variable with a text value to given text. See #470. Does not accept nested values, e.g. "child.name.first". Downloads all the JSON vars to a .json file in the "downloads" artifacts.
|
|
48
51
|
- Step to log the page's JSON variables and values in the report. Future goal: save to file. See #454.
|
|
49
52
|
- Step to log into the developer's docassemble server account using GitHub SECRETs. See #499.
|
|
@@ -51,16 +54,17 @@ Format:
|
|
|
51
54
|
- Add control of alkiln npm version to the action.yml
|
|
52
55
|
|
|
53
56
|
### Changed
|
|
54
|
-
- NPM package is now suffolklitlab scoped package. Refer to @suffolklitlab/alkiln from now on.
|
|
55
|
-
- Update to v13 of puppeteer
|
|
56
|
-
- Update to v7 of cucumber
|
|
57
|
-
- Update action.yml node to v17
|
|
58
|
-
- Use alkiln v4 package.json we create during the test run in `action.yml` (as discussed in #489)
|
|
57
|
+
- BREAKING: NPM package is now suffolklitlab scoped package. Refer to @suffolklitlab/alkiln from now on.
|
|
58
|
+
- BREAKING: Update to v13 of puppeteer
|
|
59
|
+
- BREAKING: Update to v7 of cucumber
|
|
60
|
+
- BREAKING: Update action.yml node to v17
|
|
61
|
+
- BREAKING: Use alkiln v4 package.json we create during the test run in `action.yml` (as discussed in #489)
|
|
62
|
+
- BREAKING: Use API key to access da server, create projects, pull code, delete projects, and check for server restart.
|
|
63
|
+
- BREAKING: Throw an error for uploading nonexistent file, changed to error instead of warning in reports.
|
|
64
|
+
- Update test setup interview to v4 of ALKiln.
|
|
59
65
|
- Remove and ignore package-lock.json so that our tests will behave more like our users' tests
|
|
60
|
-
- Use API key to access da server, create projects, pull code, delete projects, and check for server restart.
|
|
61
66
|
- Add warning in steps.js for name input with too many parts. Add test in reports.
|
|
62
|
-
- fix typo in the report
|
|
63
|
-
- Throw an error for uploading nonexistent file, changed to error instead of warning in reports.
|
|
67
|
+
- fix typo in the report
|
|
64
68
|
- Allow a developer to wait as a first Step. See #387.
|
|
65
69
|
|
|
66
70
|
<!-- ## [3.0.1-peer-deps.1] - 2021-12-07 -->
|
|
@@ -75,6 +79,57 @@ Format:
|
|
|
75
79
|
- fix typo in the report.
|
|
76
80
|
- session_vars branch path regex not removing start of branch path
|
|
77
81
|
|
|
82
|
+
## [3.2.1] - 2022-02-09
|
|
83
|
+
### Fixed
|
|
84
|
+
- Typo in report that was messing up copy/paste of test table
|
|
85
|
+
|
|
86
|
+
## [3.2.0] - 2022-02-09
|
|
87
|
+
### Added
|
|
88
|
+
- Add Step to log into server account using GitHub SECRETs (environment variables). See #499.
|
|
89
|
+
|
|
90
|
+
## [3.1.11] - 2022-02-02
|
|
91
|
+
### Changed
|
|
92
|
+
- Handle an Assembly Line name Step value that has too many names, including a warning for the developer. Include the first three names and the last name to try to catch suffixes.
|
|
93
|
+
|
|
94
|
+
## [3.0.11] - 2022-01-25
|
|
95
|
+
### Added
|
|
96
|
+
- Step: `the text in the JSON variable "variable_with_text_value" should be`. Compare JSON variable with a text value to given text. See #470. Does not accept nested values, e.g. "child.name.first". Downloads all the JSON vars to a .json file in the "downloads" artifacts.
|
|
97
|
+
|
|
98
|
+
## [3.0.10] - 2022-01-24
|
|
99
|
+
### Added
|
|
100
|
+
- Step to log the page's JSON variables and values in the report. Future goal: save to file. See #454.
|
|
101
|
+
|
|
102
|
+
## [3.0.9] - 2022-01-10
|
|
103
|
+
### Security
|
|
104
|
+
- Install the security fix in the action's package.json as well. Because dependents have to require cucumber itself, if they're not using our action (if they have their own package.json), they're going to have to implement this fix in their repo too. That's why our fix isn't working. We'll start the process of helping developers update. We also need to deprecate all previous versions of ALKiln. See #489.
|
|
105
|
+
|
|
106
|
+
## [3.0.8] - 2022-01-10
|
|
107
|
+
### Security
|
|
108
|
+
- Fully publish the security fix in v3.0.7
|
|
109
|
+
|
|
110
|
+
## [3.0.7] - 2022-01-10
|
|
111
|
+
### Security
|
|
112
|
+
- Pinned colors library to 1.4.0. See https://www.bleepingcomputer.com/news/security/dev-corrupts-npm-libs-colors-and-faker-breaking-thousands-of-apps/. We don't use faker, so we don't need to handle that.
|
|
113
|
+
|
|
114
|
+
## [3.0.6] - 2022-01-07
|
|
115
|
+
### Changed
|
|
116
|
+
- See #371
|
|
117
|
+
- Scenario heading tags are now on a second line
|
|
118
|
+
- Artifact filenames use dashes more and are created separately from Scenario headings
|
|
119
|
+
- Scenario ids are separated from filenames and Scenario headings
|
|
120
|
+
|
|
121
|
+
## [3.0.5] - 2022-01-06
|
|
122
|
+
### Added
|
|
123
|
+
- Add action.yml that runs most of what users' workflows run now, along with notes for a new user workflow that will take less maintainance. See #420. We need to add documentation on how to write a workflow file as it currently is if they want to take back control. Setup interview has not yet been updated with this workflow.
|
|
124
|
+
- Add package.json creation/overwriting to action.yml. Simplify package.json.
|
|
125
|
+
|
|
126
|
+
<!-- ## [3.0.1-peer-deps.1] - 2021-12-07 -->
|
|
127
|
+
### Removed
|
|
128
|
+
- Peer dependencies and dev dependencies. Now `cucumber` is just a dependency. See https://github.com/SuffolkLITLab/ALKiln/issues/396 for discussion. Setup interview has not been updated to remove dependencies.
|
|
129
|
+
|
|
130
|
+
### Fixed
|
|
131
|
+
- Fixed interview name not allowing url parameters. #449
|
|
132
|
+
|
|
78
133
|
## [3.0.4] - 2021-12-11
|
|
79
134
|
### Fixed
|
|
80
135
|
- Use different criteria for testing that the package was saved.
|
package/README.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
|
1
|
-
#
|
|
2
|
-
Integrated automated end-to-end testing with docassemble, puppeteer, and
|
|
1
|
+
# ALKiln
|
|
2
|
+
Integrated automated end-to-end testing with docassemble, puppeteer, cucumber, and chai.
|
|
3
|
+
|
|
4
|
+
Works well with https://github.com/suffolklitlab/docassemble-AssemblyLine but isn't
|
|
5
|
+
dependent on it.
|
|
6
|
+
|
|
7
|
+
Formerly named "docassemble-cucumber"
|
|
8
|
+
|
|
9
|
+
## Documentation
|
|
10
|
+
Read about how to use this framework at https://suffolklitlab.org/docassemble-AssemblyLine-documentation/docs/automated_integrated_testing/.
|
|
3
11
|
|
|
4
12
|
## Integration tests
|
|
5
13
|
To setup for the integration tests, create the project on the server:
|
package/action.yml
CHANGED
|
@@ -27,7 +27,7 @@ inputs:
|
|
|
27
27
|
ALKILN_VERSION:
|
|
28
28
|
description: 'Version of ALKiln to get from npm.'
|
|
29
29
|
required: false
|
|
30
|
-
default: '4.0.0'
|
|
30
|
+
default: '^4.0.0'
|
|
31
31
|
|
|
32
32
|
runs:
|
|
33
33
|
using: "composite"
|
|
@@ -53,6 +53,7 @@ runs:
|
|
|
53
53
|
echo "PLAYGROUND_PASSWORD=${{ inputs.PLAYGROUND_PASSWORD }}" >> $GITHUB_ENV
|
|
54
54
|
echo "PLAYGROUND_ID=${{ inputs.PLAYGROUND_ID }}" >> $GITHUB_ENV
|
|
55
55
|
echo "DOCASSEMBLE_DEVELOPER_API_KEY=${{ inputs.DOCASSEMBLE_DEVELOPER_API_KEY }}" >> $GITHUB_ENV
|
|
56
|
+
echo "ALKILN_VERSION=${{ inputs.ALKILN_VERSION }}" >> $GITHUB_ENV
|
|
56
57
|
shell: bash
|
|
57
58
|
- name: "ALKiln: confirm info"
|
|
58
59
|
run: echo -e "\nRepo is $REPO_URL\nBranch ref is $BRANCH_PATH\nManually added languages are $EXTRA_LANGUAGES\nAll other data is secret\n"
|
|
@@ -60,7 +61,7 @@ runs:
|
|
|
60
61
|
|
|
61
62
|
# This doesn't use an already established package-lock.json and that
|
|
62
63
|
# comes with some types of test breaking risks. Same problem as before.
|
|
63
|
-
- name:
|
|
64
|
+
- name: Temporarily overwrite or create package.json
|
|
64
65
|
run: |
|
|
65
66
|
cat << EOF > "package.json"
|
|
66
67
|
{
|
|
@@ -243,7 +243,7 @@ da_i.loop_to_create_project = async function( creation_options ) {
|
|
|
243
243
|
throw error;
|
|
244
244
|
|
|
245
245
|
} // ends type of non-response error
|
|
246
|
-
} // ends type of
|
|
246
|
+
} // ends type of response error
|
|
247
247
|
} // ends try
|
|
248
248
|
} // ends while
|
|
249
249
|
}; // Ends da_i.loop_to_create_project()
|
package/lib/scope.js
CHANGED
|
@@ -22,6 +22,30 @@ let escape_regex = function( to_escape ) {
|
|
|
22
22
|
return to_escape.replace(/[-\/\\^$*+?.()|[\]{}]/g, '\\$&');
|
|
23
23
|
};
|
|
24
24
|
|
|
25
|
+
let get_original_row = function (row) {
|
|
26
|
+
/* `row` might be a row, a source row, or an original row. */
|
|
27
|
+
|
|
28
|
+
// Until we flatten row data a litte, we need to find the `orignal` row, which has the flags
|
|
29
|
+
let source_row = row.source || row;
|
|
30
|
+
let original = source_row.original || source_row;
|
|
31
|
+
return original;
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
let printable_var_value = function(row) {
|
|
35
|
+
/* `row` might be a row, a source row, or an original row. */
|
|
36
|
+
let original = get_original_row(row);
|
|
37
|
+
if ( is_secret_var(original) ) {
|
|
38
|
+
return original.flags.secret_name;
|
|
39
|
+
}
|
|
40
|
+
return row.value;
|
|
41
|
+
};
|
|
42
|
+
|
|
43
|
+
let is_secret_var = function(row) {
|
|
44
|
+
/* `row` might be a row, a source row, or an original row. */
|
|
45
|
+
let original = get_original_row(row);
|
|
46
|
+
return original.flags !== undefined && original.flags.secret_name !== undefined;
|
|
47
|
+
}
|
|
48
|
+
|
|
25
49
|
let proxies_regex = /(\bx\.)|(\bx\[)|(\[[ijklmn]\])/g;
|
|
26
50
|
|
|
27
51
|
module.exports = {
|
|
@@ -63,7 +87,7 @@ module.exports = {
|
|
|
63
87
|
}, // Ends scope.setReportScenarioStatus()
|
|
64
88
|
|
|
65
89
|
convertToOriginalStoryTableRow: async function( scope, { row_data }) {
|
|
66
|
-
/* Returns the original data of row into a string
|
|
90
|
+
/* Returns the original data of row into a string formatted as a story table row.
|
|
67
91
|
* Should this show the actual row data that was used instead? That could
|
|
68
92
|
* be pretty confusing to most devs. */
|
|
69
93
|
let source_row = row_data.source || row_data;
|
|
@@ -113,7 +137,7 @@ module.exports = {
|
|
|
113
137
|
}, // Ends scope.addReportHeading()
|
|
114
138
|
|
|
115
139
|
getPrintableReport: async function( scope ) {
|
|
116
|
-
/* Return a string
|
|
140
|
+
/* Return a string generated from test report data. */
|
|
117
141
|
let report = `Assembly Line Kiln Automated Testing Report - ${ (new Date()).toUTCString() }\n`;
|
|
118
142
|
|
|
119
143
|
// First list failed scenarios, then non-passed scenarios, then passed scenarios
|
|
@@ -137,7 +161,7 @@ module.exports = {
|
|
|
137
161
|
}, // Ends scope.getPrintableReport()
|
|
138
162
|
|
|
139
163
|
getPrintableScenario: async function( scope, { scenario }) {
|
|
140
|
-
/* Return a string
|
|
164
|
+
/* Return a string generated from scenario list of report items. */
|
|
141
165
|
let [ scenario_id, data ] = scenario;
|
|
142
166
|
let report = ``;
|
|
143
167
|
for ( let line of data.lines ) {
|
|
@@ -391,7 +415,7 @@ module.exports = {
|
|
|
391
415
|
scope.page.waitForSelector(`fieldset.da-field-buttons button[type="submit"]`), // other pages (this is the most consistent way)
|
|
392
416
|
scope.page.waitForSelector(`fieldset .dasigsave`), // signature page
|
|
393
417
|
]);
|
|
394
|
-
|
|
418
|
+
await elem.evaluate( el => { return el.className });
|
|
395
419
|
// Waits for navigation or user error
|
|
396
420
|
await scope.tapElement( scope, elem );
|
|
397
421
|
},
|
|
@@ -414,7 +438,7 @@ module.exports = {
|
|
|
414
438
|
await scope.page.goto(interview_url, { waitUntil: 'domcontentloaded', timeout: scope.timeout })
|
|
415
439
|
|
|
416
440
|
let load_result = await scope.getLoadData( scope );
|
|
417
|
-
// Errors, or reaching timeout, causes an
|
|
441
|
+
// Errors, or reaching timeout, causes an exception and is shown to the developer at the end
|
|
418
442
|
if ( load_result.error ) {
|
|
419
443
|
await scope.addToReport(scope, { type: `error`, value: `On final attempt to load interview, got "${ load_result.error }"` });
|
|
420
444
|
expect( load_result.error ).to.equal( '' );
|
|
@@ -467,19 +491,29 @@ module.exports = {
|
|
|
467
491
|
|
|
468
492
|
normalizeTable: async function ( scope, { var_data, from_story_table=true }) {
|
|
469
493
|
/* Return data derived from cucumber variable-setting data.
|
|
470
|
-
|
|
494
|
+
* Table must have headers. Arrays are not supported. */
|
|
471
495
|
|
|
472
496
|
// Support tables with no 'trigger' column
|
|
473
497
|
let supported_table = [];
|
|
474
498
|
for ( let row of var_data ) {
|
|
475
499
|
// See tests > unit_tests > tables.fixtures.js > tables.old_to_current_formatting
|
|
500
|
+
let actual_var_value = row.value;
|
|
501
|
+
if (is_secret_var(row)) {
|
|
502
|
+
actual_var_value = process.env[row.flags.secret_name];
|
|
503
|
+
if ( actual_var_value === undefined ) {
|
|
504
|
+
let not_in_env_msg = `The GitHub SECRET "${ row.flags.secret_name }" doesn't exist. Check for a typo and check your workflow file.`;
|
|
505
|
+
await scope.addToReport( scope, { type: `error`, value: not_in_env_msg })
|
|
506
|
+
throw ReferenceError(`Could not find "${row.flags.secret_name }": ${not_in_env_msg}.`)
|
|
507
|
+
}
|
|
508
|
+
}
|
|
509
|
+
|
|
476
510
|
let result = {
|
|
477
511
|
original: row,
|
|
478
512
|
// Give fake data if needed to prevent some kind of hidden input field from being found
|
|
479
513
|
// See bug at https://github.com/plocket/docassemble-cucumber/issues/79
|
|
480
514
|
// May be able to remove this once converted to `getAllFields`
|
|
481
515
|
var: row.var || 'al_no_var_name',
|
|
482
|
-
value:
|
|
516
|
+
value: actual_var_value,
|
|
483
517
|
trigger: row.trigger || '',
|
|
484
518
|
times_used: 0,
|
|
485
519
|
}
|
|
@@ -487,7 +521,9 @@ module.exports = {
|
|
|
487
521
|
supported_table.push( result );
|
|
488
522
|
}
|
|
489
523
|
|
|
490
|
-
if ( session_vars.get_debug() ) {
|
|
524
|
+
if ( session_vars.get_debug() ) {
|
|
525
|
+
console.log( 'scope.normalizedTable():\n', JSON.stringify( supported_table ));
|
|
526
|
+
}
|
|
491
527
|
|
|
492
528
|
return supported_table;
|
|
493
529
|
}, // Ends scope.normalizeTable()
|
|
@@ -639,7 +675,7 @@ module.exports = {
|
|
|
639
675
|
|
|
640
676
|
// Make sure this selector matches only one node
|
|
641
677
|
let matching_nodes = $( selector );
|
|
642
|
-
// This check is here to ensure
|
|
678
|
+
// This check is here to ensure safety of future development. If there is more
|
|
643
679
|
// or less than one match, this code may have unexpected behavior later.
|
|
644
680
|
// Really need everything to be unique here if at all possible. Shall we resort
|
|
645
681
|
// to parents and n-th child?
|
|
@@ -860,7 +896,7 @@ module.exports = {
|
|
|
860
896
|
let uses_proxies = false; // See "proxies" notes further down
|
|
861
897
|
let trigger_might_be_needed = true;
|
|
862
898
|
|
|
863
|
-
// There are
|
|
899
|
+
// There are multiple guesses because field names are sometimes encoded
|
|
864
900
|
for ( let guess of field.guesses ) {
|
|
865
901
|
for ( let table_row of var_data ) {
|
|
866
902
|
let this_row_matches_this_field = false;
|
|
@@ -1085,7 +1121,7 @@ module.exports = {
|
|
|
1085
1121
|
|
|
1086
1122
|
let { tag, type } = field;
|
|
1087
1123
|
// We use the last match. We wrote a warning for the user if there were
|
|
1088
|
-
//
|
|
1124
|
+
// multiple matches in the story that the last value will be used. In future,
|
|
1089
1125
|
// Steps will be able to set this data gradually and we don't want the developer
|
|
1090
1126
|
// to get confused thinking that an earlier Step was used instead of a later one.
|
|
1091
1127
|
let row = matching_input_rows[ matching_input_rows.length - 1 ];
|
|
@@ -1360,7 +1396,7 @@ module.exports = {
|
|
|
1360
1396
|
},
|
|
1361
1397
|
|
|
1362
1398
|
examinePageID: async function ( scope, question_id = '' ) {
|
|
1363
|
-
/* Looks for a
|
|
1399
|
+
/* Looks for a sanitized version of the question id as it's written
|
|
1364
1400
|
* in the .yml. docassemble's way in python:
|
|
1365
1401
|
* re.sub(r'[^A-Za-z0-9]+', '-', interview_status.question.id.lower()) */
|
|
1366
1402
|
let id_as_class = question_id.toLowerCase().replace(/[^A-Za-z0-9]+/g, '-');
|
|
@@ -1506,9 +1542,10 @@ module.exports = {
|
|
|
1506
1542
|
if ( row.times_used === 0 ) {
|
|
1507
1543
|
num_unused_rows += 1;
|
|
1508
1544
|
let source_row = row;
|
|
1545
|
+
// TODO: Make the row object flatter so we don't need to dig in this way
|
|
1509
1546
|
if ( row.source ) { source_row = row.source; }
|
|
1510
1547
|
let original_row = source_row.original;
|
|
1511
|
-
let msg = `Did not find a field on this page for the variable
|
|
1548
|
+
let msg = `Did not find a field on this page for the variable "${ original_row.var }" that could be set to "${ original_row.value }"`;
|
|
1512
1549
|
await scope.addToReport(scope, { type: `warning`, value: msg });
|
|
1513
1550
|
}
|
|
1514
1551
|
}
|
|
@@ -1605,25 +1642,29 @@ module.exports = {
|
|
|
1605
1642
|
* we believe we've set it up so that people will make it a number.
|
|
1606
1643
|
* Sometimes this will be from `.there_is_another`. We think we've ensured
|
|
1607
1644
|
* that it will only come from Steps and should be `True` or `False`.
|
|
1608
|
-
* This feels precarious, but I can't see another
|
|
1645
|
+
* This feels precarious, but I can't see another permutation.
|
|
1609
1646
|
*
|
|
1610
1647
|
* This may in future be used to normalize all values that need `True`/`False`
|
|
1611
1648
|
* normalization.
|
|
1612
1649
|
*/
|
|
1613
|
-
let value = row.value;
|
|
1650
|
+
let value = row.value.toLowerCase();
|
|
1651
|
+
let val_to_print = printable_var_value(row);
|
|
1614
1652
|
let int_val = parseInt( value );
|
|
1615
1653
|
|
|
1616
1654
|
// Step stuff. Take into account a lot of possible values
|
|
1617
1655
|
if ( value.match( /(false|no|unchecked|uncheck|deselected|deselect)/i )) {
|
|
1656
|
+
// Should this also contain a warning if this is trying to set `target_number`?
|
|
1618
1657
|
return `False`;
|
|
1619
1658
|
|
|
1620
1659
|
} else if ( value.match( /(true|yes|checked|check|selected|select)/i )) {
|
|
1621
1660
|
if ( from_story_table ) {
|
|
1622
|
-
// Just in case, but this should never happen.
|
|
1623
1661
|
if ( row.var.match( /\.there_is_another$/ ) ) {
|
|
1662
|
+
// `.there_is_another` should never get in here, but just in case
|
|
1624
1663
|
await scope.addToReport( scope, { type: `warning`, value: `The attribute \`.there_is_another\` is invalid in story table tests. Replace it with \`.target_number\` in your \`var\` column. Set the \`value\` to the number of items in that list. This test will now set this row's \`value\` to \`False\`. See https://suffolklitlab.github.io/docassemble-AssemblyLine-documentation/docs/automated_integrated_testing#there_is_another-loop. The row data is\n${ JSON.stringify( row.original )}` });
|
|
1625
1664
|
} else {
|
|
1626
|
-
|
|
1665
|
+
// If `target_number` is set to True instead of a number
|
|
1666
|
+
// Since secret vars can't be passed in a table, this should never be a secret var
|
|
1667
|
+
await scope.addToReport( scope, { type: `warning`, value: `${ val_to_print } value is not a valid value for ${ row.var } here. This test will default to \`False\` to avoid problems.` });
|
|
1627
1668
|
}
|
|
1628
1669
|
return `False`;
|
|
1629
1670
|
} else { return `True`; }
|
|
@@ -1638,7 +1679,8 @@ module.exports = {
|
|
|
1638
1679
|
return `False`;
|
|
1639
1680
|
|
|
1640
1681
|
} else if ( isNaN( parseInt( value )) ) { // Loop will end
|
|
1641
|
-
|
|
1682
|
+
// I can't figure out how to trigger this warning either
|
|
1683
|
+
await scope.addToReport( scope, { type: `warning`, value: `${ val_to_print } value is not a valid value for ${ row.var } here. This test will default to \`False\` to avoid problems.` });
|
|
1642
1684
|
return `False`;
|
|
1643
1685
|
|
|
1644
1686
|
} else { // It's a number. Loop will continue.
|
|
@@ -1680,7 +1722,7 @@ module.exports = {
|
|
|
1680
1722
|
// All results are lists to match `$x`
|
|
1681
1723
|
let possible_errors = {
|
|
1682
1724
|
complex_user_error: await scope.page.$$('.alert-danger'),
|
|
1683
|
-
// The below does not work if there's not a space
|
|
1725
|
+
// The below does not work if there's not a space between ':' and 'none'
|
|
1684
1726
|
simple_user_error: await scope.page.$x(`//*[contains(@class, "da-has-error")][not(contains(@style,"display: none"))]`),
|
|
1685
1727
|
system_error: await scope.page.$x( '//meta[@content="docassemble: Error"]' ),
|
|
1686
1728
|
};
|
|
@@ -1703,13 +1745,13 @@ module.exports = {
|
|
|
1703
1745
|
throwPageError: async function ( scope, { error_handlers = {}, id = '' }) {
|
|
1704
1746
|
/* Throw a cucumber error with the error that appeared on the page. */
|
|
1705
1747
|
if ( error_handlers.system_error !== undefined ) {
|
|
1706
|
-
let sys_err_message = `The test tried to continue to the next page, but there was a system error. Check the screenshot.`
|
|
1748
|
+
let sys_err_message = `The test tried to continue to the next page after ${ id }, but there was a system error. Check the screenshot.`
|
|
1707
1749
|
await scope.addToReport(scope, { type: `error`, value: sys_err_message });
|
|
1708
1750
|
expect( was_system_error, sys_err_message ).to.be.false;
|
|
1709
1751
|
|
|
1710
1752
|
} else {
|
|
1711
1753
|
// If it wasn't a system error, it was a user error
|
|
1712
|
-
let user_err_message = `The test tried to continue to the next page, but the user got an error. Check the screenshot.`
|
|
1754
|
+
let user_err_message = `The test tried to continue to the next page after ${ id }, but the user got an error. Check the screenshot.`
|
|
1713
1755
|
await scope.addToReport(scope, { type: `error`, value: user_err_message });
|
|
1714
1756
|
expect( true, user_err_message ).to.be.false;
|
|
1715
1757
|
}
|
|
@@ -1792,6 +1834,21 @@ module.exports = {
|
|
|
1792
1834
|
await scope.setFields( scope, { var_data, ensure_navigation: false, ensure_all_vars_used: true });
|
|
1793
1835
|
}, // Ends set_regular_var()
|
|
1794
1836
|
|
|
1837
|
+
set_secret_var: async ( scope, var_name, set_to_env_name ) => {
|
|
1838
|
+
/** Sets a non-story table variable to a "secret" value (i.e. an env var) */
|
|
1839
|
+
let var_data = await scope.normalizeTable( scope, {
|
|
1840
|
+
var_data: [{
|
|
1841
|
+
var: var_name,
|
|
1842
|
+
value: set_to_env_name,
|
|
1843
|
+
trigger: scope.trigger_not_needed_flag,
|
|
1844
|
+
flags: {secret_name: set_to_env_name}
|
|
1845
|
+
}],
|
|
1846
|
+
});
|
|
1847
|
+
// Don't continue if the variable doesn't cause navigation itself. Also,
|
|
1848
|
+
// if this variable cannot be set on this page, trigger an error
|
|
1849
|
+
await scope.setFields( scope, { var_data, ensure_navigation: false, ensure_all_vars_used: true });
|
|
1850
|
+
},
|
|
1851
|
+
|
|
1795
1852
|
tap_term: async ( scope, phrase ) => {
|
|
1796
1853
|
/** Tap a link that doesn't navigate. Depends on the language of the text. */
|
|
1797
1854
|
const [link] = await scope.page.$x(`//a[contains(text(), "${phrase}")]`);
|
package/lib/steps.js
CHANGED
|
@@ -14,13 +14,11 @@ const wrapPromiseWithTimeout = require('./cucumber_8_shim');
|
|
|
14
14
|
const fs = require('fs');
|
|
15
15
|
const path = require('path');
|
|
16
16
|
const { expect } = require('chai');
|
|
17
|
-
const puppeteer = require('puppeteer');
|
|
18
17
|
const { v4: uuidv4 } = require('uuid');
|
|
19
18
|
|
|
20
19
|
// Ours
|
|
21
20
|
const scope = require('./scope');
|
|
22
21
|
const session_vars = require('./utils/session_vars');
|
|
23
|
-
|
|
24
22
|
/* Of Note:
|
|
25
23
|
- We're using `*=` because sometimes da text has funny characters in it that are hard to anticipate
|
|
26
24
|
*/
|
|
@@ -33,7 +31,7 @@ require("dotenv").config(); // Where did this go?
|
|
|
33
31
|
1. 'choice' to have a more specific way to access each item. For
|
|
34
32
|
example, for a list collect or other things that have multiple
|
|
35
33
|
things with the same text on the page.
|
|
36
|
-
1. Figure out how to test allowing
|
|
34
|
+
1. Figure out how to test allowing flexibility for coder. For example:
|
|
37
35
|
there is placeholder text for the title of the form and if it's not
|
|
38
36
|
defined, placeholder text should appear (though that behavior may
|
|
39
37
|
bear discussion).
|
|
@@ -180,7 +178,7 @@ Given(/I start the interview at "([^"]+)"(?: in lang "([^"]+)")?/, {timeout: -1}
|
|
|
180
178
|
|
|
181
179
|
// If response has a file in it that matches the file we are downloading
|
|
182
180
|
if ( response._headers['content-disposition'] && (response._headers['content-disposition']).includes(`attachment;filename=${scope.toDownload}`) ) {
|
|
183
|
-
// We're
|
|
181
|
+
// We're choosing to avoid removing the event listener in case removal
|
|
184
182
|
// would prevent downloading other files. The listener should get
|
|
185
183
|
// destroyed when the page closes anyway.
|
|
186
184
|
|
|
@@ -219,7 +217,7 @@ Given(/I start the interview at "([^"]+)"(?: in lang "([^"]+)")?/, {timeout: -1}
|
|
|
219
217
|
await custom_lang_timeout;
|
|
220
218
|
|
|
221
219
|
|
|
222
|
-
let
|
|
220
|
+
let custom_afterStep_timeout = new Promise(async ( resolve, reject ) => {
|
|
223
221
|
let page_timeout = setTimeout(function() { reject(`\`afterStep()\` took too long.`); }, scope.timeout);
|
|
224
222
|
// self-invoke an anonymous async function so we don't have to
|
|
225
223
|
// abstract the details of resolving and rejecting.
|
|
@@ -230,8 +228,8 @@ Given(/I start the interview at "([^"]+)"(?: in lang "([^"]+)")?/, {timeout: -1}
|
|
|
230
228
|
// prevent temporary hang at end of tests
|
|
231
229
|
finally { clearTimeout( page_timeout ); }
|
|
232
230
|
})();
|
|
233
|
-
}); // ends
|
|
234
|
-
await
|
|
231
|
+
}); // ends custom_afterStep_timeout
|
|
232
|
+
await custom_afterStep_timeout;
|
|
235
233
|
});
|
|
236
234
|
|
|
237
235
|
Given(/I (?:sign|log) ?(?:in)?(?:on)?(?:to the server)? with(?: the email)? "([^"]+)" and(?: the password)? "([^"]+)"(?: SECRETs)?/i, {timeout: -1}, async ( email, password ) => {
|
|
@@ -252,7 +250,7 @@ Given(/I (?:sign|log) ?(?:in)?(?:on)?(?:to the server)? with(?: the email)? "([^
|
|
|
252
250
|
);
|
|
253
251
|
});
|
|
254
252
|
|
|
255
|
-
// I am using a
|
|
253
|
+
// I am using a mobile/pc
|
|
256
254
|
When(/I am using an? (.*)/, async ( device ) => {
|
|
257
255
|
// Let developer pick mobile device if they want to
|
|
258
256
|
// TODO: Add tablet?
|
|
@@ -350,7 +348,7 @@ When(/^(?:the user gets)?(?:I get)? to ?(?:the)?(?: ?question)?(?: id)? "([^"]+)
|
|
|
350
348
|
for ( let row of supported_table ) {
|
|
351
349
|
row.report_str = `${ await scope.convertToOriginalStoryTableRow(scope, { row_data: row }) }`;
|
|
352
350
|
}
|
|
353
|
-
// Sort into
|
|
351
|
+
// Sort into alphabetical order, as is recommended for writing story tables
|
|
354
352
|
let sorted_rows = supported_table.sort( function ( row1, row2 ) {
|
|
355
353
|
if( row1.report_str < row2.report_str ) { return -1; }
|
|
356
354
|
if( row1.report_str > row2.report_str ) { return 1; }
|
|
@@ -404,7 +402,7 @@ When(/I wait (\d*\.?\d+) seconds?/, { timeout: -1 }, async ( seconds ) => {
|
|
|
404
402
|
* - Should this really reset the 'navigated' flag? Should the below
|
|
405
403
|
* return succeed or fail? e.g:
|
|
406
404
|
* I tap to continue; I wait 10 seconds; I arrive at the next page;
|
|
407
|
-
* Maybe the '
|
|
405
|
+
* Maybe the 'navigated' flag should only be reset just before trying to
|
|
408
406
|
* navigate (after pressing a button)
|
|
409
407
|
*/
|
|
410
408
|
return wrapPromiseWithTimeout(
|
|
@@ -426,7 +424,7 @@ Then(/I take a screenshot ?(?:named "([^"]+)")?/, { timeout: -1 }, async ( name
|
|
|
426
424
|
When("I do nothing", async () => { await scope.afterStep(scope); }); // √
|
|
427
425
|
|
|
428
426
|
Then('the question id should be {string}', { timeout: -1 }, async ( question_id ) => {
|
|
429
|
-
/* Looks for a
|
|
427
|
+
/* Looks for a sanitized version of the question id as it's written
|
|
430
428
|
* in the .yml. docassemble's way */
|
|
431
429
|
let { question_has_id, id, id_matches } = await scope.examinePageID( scope, question_id );
|
|
432
430
|
|
|
@@ -449,7 +447,7 @@ Then('the question id should be {string}', { timeout: -1 }, async ( question_id
|
|
|
449
447
|
});
|
|
450
448
|
|
|
451
449
|
// Allow emphasis with capital letters
|
|
452
|
-
Then(/I should see the phrase "([^"]+)"/i, async ( phrase ) => { // √
|
|
450
|
+
Then(/I ?(?:should)? see the phrase "([^"]+)"/i, async ( phrase ) => { // √
|
|
453
451
|
/* In Chrome, this `innerText` gets only visible text */
|
|
454
452
|
const bodyText = await scope.page.$eval('body', elem => elem.innerText);
|
|
455
453
|
|
|
@@ -567,7 +565,7 @@ Then('I arrive at the next page', async () => { // √
|
|
|
567
565
|
|
|
568
566
|
// Link observations have the 'Escape' button link in mind
|
|
569
567
|
Then('I should see the link {string}', async ( linkText ) => { // √
|
|
570
|
-
/*
|
|
568
|
+
/* Loosely match link visible text. Should probably deprecate this. */
|
|
571
569
|
let [link] = await scope.page.$x(`//a[contains(text(), "${linkText}")]`);
|
|
572
570
|
|
|
573
571
|
let msg = `Cannot find a link with the text "${ linkText }".`;
|
|
@@ -668,7 +666,7 @@ Then(/I get the(?: page's)?(?: JSON)? var(?:iable)?s and val(?:ue)?s/i, { timeou
|
|
|
668
666
|
* to get the variable values on the page of the interview and add
|
|
669
667
|
* them to the report.
|
|
670
668
|
*
|
|
671
|
-
*
|
|
669
|
+
* Variations:
|
|
672
670
|
* I get the page's JSON variables and values
|
|
673
671
|
* I get the json vars and vals
|
|
674
672
|
* I get the page's vars and vals
|
|
@@ -735,6 +733,15 @@ When(/I set the var(?:iable)? "([^"]+)" to "([^"]+)"/, { timeout: -1 }, async (
|
|
|
735
733
|
);
|
|
736
734
|
});
|
|
737
735
|
|
|
736
|
+
When(/I set the var(?:iable)? "([^"]+)" to ?(?:the)? ?(?:GitHub)? secret "([^"]+)"/i, { timeout: -1 }, async ( var_name, set_to_env_name ) => {
|
|
737
|
+
/* Set a non-story table variable to a secrete value. Same as the above, but reads the value from `process.env` */
|
|
738
|
+
return wrapPromiseWithTimeout(
|
|
739
|
+
scope.steps.set_secret_var( scope, var_name, set_to_env_name ),
|
|
740
|
+
scope.timeout
|
|
741
|
+
);
|
|
742
|
+
});
|
|
743
|
+
|
|
744
|
+
|
|
738
745
|
Then(/I upload "([^"]+)" to "([^"]+)"/, { timeout: -1 }, async ( filenames, var_name ) => {
|
|
739
746
|
/* Uploads file (docx, jpg, etc) in same folder to input var. Can actually use
|
|
740
747
|
* 'I set the var "" to ""', but this is more explicit if it's desired. */
|
package/lib/utils/langs.js
CHANGED
|
@@ -20,7 +20,7 @@ for ( let full_path of filepaths ) { // Handle old language files before creati
|
|
|
20
20
|
// Don't apply this to the example test that this package always provides
|
|
21
21
|
if ( full_path.includes( 'example_test.feature' )) { continue; }
|
|
22
22
|
|
|
23
|
-
// Delete
|
|
23
|
+
// Delete already existing non-default language files and skip the rest of this. For example,
|
|
24
24
|
// if 3 languages were tested previously and now only two are being tested.
|
|
25
25
|
if ( full_path.includes( flag )) {
|
|
26
26
|
fs.unlinkSync( full_path );
|
|
@@ -10,7 +10,7 @@ require('dotenv').config();
|
|
|
10
10
|
/* This file provides user info variables. E.g. server address. They're either
|
|
11
11
|
* `process.env` vars or vars that have been built from `process.env` vars.
|
|
12
12
|
* It is meant to avoid duplication, make testing easier, and leave room for
|
|
13
|
-
* future
|
|
13
|
+
* future manipulation. No other file should need to use `process.env`.
|
|
14
14
|
*
|
|
15
15
|
* We have a bunch of `.get_foo()` functions partly because it makes the code
|
|
16
16
|
* much easier to test.
|
|
@@ -105,7 +105,7 @@ session_vars.get_project_name = function () {
|
|
|
105
105
|
|
|
106
106
|
session_vars.delete_project_name = function () {
|
|
107
107
|
/* Delete the file storing the name of the docassemble project */
|
|
108
|
-
// Avoid race
|
|
108
|
+
// Avoid race conditions possibly caused by checking for existence first
|
|
109
109
|
// Delete file if possible
|
|
110
110
|
try {
|
|
111
111
|
fs.unlinkSync( project_name_file_path );
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@suffolklitlab/alkiln",
|
|
3
|
-
"version": "4.
|
|
3
|
+
"version": "4.1.1",
|
|
4
4
|
"description": "Integrated automated end-to-end testing with docassemble, puppeteer, and cucumber.",
|
|
5
5
|
"main": "lib/index.js",
|
|
6
6
|
"scripts": {
|
|
@@ -32,7 +32,7 @@
|
|
|
32
32
|
"bugs": {
|
|
33
33
|
"url": "https://github.com/SuffolkLITLab/ALKiln/issues"
|
|
34
34
|
},
|
|
35
|
-
"homepage": "https://suffolklitlab.
|
|
35
|
+
"homepage": "https://suffolklitlab.org/docassemble-AssemblyLine-documentation/docs/automated_integrated_testing/",
|
|
36
36
|
"dependencies": {
|
|
37
37
|
"@cucumber/cucumber": "^7.3.2",
|
|
38
38
|
"axios": "^0.24.0",
|
|
@@ -83,3 +83,21 @@ Scenario: I set text-type values
|
|
|
83
83
|
And I tap to continue
|
|
84
84
|
# Next page
|
|
85
85
|
Then the question id should be "the end"
|
|
86
|
+
|
|
87
|
+
@fast @i2 @secret
|
|
88
|
+
Scenario: handles settings from Github secrets
|
|
89
|
+
Given the final Scenario status should be "failed"
|
|
90
|
+
And the Scenario report should include:
|
|
91
|
+
"""
|
|
92
|
+
The GitHub SECRET "SECRET_NOT_THERE" doesn't exist
|
|
93
|
+
"""
|
|
94
|
+
Given I start the interview at "test_secret_vars"
|
|
95
|
+
When I set the var "first_text_entry" to the secret "SECRET_VAR1"
|
|
96
|
+
And I tap to continue
|
|
97
|
+
Then I see the phrase "secret-var1-value"
|
|
98
|
+
When I tap to continue
|
|
99
|
+
And I set the var "second_text_entry" to the github secret "SECRET_VAR2"
|
|
100
|
+
And I tap to continue
|
|
101
|
+
Then I see the phrase "secret-var2-value"
|
|
102
|
+
When I tap to continue
|
|
103
|
+
And I set the variable "third_text_entry" to secret "SECRET_NOT_THERE"
|
|
@@ -376,6 +376,27 @@ Scenario: Fail with 2 wrong signin secrets
|
|
|
376
376
|
"""
|
|
377
377
|
Given I sign in with the email "WRONG_EMAIL_NAME" and the password "WRONG_PASSWORD_NAME"
|
|
378
378
|
|
|
379
|
+
@fast @rf25 @upload
|
|
380
|
+
Scenario: Fail with could not find files
|
|
381
|
+
Given the final Scenario status should be "failed"
|
|
382
|
+
Given the Scenario report should include:
|
|
383
|
+
"""
|
|
384
|
+
Could not find "nonexistant1.pdf, nonexistant2.pdf"
|
|
385
|
+
"""
|
|
386
|
+
Given I start the interview at "all_tests"
|
|
387
|
+
And I upload "nonexistant1.pdf, nonexistant2.pdf" to "upload_files_visible"
|
|
388
|
+
|
|
389
|
+
@fast @rf26 @secret @error
|
|
390
|
+
Scenario: Fail to find var while keeping value secret
|
|
391
|
+
Given the final Scenario status should be "failed"
|
|
392
|
+
Given the Scenario report should include:
|
|
393
|
+
"""
|
|
394
|
+
Did not find a field on this page for the variable "missing_var" that could be set to "SECRET_FOR_MISSING_FIELD"
|
|
395
|
+
"""
|
|
396
|
+
And I start the interview at "test_secret_vars"
|
|
397
|
+
And I set the var "missing_var" to the secret "SECRET_FOR_MISSING_FIELD"
|
|
398
|
+
|
|
399
|
+
|
|
379
400
|
# scope.js
|
|
380
401
|
# I upload "___" to "___"
|
|
381
402
|
# No need to test wrong var name
|
|
@@ -384,6 +405,7 @@ Scenario: Fail with 2 wrong signin secrets
|
|
|
384
405
|
#
|
|
385
406
|
# I sign
|
|
386
407
|
|
|
408
|
+
|
|
387
409
|
# ===============================
|
|
388
410
|
# Reports for Scenarios with warnings
|
|
389
411
|
# ===============================
|
|
@@ -398,6 +420,7 @@ Scenario: Warn when there are too many names
|
|
|
398
420
|
And I set the name of "users[0]" to "Uli Udo User Sampson Jr"
|
|
399
421
|
And I tap to continue
|
|
400
422
|
|
|
423
|
+
|
|
401
424
|
# ===============================
|
|
402
425
|
# Reports for "passed" Scenarios
|
|
403
426
|
# ===============================
|