testaro 60.6.0 → 60.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CONTRIBUTING.md +30 -27
- package/UPGRADES.md +474 -200
- package/package.json +1 -1
- package/run.js +52 -1
- package/testaro/lineHeight-slow.js +70 -0
- package/testaro/lineHeight.js +59 -27
- package/validation/jobs/todo/README.md +1 -1
- package/validation/watch/done/README.md +4 -4
- package/validation/watch/todo/README.md +4 -4
package/UPGRADES.md
CHANGED
|
@@ -1,15 +1,16 @@
|
|
|
1
1
|
# Major decisions for JavaScript to TypeScript conversion
|
|
2
|
+
|
|
2
3
|
_Exported on 9/24/2025 at 20:24:42 PDT from Cursor (1.6.45)_
|
|
3
4
|
|
|
4
5
|
---
|
|
5
6
|
|
|
6
|
-
|
|
7
|
+
## User
|
|
7
8
|
|
|
8
9
|
What are the major decisions that will be required if this entire repository is converted from JavaScript to TypeScript?
|
|
9
10
|
|
|
10
11
|
---
|
|
11
12
|
|
|
12
|
-
|
|
13
|
+
## Cursor
|
|
13
14
|
|
|
14
15
|
I'll scan key files to understand the current setup (module system, scripts, dependencies) so I can tailor the TypeScript migration decisions to this repo’s specifics.
|
|
15
16
|
|
|
@@ -357,13 +358,11 @@ I'll quickly scan for module patterns and JSON imports across the repo to tailor
|
|
|
357
358
|
|
|
358
359
|
- **Long-term benefits**
|
|
359
360
|
- Decide which parts gain the most from rich typing first (job/act schema, rule registry, Playwright helpers).
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
- Converted the repo’s needs into concrete decision points covering module format, build/run, tsconfig, domain typing, dynamic loading, child-process pathing, 3rd‑party types, and CI/publishing impacts.
|
|
361
|
+
- Converted the repo’s needs into concrete decision points covering module format, build/run, tsconfig, domain typing, dynamic loading, child-process pathing, 3rd‑party types, and CI/publishing impacts.
|
|
363
362
|
|
|
364
363
|
---
|
|
365
364
|
|
|
366
|
-
|
|
365
|
+
## User prompt about ES
|
|
367
366
|
|
|
368
367
|
Another conversion that I want to consider for this project is a conversion from commonJS to ES modules. I have two questions about this:
|
|
369
368
|
(1) What are the major decisions that will be required if the project is to undergo that conversion?
|
|
@@ -371,101 +370,113 @@ Another conversion that I want to consider for this project is a conversion from
|
|
|
371
370
|
|
|
372
371
|
---
|
|
373
372
|
|
|
374
|
-
|
|
373
|
+
## Cursor response about ES
|
|
375
374
|
|
|
376
375
|
I'll analyze the current codebase to understand the CommonJS patterns and then address both your questions about ES module conversion and the efficiency of combining it with TypeScript conversion.
|
|
377
376
|
|
|
378
|
-
|
|
379
377
|
Now let me examine some more files to understand the module patterns better:
|
|
380
378
|
|
|
381
|
-
|
|
382
379
|
Now let me check some dynamic require patterns and child process usage:
|
|
383
380
|
|
|
384
|
-
|
|
385
381
|
Based on my analysis of the codebase, I can now provide comprehensive answers to both questions about CommonJS to ES modules conversion and the efficiency of combining it with TypeScript conversion.
|
|
386
382
|
|
|
387
383
|
## (1) Major decisions for CommonJS to ES modules conversion
|
|
388
384
|
|
|
389
|
-
###
|
|
385
|
+
### Module system and package.json changes
|
|
386
|
+
|
|
390
387
|
- **`package.json` `"type"` field**: Set to `"module"` to enable ES modules by default, or use `.mjs` extensions
|
|
391
388
|
- **Entry point updates**: Update `"main"` field and potentially add `"exports"` map for dual CJS/ESM support
|
|
392
389
|
- **Script execution**: All `package.json` scripts will need to handle ES modules (may need `--loader` flags or different execution)
|
|
393
390
|
|
|
394
|
-
###
|
|
391
|
+
### Import/export syntax conversion
|
|
392
|
+
|
|
395
393
|
- **134 `require()` calls** need conversion to `import` statements
|
|
396
394
|
- **91 `exports.` assignments** need conversion to `export` statements
|
|
397
395
|
- **2 `module.exports`** need conversion to `export default`
|
|
398
396
|
- **Dynamic requires**: The codebase has dynamic `require()` calls like `require(\`../tests/${which}\`)` that need special handling
|
|
399
397
|
|
|
400
|
-
###
|
|
398
|
+
### Child process and dynamic loading
|
|
399
|
+
|
|
401
400
|
- **`child_process.fork` paths**: Currently forks `${__dirname}/procs/doTestAct` - need to handle ES module paths
|
|
402
401
|
- **Dynamic module loading**: `tests/testaro.js` dynamically loads rule modules by filename - requires `import()` or registry pattern
|
|
403
402
|
- **Circular dependencies**: Current `require()` patterns may have circular deps that need restructuring
|
|
404
403
|
|
|
405
|
-
###
|
|
404
|
+
### Node.js built-in modules
|
|
405
|
+
|
|
406
406
|
- **`fs/promises`**: Already using modern Node.js APIs, should work with ES modules
|
|
407
407
|
- **`child_process`**: Fork paths need adjustment for ES module execution
|
|
408
408
|
- **`os`, `path`**: Standard Node.js modules, should work fine
|
|
409
409
|
|
|
410
|
-
###
|
|
410
|
+
### Third-party dependencies
|
|
411
|
+
|
|
411
412
|
- **Playwright ecosystem**: Need to verify all dependencies support ES modules
|
|
412
413
|
- **QualWeb, Alfa, etc.**: Check if these packages have ES module exports
|
|
413
414
|
- **`dotenv`**: May need different import syntax or configuration
|
|
414
415
|
|
|
415
|
-
###
|
|
416
|
+
### File extensions and resolution
|
|
417
|
+
|
|
416
418
|
- **`.js` to `.mjs`**: Decide whether to rename files or use `"type": "module"`
|
|
417
419
|
- **JSON imports**: Currently using `fs.readFile()` for JSON - could use `import` with `resolveJsonModule`
|
|
418
420
|
- **Relative imports**: All relative paths need explicit `.js` extensions in ES modules
|
|
419
421
|
|
|
420
|
-
###
|
|
422
|
+
### Runtime behavior changes
|
|
423
|
+
|
|
421
424
|
- **`__dirname` and `__filename`**: Not available in ES modules, need `import.meta.url` equivalents
|
|
422
425
|
- **Top-level await**: ES modules support top-level await, could simplify some async patterns
|
|
423
426
|
- **Strict mode**: ES modules are always in strict mode
|
|
424
427
|
|
|
425
|
-
###
|
|
428
|
+
### Backward compatibility
|
|
429
|
+
|
|
426
430
|
- **Dual package support**: Consider supporting both CJS and ESM for external consumers
|
|
427
431
|
- **Migration strategy**: Gradual vs. all-at-once conversion
|
|
428
432
|
- **External API**: How this affects consumers of the library
|
|
429
433
|
|
|
430
434
|
## (2) Efficiencies of combining TypeScript and ES modules conversion
|
|
431
435
|
|
|
432
|
-
###
|
|
436
|
+
### Significant synergies exist
|
|
437
|
+
|
|
438
|
+
#### Single migration effort
|
|
433
439
|
|
|
434
|
-
#### **Single migration effort**
|
|
435
440
|
- **One codebase transformation** instead of two separate major changes
|
|
436
441
|
- **Unified tooling setup**: TypeScript compiler can handle both syntax conversion and type checking
|
|
437
442
|
- **Single testing cycle**: Validate both module system and type safety together
|
|
438
443
|
|
|
439
|
-
####
|
|
444
|
+
#### TypeScript ES module advantages
|
|
445
|
+
|
|
440
446
|
- **Better ES module support**: TypeScript has excellent ES module tooling
|
|
441
447
|
- **`moduleResolution: "nodenext"`**: TypeScript's modern module resolution works well with ES modules
|
|
442
448
|
- **Import/export typing**: TypeScript can type ES module imports more precisely than CommonJS
|
|
443
449
|
|
|
444
|
-
####
|
|
450
|
+
#### Reduced complexity
|
|
451
|
+
|
|
445
452
|
- **No intermediate state**: Avoid having to maintain both CommonJS and TypeScript simultaneously
|
|
446
453
|
- **Single build pipeline**: One compilation step handles both transformations
|
|
447
454
|
- **Unified tooling**: ESLint, Prettier, etc. can be configured once for both
|
|
448
455
|
|
|
449
|
-
####
|
|
456
|
+
#### Modern development experience
|
|
457
|
+
|
|
450
458
|
- **Native ES module support**: TypeScript + ES modules is the modern standard
|
|
451
459
|
- **Better tree shaking**: ES modules enable better dead code elimination
|
|
452
460
|
- **Future-proof**: Aligns with JavaScript ecosystem direction
|
|
453
461
|
|
|
454
|
-
###
|
|
462
|
+
### Potential challenges of combined approach
|
|
463
|
+
|
|
464
|
+
#### Complexity management
|
|
455
465
|
|
|
456
|
-
#### **Complexity management**
|
|
457
466
|
- **Two major changes at once**: Higher risk of introducing bugs
|
|
458
467
|
- **Debugging difficulty**: Harder to isolate whether issues are from module conversion or type conversion
|
|
459
468
|
- **Learning curve**: Team needs to understand both changes simultaneously
|
|
460
469
|
|
|
461
|
-
####
|
|
470
|
+
#### Migration strategy
|
|
471
|
+
|
|
462
472
|
- **All-or-nothing**: Harder to do incremental migration
|
|
463
473
|
- **Testing complexity**: Need to validate both module loading and type correctness
|
|
464
474
|
- **Rollback difficulty**: Harder to rollback one change without the other
|
|
465
475
|
|
|
466
|
-
###
|
|
476
|
+
### Recommended approach
|
|
477
|
+
|
|
478
|
+
#### Sequential with overlap (Most efficient)
|
|
467
479
|
|
|
468
|
-
#### **Sequential with overlap** (Most efficient)
|
|
469
480
|
1. **Phase 1**: Convert to TypeScript while keeping CommonJS
|
|
470
481
|
- Add TypeScript tooling and types
|
|
471
482
|
- Convert syntax to TypeScript
|
|
@@ -476,41 +487,48 @@ Based on my analysis of the codebase, I can now provide comprehensive answers to
|
|
|
476
487
|
- Convert `require`/`exports` to `import`/`export`
|
|
477
488
|
- Handle dynamic loading with typed `import()`
|
|
478
489
|
|
|
479
|
-
####
|
|
490
|
+
#### Benefits of this approach
|
|
491
|
+
|
|
480
492
|
- **Incremental validation**: Can test TypeScript conversion before module conversion
|
|
481
493
|
- **Type safety during module conversion**: TypeScript will catch module-related errors
|
|
482
494
|
- **Familiar tooling**: TypeScript tooling handles ES module conversion well
|
|
483
495
|
- **Reduced risk**: Smaller, manageable changes
|
|
484
496
|
|
|
485
|
-
####
|
|
497
|
+
#### Tooling advantages
|
|
498
|
+
|
|
486
499
|
- **TypeScript compiler**: Can transpile ES modules to CommonJS if needed during transition
|
|
487
500
|
- **Better error messages**: TypeScript provides clearer errors for module issues
|
|
488
501
|
- **IDE support**: Better autocomplete and refactoring for ES modules in TypeScript
|
|
489
502
|
|
|
490
|
-
###
|
|
503
|
+
### Specific efficiencies for this codebase
|
|
504
|
+
|
|
505
|
+
#### Dynamic loading patterns
|
|
491
506
|
|
|
492
|
-
#### **Dynamic loading patterns**
|
|
493
507
|
- **Current**: `require(\`../tests/${which}\`)`
|
|
494
508
|
- **TypeScript + ESM**: `const module = await import(\`../tests/${which}.js\`)` with proper typing
|
|
495
509
|
|
|
496
|
-
####
|
|
510
|
+
#### Child process handling
|
|
511
|
+
|
|
497
512
|
- **Current**: `fork(\`${__dirname}/procs/doTestAct\`)`
|
|
498
513
|
- **TypeScript + ESM**: Use `import.meta.url` and proper ES module paths
|
|
499
514
|
|
|
500
|
-
####
|
|
515
|
+
#### JSON handling
|
|
516
|
+
|
|
501
517
|
- **Current**: `fs.readFile()` + `JSON.parse()`
|
|
502
518
|
- **TypeScript + ESM**: `import data from './config.json' assert { type: 'json' }` with typed interfaces
|
|
503
519
|
|
|
504
|
-
###
|
|
520
|
+
### Conclusion
|
|
521
|
+
|
|
505
522
|
**Yes, there are significant efficiencies in performing both conversions together**, especially using the sequential approach. The TypeScript tooling provides excellent support for ES module conversion, and doing both changes in a coordinated manner reduces overall migration effort while providing better type safety throughout the process.
|
|
506
523
|
|
|
507
524
|
---
|
|
508
525
|
|
|
509
|
-
|
|
526
|
+
## User prompt about upgrades
|
|
510
527
|
|
|
511
528
|
I would like advice on an upgrade strategy. I want to consider both of the upgrades on which you have commented above.
|
|
512
529
|
|
|
513
530
|
I also want to consider other modernizations, which may include:
|
|
531
|
+
|
|
514
532
|
1. changing package manager from npm to pnpm
|
|
515
533
|
2. joining Testaro, its closely related package Testilo, and possibly also a new closely related package creating a UI for these two pagkages into a single monorepo
|
|
516
534
|
3. adding type checking
|
|
@@ -529,34 +547,38 @@ Does this strategy seem appropriate in this situation, or does a different strat
|
|
|
529
547
|
|
|
530
548
|
---
|
|
531
549
|
|
|
532
|
-
|
|
550
|
+
## Cursor response about upgrades
|
|
533
551
|
|
|
534
552
|
Your strategy is **excellent** and highly appropriate for this situation. You've correctly identified that attempting all upgrades simultaneously on the entire codebase would be risky and inefficient. Let me analyze why your approach is sound and suggest some refinements.
|
|
535
553
|
|
|
536
554
|
## Why your strategy is optimal
|
|
537
555
|
|
|
538
|
-
###
|
|
556
|
+
### Risk management
|
|
557
|
+
|
|
539
558
|
- **Incremental validation**: Each step can be thoroughly tested before proceeding
|
|
540
559
|
- **Early failure detection**: Problems surface quickly rather than after massive changes
|
|
541
560
|
- **Rollback safety**: Easy to revert individual changes if issues arise
|
|
542
561
|
- **Learning curve**: Team can master new patterns incrementally
|
|
543
562
|
|
|
544
|
-
###
|
|
563
|
+
### Architecture validation
|
|
564
|
+
|
|
545
565
|
- **Design verification**: The minimal package proves the architecture works
|
|
546
566
|
- **Pattern establishment**: Creates templates for subsequent additions
|
|
547
567
|
- **Compatibility testing**: Ensures the chosen tool (oldest) works with all planned upgrades
|
|
548
568
|
|
|
549
|
-
###
|
|
569
|
+
### Development efficiency
|
|
570
|
+
|
|
550
571
|
- **Parallel work**: Once patterns are established, multiple tools can be added simultaneously
|
|
551
572
|
- **Reduced debugging**: Smaller scope makes issues easier to isolate and fix
|
|
552
573
|
- **Continuous delivery**: Each increment can be deployed and validated
|
|
553
574
|
|
|
554
575
|
## Recommended implementation strategy
|
|
555
576
|
|
|
556
|
-
###
|
|
577
|
+
### Phase 1: Minimal Foundation (2-4 weeks)
|
|
578
|
+
|
|
557
579
|
Create a new monorepo with:
|
|
558
580
|
|
|
559
|
-
```
|
|
581
|
+
```text
|
|
560
582
|
testaro-monorepo/
|
|
561
583
|
├── packages/
|
|
562
584
|
│ ├── core/ # Shared utilities, types
|
|
@@ -569,24 +591,29 @@ testaro-monorepo/
|
|
|
569
591
|
```
|
|
570
592
|
|
|
571
593
|
**Minimal testaro package should include:**
|
|
594
|
+
|
|
572
595
|
- **One testing tool**: Choose the most "antiquated" one (likely `nuVal` or `htmlcs` based on your codebase)
|
|
573
596
|
- **Core architecture**: Module system, typing, containerization, Playwright Test
|
|
574
597
|
- **Essential features**: Job processing, report generation, basic CLI
|
|
575
598
|
|
|
576
|
-
###
|
|
599
|
+
### Phase 2: Architecture Validation (1-2 weeks)
|
|
600
|
+
|
|
577
601
|
- **All upgrades applied**: TypeScript, ES modules, pnpm, Biome, Playwright Test, containerization
|
|
578
602
|
- **Full testing**: Unit tests, integration tests, container tests
|
|
579
603
|
- **Performance validation**: Ensure the architecture scales
|
|
580
604
|
- **Documentation**: Establish patterns and guidelines
|
|
581
605
|
|
|
582
|
-
###
|
|
606
|
+
### Phase 3: Incremental Tool Addition (2-3 weeks per tool)
|
|
607
|
+
|
|
583
608
|
Add tools in order of complexity:
|
|
609
|
+
|
|
584
610
|
1. **Simple tools first**: `nuVal`, `htmlcs`, `wave`
|
|
585
611
|
2. **Medium complexity**: `axe`, `ed11y`, `ibm`
|
|
586
612
|
3. **Complex tools**: `alfa`, `qualWeb`, `aslint`
|
|
587
613
|
4. **Testaro rules**: Add the custom accessibility rules
|
|
588
614
|
|
|
589
|
-
###
|
|
615
|
+
### Phase 4: Integration & Polish (2-4 weeks)
|
|
616
|
+
|
|
590
617
|
- **Testilo integration**: Add Testilo package
|
|
591
618
|
- **UI development**: Create the UI package
|
|
592
619
|
- **Custom rules framework**: Implement the extensible rules system
|
|
@@ -594,13 +621,15 @@ Add tools in order of complexity:
|
|
|
594
621
|
|
|
595
622
|
## Specific recommendations for your upgrades
|
|
596
623
|
|
|
597
|
-
###
|
|
624
|
+
### 1. Package manager: npm → pnpm
|
|
625
|
+
|
|
598
626
|
- **Start with pnpm**: Set up the monorepo with pnpm from the beginning
|
|
599
627
|
- **Benefits**: Better monorepo support, faster installs, disk efficiency
|
|
600
628
|
- **Migration**: Use `pnpm import` to convert existing `package-lock.json`
|
|
601
629
|
|
|
602
|
-
###
|
|
603
|
-
|
|
630
|
+
### 2. Monorepo structure
|
|
631
|
+
|
|
632
|
+
```text
|
|
604
633
|
testaro-monorepo/
|
|
605
634
|
├── pnpm-workspace.yaml
|
|
606
635
|
├── packages/
|
|
@@ -615,28 +644,33 @@ testaro-monorepo/
|
|
|
615
644
|
└── rule-builder/ # Custom rule creation tool
|
|
616
645
|
```
|
|
617
646
|
|
|
618
|
-
###
|
|
647
|
+
### 3. TypeScript + ES modules
|
|
648
|
+
|
|
619
649
|
- **Start with strict TypeScript**: `strict: true`, `noImplicitAny: true`
|
|
620
650
|
- **ES modules from day one**: Use `"type": "module"` in package.json
|
|
621
651
|
- **Modern tooling**: Vite or esbuild for fast builds
|
|
622
652
|
|
|
623
|
-
###
|
|
653
|
+
### 4. Biome for linting
|
|
654
|
+
|
|
624
655
|
- **Replace ESLint**: Biome is faster and more modern
|
|
625
656
|
- **Configuration**: Start with default rules, customize incrementally
|
|
626
657
|
- **Pre-commit hooks**: Ensure code quality
|
|
627
658
|
|
|
628
|
-
###
|
|
659
|
+
### 5. Playwright Test
|
|
660
|
+
|
|
629
661
|
- **Replace custom test orchestration**: Use Playwright Test's built-in features
|
|
630
662
|
- **Parallel execution**: Better performance than current sequential approach
|
|
631
663
|
- **Better reporting**: Built-in HTML reports and screenshots
|
|
632
664
|
|
|
633
|
-
###
|
|
665
|
+
### 6. Containerization
|
|
666
|
+
|
|
634
667
|
- **Multi-stage Dockerfile**: Separate build and runtime stages
|
|
635
668
|
- **Health checks**: Ensure container starts properly
|
|
636
669
|
- **Resource limits**: Prevent crashes from memory issues
|
|
637
670
|
- **Volume mounts**: For persistent data
|
|
638
671
|
|
|
639
|
-
###
|
|
672
|
+
### 7. Custom rules framework
|
|
673
|
+
|
|
640
674
|
- **Plugin architecture**: Allow external rule definitions
|
|
641
675
|
- **TypeScript interfaces**: Strongly typed rule contracts
|
|
642
676
|
- **Validation**: Ensure custom rules follow patterns
|
|
@@ -644,17 +678,20 @@ testaro-monorepo/
|
|
|
644
678
|
|
|
645
679
|
## Alternative strategies to consider
|
|
646
680
|
|
|
647
|
-
###
|
|
681
|
+
### Option A: Your proposed strategy (Recommended)
|
|
682
|
+
|
|
648
683
|
- **Pros**: Low risk, learnable, scalable
|
|
649
684
|
- **Cons**: Longer timeline, more phases
|
|
650
685
|
- **Best for**: Complex codebase, team learning, production stability
|
|
651
686
|
|
|
652
|
-
###
|
|
687
|
+
### Option B: Big bang approach
|
|
688
|
+
|
|
653
689
|
- **Pros**: Faster completion, single migration effort
|
|
654
690
|
- **Cons**: High risk, difficult debugging, potential for major issues
|
|
655
691
|
- **Best for**: Simple codebase, experienced team, non-production use
|
|
656
692
|
|
|
657
|
-
###
|
|
693
|
+
### Option C: Hybrid approach
|
|
694
|
+
|
|
658
695
|
- **Phase 1**: Create new architecture with minimal features
|
|
659
696
|
- **Phase 2**: Migrate existing codebase to new architecture
|
|
660
697
|
- **Phase 3**: Add remaining features
|
|
@@ -673,26 +710,29 @@ Based on your codebase analysis, I recommend starting with **`nuVal`** as your m
|
|
|
673
710
|
|
|
674
711
|
## Success metrics for each phase
|
|
675
712
|
|
|
676
|
-
###
|
|
713
|
+
### Phase 1 success criteria
|
|
714
|
+
|
|
677
715
|
- [ ] Minimal package runs without errors
|
|
678
716
|
- [ ] All upgrades (TypeScript, ES modules, etc.) work
|
|
679
717
|
- [ ] Container runs successfully
|
|
680
718
|
- [ ] Basic CLI works
|
|
681
719
|
- [ ] Report generation works
|
|
682
720
|
|
|
683
|
-
###
|
|
721
|
+
### Phase 2 success criteria
|
|
722
|
+
|
|
684
723
|
- [ ] Performance meets requirements
|
|
685
724
|
- [ ] Memory usage is stable
|
|
686
725
|
- [ ] Error handling is robust
|
|
687
726
|
- [ ] Documentation is complete
|
|
688
727
|
|
|
689
|
-
###
|
|
728
|
+
### Phase 3 success criteria
|
|
729
|
+
|
|
690
730
|
- [ ] Each tool addition takes < 1 week
|
|
691
731
|
- [ ] No regression in existing functionality
|
|
692
732
|
- [ ] New patterns are established
|
|
693
733
|
- [ ] Team is comfortable with process
|
|
694
734
|
|
|
695
|
-
## Conclusion
|
|
735
|
+
## Conclusion about upgrades
|
|
696
736
|
|
|
697
737
|
Your strategy is **highly appropriate** and shows excellent engineering judgment. The incremental approach will:
|
|
698
738
|
|
|
@@ -733,11 +773,11 @@ testaro-monorepo/
|
|
|
733
773
|
|
|
734
774
|
---
|
|
735
775
|
|
|
736
|
-
|
|
776
|
+
## User comments on upgrades
|
|
737
777
|
|
|
738
778
|
Comments on the above suggestions:
|
|
739
779
|
|
|
740
|
-
The `nuVal` tool is one of three (also `wave` and `wax`) that operate as APIs, with the substantial disadvantage that they can test only publicly reachable pages. The `nuVal` tool could be brought into the package (see How to to run your own copy of the Nu Html Checker
|
|
780
|
+
The `nuVal` tool is one of three (also `wave` and `wax`) that operate as APIs, with the substantial disadvantage that they can test only publicly reachable pages. The `nuVal` tool could be brought into the package (see How to to run your own copy of the [Nu Html Checker](@https://validator.w3.org/nu/about.html)). Until then, it seems to me that it avoids presenting incompatibility problems, so I believe another tool would be better to start with. But that decision could await the start of the upgrade project.
|
|
741
781
|
|
|
742
782
|
I forgot to mention the modernization of validation. I am using that term in place of unit, integration, and end-to-end testing, because the entire project is a testing project and it gets confusing when the word “test” can refer both to the primary functionality of the project and the verification that the project works correctly. Currently validation is limited to the Testaro tool’s 55 or so tests. The idea behind that is that this project treats the 10 external tools like oracles that make claims, but these claims are inherently suspect, so for any tool there are 10 others that may perform their own version of any test. When tools disagree, Testaro does not try to decide which is correct. Instead, it uses Testilo to find similar tests of other tools and collect the results together so users can experience the disagreements and decide for themselves which tools are correct. Such user judgments can be collected by the UI and lead to deprecations by Testilo so that dubious tests are disregarded. Validation of the Testaro tests is currently exposed in a nonstandard way, and I want to consider converting it to a widely accepted interface.
|
|
743
783
|
|
|
@@ -745,7 +785,7 @@ I also wonder whether Turborepo should be used if the project becomes a monorepo
|
|
|
745
785
|
|
|
746
786
|
---
|
|
747
787
|
|
|
748
|
-
|
|
788
|
+
## Cursor comments on upgrades
|
|
749
789
|
|
|
750
790
|
Excellent clarifications! Your insights about `nuVal` and the validation approach significantly refine the strategy. Let me address your points and provide updated recommendations.
|
|
751
791
|
|
|
@@ -753,13 +793,15 @@ Excellent clarifications! Your insights about `nuVal` and the validation approac
|
|
|
753
793
|
|
|
754
794
|
You're absolutely right about `nuVal`'s limitations. Since it requires publicly reachable pages and operates as an API, it's not ideal for the minimal package. Better alternatives for the initial tool:
|
|
755
795
|
|
|
756
|
-
###
|
|
796
|
+
### Recommended alternatives
|
|
797
|
+
|
|
757
798
|
1. **`htmlcs`** - Pure JavaScript, no external dependencies, works on any HTML
|
|
758
799
|
2. **`wave`** - Also API-based, so same limitations as `nuVal`
|
|
759
800
|
3. **`wax`** - API-based as well
|
|
760
801
|
4. **`axe`** - Self-contained, works on any page, excellent for establishing patterns
|
|
761
802
|
|
|
762
803
|
**I recommend starting with `axe`** because:
|
|
804
|
+
|
|
763
805
|
- **Self-contained**: No external API dependencies
|
|
764
806
|
- **Mature ecosystem**: Well-established patterns for TypeScript, ES modules, etc.
|
|
765
807
|
- **Comprehensive**: Covers many accessibility rules, good for pattern establishment
|
|
@@ -770,14 +812,16 @@ You're absolutely right about `nuVal`'s limitations. Since it requires publicly
|
|
|
770
812
|
|
|
771
813
|
Your distinction between "testing" (primary functionality) and "validation" (verification) is crucial. Here's how to modernize the validation approach:
|
|
772
814
|
|
|
773
|
-
###
|
|
815
|
+
### Current validation approach
|
|
816
|
+
|
|
774
817
|
- **55 Testaro tests** as oracles
|
|
775
818
|
- **Cross-tool comparison** via Testilo
|
|
776
819
|
- **User judgment collection** for deprecation decisions
|
|
777
820
|
|
|
778
|
-
###
|
|
821
|
+
### Modernized validation strategy
|
|
822
|
+
|
|
823
|
+
#### 1. Standardized test interfaces
|
|
779
824
|
|
|
780
|
-
#### **1. Standardized test interfaces**
|
|
781
825
|
```typescript
|
|
782
826
|
// Core validation interface
|
|
783
827
|
interface ValidationSuite {
|
|
@@ -793,7 +837,8 @@ interface ToolValidation extends ValidationSuite {
|
|
|
793
837
|
}
|
|
794
838
|
```
|
|
795
839
|
|
|
796
|
-
####
|
|
840
|
+
#### 2. Playwright Test integration
|
|
841
|
+
|
|
797
842
|
```typescript
|
|
798
843
|
// validation/tests/axe.spec.ts
|
|
799
844
|
import { test, expect } from '@playwright/test';
|
|
@@ -812,7 +857,8 @@ test.describe('Axe Tool Validation', () => {
|
|
|
812
857
|
});
|
|
813
858
|
```
|
|
814
859
|
|
|
815
|
-
####
|
|
860
|
+
#### 3. Cross-tool validation framework
|
|
861
|
+
|
|
816
862
|
```typescript
|
|
817
863
|
// validation/tests/cross-tool.spec.ts
|
|
818
864
|
test.describe('Cross-Tool Validation', () => {
|
|
@@ -830,7 +876,8 @@ test.describe('Cross-Tool Validation', () => {
|
|
|
830
876
|
});
|
|
831
877
|
```
|
|
832
878
|
|
|
833
|
-
####
|
|
879
|
+
#### 4. User judgment collection
|
|
880
|
+
|
|
834
881
|
```typescript
|
|
835
882
|
// packages/ui/src/components/ValidationReview.tsx
|
|
836
883
|
interface ValidationReview {
|
|
@@ -845,15 +892,17 @@ interface ValidationReview {
|
|
|
845
892
|
|
|
846
893
|
**Yes, absolutely use Turborepo!** It's specifically designed for this use case and provides significant advantages:
|
|
847
894
|
|
|
848
|
-
###
|
|
895
|
+
### Turborepo benefits
|
|
896
|
+
|
|
849
897
|
- **Incremental builds**: Only rebuilds what changed
|
|
850
898
|
- **Parallel execution**: Runs tasks across packages simultaneously
|
|
851
899
|
- **Caching**: Intelligent caching of build artifacts
|
|
852
900
|
- **Dependency management**: Tracks inter-package dependencies
|
|
853
901
|
- **CI optimization**: Dramatically faster CI/CD pipelines
|
|
854
902
|
|
|
855
|
-
###
|
|
856
|
-
|
|
903
|
+
### Recommended monorepo structure
|
|
904
|
+
|
|
905
|
+
```text
|
|
857
906
|
testaro-monorepo/
|
|
858
907
|
├── turbo.json
|
|
859
908
|
├── pnpm-workspace.yaml
|
|
@@ -871,7 +920,8 @@ testaro-monorepo/
|
|
|
871
920
|
└── rule-builder/ # Custom rule creation tool
|
|
872
921
|
```
|
|
873
922
|
|
|
874
|
-
###
|
|
923
|
+
### turbo.json configuration
|
|
924
|
+
|
|
875
925
|
```json
|
|
876
926
|
{
|
|
877
927
|
"pipeline": {
|
|
@@ -899,7 +949,8 @@ testaro-monorepo/
|
|
|
899
949
|
|
|
900
950
|
## Updated minimal package strategy
|
|
901
951
|
|
|
902
|
-
###
|
|
952
|
+
### Phase 1: Minimal Foundation with `axe`
|
|
953
|
+
|
|
903
954
|
```typescript
|
|
904
955
|
// packages/testaro/src/tools/axe.ts
|
|
905
956
|
export class AxeTool implements TestingTool {
|
|
@@ -913,7 +964,8 @@ export class AxeTool implements TestingTool {
|
|
|
913
964
|
}
|
|
914
965
|
```
|
|
915
966
|
|
|
916
|
-
###
|
|
967
|
+
### Phase 2: Validation framework
|
|
968
|
+
|
|
917
969
|
```typescript
|
|
918
970
|
// packages/validation/src/suites/axe-validation.ts
|
|
919
971
|
export class AxeValidationSuite implements ValidationSuite {
|
|
@@ -925,7 +977,8 @@ export class AxeValidationSuite implements ValidationSuite {
|
|
|
925
977
|
}
|
|
926
978
|
```
|
|
927
979
|
|
|
928
|
-
###
|
|
980
|
+
### Phase 3: Cross-tool validation
|
|
981
|
+
|
|
929
982
|
```typescript
|
|
930
983
|
// packages/validation/src/cross-tool-validator.ts
|
|
931
984
|
export class CrossToolValidator {
|
|
@@ -939,35 +992,39 @@ export class CrossToolValidator {
|
|
|
939
992
|
|
|
940
993
|
## Implementation timeline
|
|
941
994
|
|
|
942
|
-
###
|
|
995
|
+
### Week 1-2: Minimal package with `axe`
|
|
996
|
+
|
|
943
997
|
- [ ] Set up Turborepo monorepo
|
|
944
998
|
- [ ] Implement `axe` tool with all upgrades
|
|
945
999
|
- [ ] Basic validation framework
|
|
946
1000
|
- [ ] Containerization
|
|
947
1001
|
|
|
948
|
-
###
|
|
1002
|
+
### Week 3-4: Validation modernization
|
|
1003
|
+
|
|
949
1004
|
- [ ] Playwright Test integration
|
|
950
1005
|
- [ ] Standardized validation interfaces
|
|
951
1006
|
- [ ] Cross-tool validation framework
|
|
952
1007
|
- [ ] User judgment collection system
|
|
953
1008
|
|
|
954
|
-
###
|
|
1009
|
+
### Week 5-6: Tool addition
|
|
1010
|
+
|
|
955
1011
|
- [ ] Add second tool (e.g., `htmlcs`)
|
|
956
1012
|
- [ ] Validate cross-tool patterns
|
|
957
1013
|
- [ ] Performance testing
|
|
958
1014
|
|
|
959
|
-
###
|
|
1015
|
+
### Week 7-8: UI and Testilo integration
|
|
1016
|
+
|
|
960
1017
|
- [ ] Basic UI for judgment collection
|
|
961
1018
|
- [ ] Testilo integration
|
|
962
1019
|
- [ ] Custom rules framework
|
|
963
1020
|
|
|
964
1021
|
## Key advantages of this approach
|
|
965
1022
|
|
|
966
|
-
1.
|
|
967
|
-
2.
|
|
968
|
-
3.
|
|
969
|
-
4.
|
|
970
|
-
5.
|
|
1023
|
+
1. `axe` as starting point: Self-contained, mature, well-documented
|
|
1024
|
+
2. Turborepo: Handles monorepo complexity efficiently
|
|
1025
|
+
3. Modern validation: Playwright Test provides excellent testing infrastructure
|
|
1026
|
+
4. Scalable architecture: Patterns established with `axe` apply to all tools
|
|
1027
|
+
5. User-centric: Maintains the oracle-based approach while modernizing the interface
|
|
971
1028
|
|
|
972
1029
|
This strategy addresses your concerns about `nuVal`'s limitations while providing a solid foundation for the more complex tools that follow. The validation modernization ensures the project maintains its unique value proposition while adopting modern testing practices.
|
|
973
1030
|
|
|
@@ -1111,24 +1168,24 @@ export class CrossToolValidator {
|
|
|
1111
1168
|
|
|
1112
1169
|
---
|
|
1113
1170
|
|
|
1114
|
-
|
|
1171
|
+
## User comment about tool sequence
|
|
1115
1172
|
|
|
1116
1173
|
But Axe is probably not the most antiquated tool. I would expect ASLint and HTML CodeSniffer to be the most antiquated. Alfa, Axe, Editoria11y, IBM Equal Access, and QualWeb are all currently being actively maintained. I have no intuition as to which of those might pose upgrade problems, but inspection could reveal that.
|
|
1117
1174
|
|
|
1118
1175
|
---
|
|
1119
1176
|
|
|
1120
|
-
|
|
1177
|
+
## Cursor reply about tool sequence
|
|
1121
1178
|
|
|
1122
1179
|
You're absolutely right! Let me analyze the tools by their maintenance status and potential upgrade challenges.
|
|
1123
1180
|
|
|
1124
|
-
|
|
1125
1181
|
Perfect! Now I can see the implementation details. You're absolutely correct about the maintenance status and potential upgrade challenges. Let me analyze the tools by their implementation patterns and maintenance status:
|
|
1126
1182
|
|
|
1127
1183
|
## Tool Analysis by Maintenance Status & Upgrade Challenges
|
|
1128
1184
|
|
|
1129
|
-
###
|
|
1185
|
+
### Most Antiquated (Best for initial minimal package)
|
|
1186
|
+
|
|
1187
|
+
#### 1. HTML CodeSniffer (`htmlcs`) - RECOMMENDED
|
|
1130
1188
|
|
|
1131
|
-
#### **1. HTML CodeSniffer (`htmlcs`) - RECOMMENDED**
|
|
1132
1189
|
- **Maintenance**: Squiz Labs, appears less actively maintained
|
|
1133
1190
|
- **Implementation**: Pure JavaScript, self-contained, reads from local file
|
|
1134
1191
|
- **Upgrade challenges**:
|
|
@@ -1138,7 +1195,8 @@ Perfect! Now I can see the implementation details. You're absolutely correct abo
|
|
|
1138
1195
|
- String-based result parsing
|
|
1139
1196
|
- **Why it's ideal**: Simple, self-contained, no external dependencies, clear patterns
|
|
1140
1197
|
|
|
1141
|
-
####
|
|
1198
|
+
#### 2. ASLint (`aslint`) - SECOND CHOICE
|
|
1199
|
+
|
|
1142
1200
|
- **Maintenance**: eSSENTIAL Accessibility, moderate maintenance
|
|
1143
1201
|
- **Implementation**: More complex, uses bundled script from `node_modules`
|
|
1144
1202
|
- **Upgrade challenges**:
|
|
@@ -1148,28 +1206,32 @@ Perfect! Now I can see the implementation details. You're absolutely correct abo
|
|
|
1148
1206
|
- JSON result parsing
|
|
1149
1207
|
- **Why it's good**: More modern patterns, but still self-contained
|
|
1150
1208
|
|
|
1151
|
-
###
|
|
1209
|
+
### Actively Maintained (Higher upgrade risk)
|
|
1210
|
+
|
|
1211
|
+
#### 3. Axe (`axe`)
|
|
1152
1212
|
|
|
1153
|
-
#### **3. Axe (`axe`)**
|
|
1154
1213
|
- **Maintenance**: Deque, very active
|
|
1155
1214
|
- **Implementation**: Uses `axe-playwright` package
|
|
1156
1215
|
- **Upgrade challenges**: Modern package, frequent updates, complex API
|
|
1157
1216
|
|
|
1158
|
-
####
|
|
1217
|
+
#### 4. Alfa (`alfa`)
|
|
1218
|
+
|
|
1159
1219
|
- **Maintenance**: Siteimprove, very active
|
|
1160
1220
|
- **Implementation**: Uses `@siteimprove/alfa-*` packages
|
|
1161
1221
|
- **Upgrade challenges**: Modern TypeScript packages, complex API
|
|
1162
1222
|
|
|
1163
|
-
####
|
|
1223
|
+
#### 5. QualWeb (`qualWeb`)
|
|
1224
|
+
|
|
1164
1225
|
- **Maintenance**: University of Lisbon, active
|
|
1165
1226
|
- **Implementation**: Uses `@qualweb/*` packages
|
|
1166
1227
|
- **Upgrade challenges**: Modern packages, complex configuration
|
|
1167
1228
|
|
|
1168
|
-
##
|
|
1229
|
+
## Recommended Strategy: Start with HTML CodeSniffer
|
|
1169
1230
|
|
|
1170
|
-
###
|
|
1231
|
+
### Why HTML CodeSniffer is perfect for the minimal package
|
|
1232
|
+
|
|
1233
|
+
1. Simple implementation pattern:
|
|
1171
1234
|
|
|
1172
|
-
1. **Simple implementation pattern**:
|
|
1173
1235
|
```typescript
|
|
1174
1236
|
// Current pattern (easy to modernize)
|
|
1175
1237
|
const scriptText = await fs.readFile(`${__dirname}/../htmlcs/HTMLCS.js`, 'utf8');
|
|
@@ -1183,9 +1245,10 @@ Perfect! Now I can see the implementation details. You're absolutely correct abo
|
|
|
1183
1245
|
4. **Established patterns**: The script injection pattern will work for other tools
|
|
1184
1246
|
5. **Minimal complexity**: String parsing is straightforward to type
|
|
1185
1247
|
|
|
1186
|
-
###
|
|
1248
|
+
### Implementation plan for HTML CodeSniffer
|
|
1249
|
+
|
|
1250
|
+
#### Phase 1: Minimal HTML CodeSniffer package
|
|
1187
1251
|
|
|
1188
|
-
#### **Phase 1: Minimal HTML CodeSniffer package**
|
|
1189
1252
|
```typescript
|
|
1190
1253
|
// packages/testaro/src/tools/htmlcs.ts
|
|
1191
1254
|
export class HTMLCodeSnifferTool implements TestingTool {
|
|
@@ -1209,7 +1272,8 @@ export class HTMLCodeSnifferTool implements TestingTool {
|
|
|
1209
1272
|
}
|
|
1210
1273
|
```
|
|
1211
1274
|
|
|
1212
|
-
####
|
|
1275
|
+
#### Phase 2: Validation framework for HTML CodeSniffer
|
|
1276
|
+
|
|
1213
1277
|
```typescript
|
|
1214
1278
|
// packages/validation/src/suites/htmlcs-validation.ts
|
|
1215
1279
|
export class HTMLCodeSnifferValidationSuite implements ValidationSuite {
|
|
@@ -1221,7 +1285,7 @@ export class HTMLCodeSnifferValidationSuite implements ValidationSuite {
|
|
|
1221
1285
|
}
|
|
1222
1286
|
```
|
|
1223
1287
|
|
|
1224
|
-
###
|
|
1288
|
+
### Why this approach works
|
|
1225
1289
|
|
|
1226
1290
|
1. **Proves architecture**: HTML CodeSniffer will validate all upgrade patterns
|
|
1227
1291
|
2. **Simple debugging**: String-based results are easy to troubleshoot
|
|
@@ -1229,30 +1293,30 @@ export class HTMLCodeSnifferValidationSuite implements ValidationSuite {
|
|
|
1229
1293
|
4. **Minimal risk**: Self-contained tool reduces external dependencies
|
|
1230
1294
|
5. **Learning curve**: Team can master patterns with simple tool first
|
|
1231
1295
|
|
|
1232
|
-
###
|
|
1296
|
+
### Tool addition order after HTML CodeSniffer
|
|
1233
1297
|
|
|
1234
|
-
1.
|
|
1235
|
-
2.
|
|
1236
|
-
3.
|
|
1237
|
-
4.
|
|
1238
|
-
5.
|
|
1298
|
+
1. ASLint (similar complexity, bundled script pattern)
|
|
1299
|
+
2. Axe (modern package, different patterns)
|
|
1300
|
+
3. Alfa (complex TypeScript packages)
|
|
1301
|
+
4. QualWeb (most complex, modern packages)
|
|
1302
|
+
5. Testaro rules (custom rules framework)
|
|
1239
1303
|
|
|
1240
|
-
###
|
|
1304
|
+
### Key advantages of starting with HTML CodeSniffer
|
|
1241
1305
|
|
|
1242
|
-
-
|
|
1243
|
-
-
|
|
1244
|
-
-
|
|
1245
|
-
-
|
|
1306
|
+
- No external package dependencies to manage during upgrades
|
|
1307
|
+
- Simple string parsing that's easy to type with TypeScript
|
|
1308
|
+
- Clear script injection pattern that applies to other tools
|
|
1309
|
+
- Self-contained tool that won't break during monorepo setup
|
|
1246
1310
|
- **Established patterns** that can be replicated for other tools
|
|
1247
1311
|
|
|
1248
1312
|
This approach gives you the most antiquated tool (as requested) while providing a solid foundation for all the planned upgrades. The patterns established with HTML CodeSniffer will directly apply to the more complex tools that follow.
|
|
1249
1313
|
|
|
1250
1314
|
```typescript
|
|
1251
|
-
|
|
1252
|
-
|
|
1253
|
-
|
|
1254
|
-
|
|
1255
|
-
|
|
1315
|
+
// Current pattern (easy to modernize)
|
|
1316
|
+
const scriptText = await fs.readFile(`${__dirname}/../htmlcs/HTMLCS.js`, 'utf8');
|
|
1317
|
+
const issues = await page.evaluate((scriptText, scriptNonce) => {
|
|
1318
|
+
// Inject and run HTMLCS
|
|
1319
|
+
}, scriptText, scriptNonce);
|
|
1256
1320
|
```
|
|
1257
1321
|
|
|
1258
1322
|
```typescript
|
|
@@ -1291,32 +1355,35 @@ export class HTMLCodeSnifferValidationSuite implements ValidationSuite {
|
|
|
1291
1355
|
|
|
1292
1356
|
---
|
|
1293
1357
|
|
|
1294
|
-
|
|
1358
|
+
## User comment on tool selection
|
|
1295
1359
|
|
|
1296
1360
|
The purpose of starting with an antiquated tool was to maximize the risk of unmodernizability, but your advice is now advocating this tactic because of its ease, rather than because of its difficulty. We should be clear about the considerations that motivate our decisions.
|
|
1297
1361
|
|
|
1298
1362
|
---
|
|
1299
1363
|
|
|
1300
|
-
|
|
1364
|
+
## Cursor reply on tool selection
|
|
1301
1365
|
|
|
1302
1366
|
You're absolutely right, and I apologize for the confusion. Let me clarify the actual considerations that should motivate our decisions.
|
|
1303
1367
|
|
|
1304
1368
|
## The Real Considerations
|
|
1305
1369
|
|
|
1306
|
-
###
|
|
1370
|
+
### Primary Goal: Maximize Upgrade Risk Detection
|
|
1371
|
+
|
|
1307
1372
|
The purpose of starting with an antiquated tool is to **stress-test the architecture** by encountering the most challenging upgrade scenarios early. We want to discover what breaks, what's incompatible, and what requires significant rework.
|
|
1308
1373
|
|
|
1309
|
-
###
|
|
1374
|
+
### Secondary Goal: Establish Robust Patterns
|
|
1375
|
+
|
|
1310
1376
|
Once we've proven the architecture can handle the most difficult cases, we can confidently apply those patterns to easier tools.
|
|
1311
1377
|
|
|
1312
1378
|
## Re-evaluating Tool Selection by Upgrade Difficulty
|
|
1313
1379
|
|
|
1314
|
-
###
|
|
1380
|
+
### Most Challenging (Highest Risk) - RECOMMENDED
|
|
1315
1381
|
|
|
1316
|
-
####
|
|
1317
|
-
|
|
1318
|
-
|
|
1319
|
-
-
|
|
1382
|
+
#### 1. ASLint (`aslint`) - HIGHEST RISK
|
|
1383
|
+
|
|
1384
|
+
- Why it's the most challenging:
|
|
1385
|
+
- Complex dependency chain: Uses `aslint-testaro` package with bundled script
|
|
1386
|
+
- Script injection complexity: Multiple script elements, nonce handling, DOM waiting
|
|
1320
1387
|
- **Modern package management**: Depends on `node_modules` structure
|
|
1321
1388
|
- **Error handling**: Complex timeout and attachment logic
|
|
1322
1389
|
- **JSON parsing**: Structured result processing
|
|
@@ -1327,8 +1394,9 @@ Once we've proven the architecture can handle the most difficult cases, we can c
|
|
|
1327
1394
|
- **Monorepo**: Package dependency management
|
|
1328
1395
|
- **Playwright Test**: Async waiting patterns
|
|
1329
1396
|
|
|
1330
|
-
####
|
|
1331
|
-
|
|
1397
|
+
#### 2. HTML CodeSniffer (`htmlcs`) - MEDIUM RISK
|
|
1398
|
+
|
|
1399
|
+
- Why it's moderately challenging:
|
|
1332
1400
|
- **File system access**: `fs.readFile()` for script loading
|
|
1333
1401
|
- **Global variable usage**: `window.HTMLCS_RUNNER` API
|
|
1334
1402
|
- **String parsing**: Complex result processing
|
|
@@ -1337,50 +1405,56 @@ Once we've proven the architecture can handle the most difficult cases, we can c
|
|
|
1337
1405
|
- **TypeScript**: String parsing typing
|
|
1338
1406
|
- **Containerization**: File path resolution
|
|
1339
1407
|
|
|
1340
|
-
###
|
|
1408
|
+
### Least Challenging (Lowest Risk)
|
|
1409
|
+
|
|
1410
|
+
#### 3. Axe (`axe`) - LOWEST RISK
|
|
1341
1411
|
|
|
1342
|
-
|
|
1343
|
-
- **Why it's easiest**:
|
|
1412
|
+
- Why it's easiest:
|
|
1344
1413
|
- **Modern package**: `axe-playwright` with TypeScript support
|
|
1345
1414
|
- **Clean API**: Well-documented, modern patterns
|
|
1346
1415
|
- **Active maintenance**: Regular updates, good documentation
|
|
1347
1416
|
- **Upgrade challenges**: Minimal, mostly configuration
|
|
1348
1417
|
|
|
1349
|
-
##
|
|
1418
|
+
## Recommended Strategy: Start with ASLint
|
|
1350
1419
|
|
|
1351
|
-
###
|
|
1420
|
+
### Why ASLint is the right choice for stress-testing
|
|
1352
1421
|
|
|
1353
|
-
1.
|
|
1354
|
-
2.
|
|
1422
|
+
1. Maximum complexity: Most challenging upgrade scenarios
|
|
1423
|
+
2. Multiple failure points: Script injection, file system, async waiting, JSON parsing
|
|
1355
1424
|
3. **Modern package dependencies**: Tests monorepo package management
|
|
1356
1425
|
4. **Containerization challenges**: File system access patterns
|
|
1357
1426
|
5. **TypeScript complexity**: Complex typing for script injection and results
|
|
1358
1427
|
|
|
1359
|
-
###
|
|
1428
|
+
### What we'll learn from ASLint
|
|
1429
|
+
|
|
1430
|
+
#### Module System Challenges
|
|
1360
1431
|
|
|
1361
|
-
#### **Module System Challenges:**
|
|
1362
1432
|
- How to handle `node_modules` paths in ES modules
|
|
1363
1433
|
- Script injection with nonce handling
|
|
1364
1434
|
- File system access patterns
|
|
1365
1435
|
|
|
1366
|
-
####
|
|
1436
|
+
#### TypeScript Challenges
|
|
1437
|
+
|
|
1367
1438
|
- Typing complex script injection
|
|
1368
1439
|
- Result parsing and validation
|
|
1369
1440
|
- Error handling patterns
|
|
1370
1441
|
|
|
1371
|
-
####
|
|
1442
|
+
#### Containerization Challenges
|
|
1443
|
+
|
|
1372
1444
|
- File system access in containers
|
|
1373
1445
|
- Script loading and execution
|
|
1374
1446
|
- Timeout and waiting patterns
|
|
1375
1447
|
|
|
1376
|
-
####
|
|
1448
|
+
#### Monorepo Challenges
|
|
1449
|
+
|
|
1377
1450
|
- Package dependency management
|
|
1378
1451
|
- Cross-package file access
|
|
1379
1452
|
- Build and deployment patterns
|
|
1380
1453
|
|
|
1381
|
-
###
|
|
1454
|
+
### Implementation approach
|
|
1455
|
+
|
|
1456
|
+
#### Phase 1: Stress-test with ASLint
|
|
1382
1457
|
|
|
1383
|
-
#### **Phase 1: Stress-test with ASLint**
|
|
1384
1458
|
```typescript
|
|
1385
1459
|
// This will be HARD - that's the point
|
|
1386
1460
|
export class ASLintTool implements TestingTool {
|
|
@@ -1421,19 +1495,20 @@ export class ASLintTool implements TestingTool {
|
|
|
1421
1495
|
}
|
|
1422
1496
|
```
|
|
1423
1497
|
|
|
1424
|
-
####
|
|
1498
|
+
#### Phase 2: Apply lessons to easier tools
|
|
1499
|
+
|
|
1425
1500
|
Once we've solved ASLint's challenges, HTML CodeSniffer and Axe will be straightforward.
|
|
1426
1501
|
|
|
1427
|
-
###
|
|
1502
|
+
### Success criteria for ASLint stress-test
|
|
1428
1503
|
|
|
1429
|
-
- [ ]
|
|
1430
|
-
- [ ]
|
|
1431
|
-
- [ ]
|
|
1432
|
-
- [ ]
|
|
1504
|
+
- [ ] ES modules: File system access works with `import.meta.url`
|
|
1505
|
+
- [ ] TypeScript: Complex script injection is properly typed
|
|
1506
|
+
- [ ] Containerization: Script loading works in containers
|
|
1507
|
+
- [ ] Monorepo: Package dependencies resolve correctly
|
|
1433
1508
|
- [ ] **Playwright Test**: Async waiting patterns work
|
|
1434
1509
|
- [ ] **Error handling**: Robust error handling for all failure points
|
|
1435
1510
|
|
|
1436
|
-
###
|
|
1511
|
+
### Why this approach is correct
|
|
1437
1512
|
|
|
1438
1513
|
1. **Risk mitigation**: Discover problems early when scope is small
|
|
1439
1514
|
2. **Pattern establishment**: Solve the hardest problems first
|
|
@@ -1441,7 +1516,7 @@ Once we've solved ASLint's challenges, HTML CodeSniffer and Axe will be straight
|
|
|
1441
1516
|
4. **Learning**: Team masters complex patterns early
|
|
1442
1517
|
5. **Efficiency**: Easier tools become trivial after solving hard ones
|
|
1443
1518
|
|
|
1444
|
-
###
|
|
1519
|
+
### Timeline expectations
|
|
1445
1520
|
|
|
1446
1521
|
- **ASLint**: 3-4 weeks (stress-testing phase)
|
|
1447
1522
|
- **HTML CodeSniffer**: 1 week (apply established patterns)
|
|
@@ -1492,7 +1567,7 @@ export class ASLintTool implements TestingTool {
|
|
|
1492
1567
|
|
|
1493
1568
|
---
|
|
1494
1569
|
|
|
1495
|
-
|
|
1570
|
+
## User prompt about packaging
|
|
1496
1571
|
|
|
1497
1572
|
The recommended monorepo structure above provides for Testaro being a package. Is it wise to consider making each testing tool that is currently within Testaro a distinct package? I imagine some benefits from this:
|
|
1498
1573
|
(1) Protection of the rest of Testaro from disruption when a particular testing tool is revised.
|
|
@@ -1501,25 +1576,28 @@ The recommended monorepo structure above provides for Testaro being a package. I
|
|
|
1501
1576
|
|
|
1502
1577
|
---
|
|
1503
1578
|
|
|
1504
|
-
|
|
1579
|
+
## Cursor reply about packaging
|
|
1505
1580
|
|
|
1506
1581
|
Excellent strategic thinking! Your analysis of the benefits is spot-on, and this approach would significantly improve the architecture. Let me analyze the implications and provide a refined monorepo structure.
|
|
1507
1582
|
|
|
1508
1583
|
## Benefits Analysis
|
|
1509
1584
|
|
|
1510
|
-
###
|
|
1585
|
+
### 1. Isolation & Protection
|
|
1586
|
+
|
|
1511
1587
|
- **Tool-specific updates**: Each tool can be updated independently
|
|
1512
1588
|
- **Dependency isolation**: Tool-specific dependencies don't conflict
|
|
1513
1589
|
- **Versioning**: Each tool can have its own versioning strategy
|
|
1514
1590
|
- **Testing**: Tool-specific validation and testing
|
|
1515
1591
|
|
|
1516
|
-
###
|
|
1592
|
+
### 2. Parallelization & Performance
|
|
1593
|
+
|
|
1517
1594
|
- **Horizontal scaling**: Run tools on separate servers/containers
|
|
1518
1595
|
- **Resource optimization**: Allocate resources per tool requirements
|
|
1519
1596
|
- **Fault tolerance**: One tool failure doesn't affect others
|
|
1520
1597
|
- **Load balancing**: Distribute tool execution across infrastructure
|
|
1521
1598
|
|
|
1522
|
-
###
|
|
1599
|
+
### 3. Custom Rules Framework
|
|
1600
|
+
|
|
1523
1601
|
- **Plugin architecture**: Each tool becomes a plugin
|
|
1524
1602
|
- **Customer-specific tools**: Custom tools as separate packages
|
|
1525
1603
|
- **Rule enforcement**: Tools can enforce custom business rules
|
|
@@ -1527,7 +1605,7 @@ Excellent strategic thinking! Your analysis of the benefits is spot-on, and this
|
|
|
1527
1605
|
|
|
1528
1606
|
## Recommended Monorepo Structure
|
|
1529
1607
|
|
|
1530
|
-
```
|
|
1608
|
+
```text
|
|
1531
1609
|
testaro-monorepo/
|
|
1532
1610
|
├── turbo.json
|
|
1533
1611
|
├── pnpm-workspace.yaml
|
|
@@ -1563,8 +1641,9 @@ testaro-monorepo/
|
|
|
1563
1641
|
|
|
1564
1642
|
## Tool Package Structure
|
|
1565
1643
|
|
|
1566
|
-
###
|
|
1567
|
-
|
|
1644
|
+
### Individual Tool Package Example
|
|
1645
|
+
|
|
1646
|
+
```text
|
|
1568
1647
|
packages/tools/axe/
|
|
1569
1648
|
├── package.json
|
|
1570
1649
|
├── src/
|
|
@@ -1578,7 +1657,8 @@ packages/tools/axe/
|
|
|
1578
1657
|
└── README.md
|
|
1579
1658
|
```
|
|
1580
1659
|
|
|
1581
|
-
###
|
|
1660
|
+
### Tool Package Interface
|
|
1661
|
+
|
|
1582
1662
|
```typescript
|
|
1583
1663
|
// packages/tools/axe/src/types.ts
|
|
1584
1664
|
export interface AxeToolOptions {
|
|
@@ -1608,7 +1688,8 @@ export class AxeTool implements TestingTool {
|
|
|
1608
1688
|
|
|
1609
1689
|
## Core Orchestrator Package
|
|
1610
1690
|
|
|
1611
|
-
###
|
|
1691
|
+
### Testaro Main Package
|
|
1692
|
+
|
|
1612
1693
|
```typescript
|
|
1613
1694
|
// packages/testaro/src/orchestrator.ts
|
|
1614
1695
|
export class TestaroOrchestrator {
|
|
@@ -1646,7 +1727,8 @@ export class TestaroOrchestrator {
|
|
|
1646
1727
|
|
|
1647
1728
|
## Parallelization Architecture
|
|
1648
1729
|
|
|
1649
|
-
###
|
|
1730
|
+
### Tool Server Implementation
|
|
1731
|
+
|
|
1650
1732
|
```typescript
|
|
1651
1733
|
// apps/tool-server/src/server.ts
|
|
1652
1734
|
export class ToolServer {
|
|
@@ -1682,7 +1764,8 @@ export class ToolServer {
|
|
|
1682
1764
|
}
|
|
1683
1765
|
```
|
|
1684
1766
|
|
|
1685
|
-
###
|
|
1767
|
+
### Distributed Execution
|
|
1768
|
+
|
|
1686
1769
|
```typescript
|
|
1687
1770
|
// packages/testaro/src/distributed-executor.ts
|
|
1688
1771
|
export class DistributedExecutor {
|
|
@@ -1723,7 +1806,8 @@ export class DistributedExecutor {
|
|
|
1723
1806
|
|
|
1724
1807
|
## Custom Rules Framework
|
|
1725
1808
|
|
|
1726
|
-
###
|
|
1809
|
+
### Custom Tool Package
|
|
1810
|
+
|
|
1727
1811
|
```typescript
|
|
1728
1812
|
// packages/custom-tools/customer-a-rules/src/index.ts
|
|
1729
1813
|
export class CustomerARulesTool implements TestingTool {
|
|
@@ -1758,19 +1842,22 @@ export class CustomerARulesTool implements TestingTool {
|
|
|
1758
1842
|
|
|
1759
1843
|
## Implementation Benefits
|
|
1760
1844
|
|
|
1761
|
-
###
|
|
1845
|
+
### 1. Isolation & Protection benefit
|
|
1846
|
+
|
|
1762
1847
|
- **Independent updates**: Each tool can be updated without affecting others
|
|
1763
1848
|
- **Dependency management**: Tool-specific dependencies are isolated
|
|
1764
1849
|
- **Versioning**: Each tool follows its own versioning strategy
|
|
1765
1850
|
- **Testing**: Tool-specific validation and testing
|
|
1766
1851
|
|
|
1767
|
-
###
|
|
1852
|
+
### 2. Parallelization & Performance benefit
|
|
1853
|
+
|
|
1768
1854
|
- **Horizontal scaling**: Tools can run on separate servers
|
|
1769
1855
|
- **Resource optimization**: Allocate resources per tool requirements
|
|
1770
1856
|
- **Fault tolerance**: One tool failure doesn't affect others
|
|
1771
1857
|
- **Load balancing**: Distribute tool execution across infrastructure
|
|
1772
1858
|
|
|
1773
|
-
###
|
|
1859
|
+
### 3. Custom Rules Framework benefit
|
|
1860
|
+
|
|
1774
1861
|
- **Plugin architecture**: Each tool becomes a plugin
|
|
1775
1862
|
- **Customer-specific tools**: Custom tools as separate packages
|
|
1776
1863
|
- **Rule enforcement**: Tools can enforce custom business rules
|
|
@@ -1778,31 +1865,35 @@ export class CustomerARulesTool implements TestingTool {
|
|
|
1778
1865
|
|
|
1779
1866
|
## Migration Strategy
|
|
1780
1867
|
|
|
1781
|
-
###
|
|
1868
|
+
### Phase 1: Tool Package Extraction
|
|
1869
|
+
|
|
1782
1870
|
1. Extract each tool into its own package
|
|
1783
1871
|
2. Establish common interfaces
|
|
1784
1872
|
3. Implement tool-specific validation
|
|
1785
1873
|
4. Test individual tool packages
|
|
1786
1874
|
|
|
1787
|
-
###
|
|
1875
|
+
### Phase 2: Orchestrator Implementation
|
|
1876
|
+
|
|
1788
1877
|
1. Implement Testaro orchestrator
|
|
1789
1878
|
2. Add parallel execution support
|
|
1790
1879
|
3. Implement distributed execution
|
|
1791
1880
|
4. Test orchestrator with tool packages
|
|
1792
1881
|
|
|
1793
|
-
###
|
|
1882
|
+
### Phase 3: Custom Rules Framework
|
|
1883
|
+
|
|
1794
1884
|
1. Implement custom tool interface
|
|
1795
1885
|
2. Create example custom tools
|
|
1796
1886
|
3. Add rule builder tool
|
|
1797
1887
|
4. Test custom rules framework
|
|
1798
1888
|
|
|
1799
|
-
###
|
|
1889
|
+
### Phase 4: Production Deployment
|
|
1890
|
+
|
|
1800
1891
|
1. Deploy tool servers
|
|
1801
1892
|
2. Implement load balancing
|
|
1802
1893
|
3. Add monitoring and logging
|
|
1803
1894
|
4. Performance optimization
|
|
1804
1895
|
|
|
1805
|
-
## Conclusion
|
|
1896
|
+
## Conclusion about packaging
|
|
1806
1897
|
|
|
1807
1898
|
Your suggestion to make each testing tool a distinct package is **architecturally sound** and provides significant benefits:
|
|
1808
1899
|
|
|
@@ -2031,13 +2122,13 @@ export class CustomerARulesTool implements TestingTool {
|
|
|
2031
2122
|
|
|
2032
2123
|
---
|
|
2033
2124
|
|
|
2034
|
-
|
|
2125
|
+
## User comment about Turborepo
|
|
2035
2126
|
|
|
2036
2127
|
I am beginning to doubt the wisdom of using Turborepo in a rearchitected project, because in another monorepo Turborepo is severely complicating my effort to tailor test and check output summaries to optimize the experience of the person who reads them.
|
|
2037
2128
|
|
|
2038
2129
|
---
|
|
2039
2130
|
|
|
2040
|
-
|
|
2131
|
+
## Cursor reply about Turborepo
|
|
2041
2132
|
|
|
2042
2133
|
That’s a reasonable doubt. If bespoke, human-friendly summaries are a priority, Turborepo’s log grouping/caching model can get in your way. You have three solid paths:
|
|
2043
2134
|
|
|
@@ -2060,6 +2151,7 @@ That’s a reasonable doubt. If bespoke, human-friendly summaries are a priority
|
|
|
2060
2151
|
- Use package-level reporters (Vitest/Playwright/ESLint/Biome) as the source of truth
|
|
2061
2152
|
|
|
2062
2153
|
Recommendation for your case
|
|
2154
|
+
|
|
2063
2155
|
- Use pnpm workspaces + a tiny custom runner (and/or a CI matrix) so you fully own output format and ordering, and avoid Turbo’s abstractions entirely. Add per-tool reporters that emit JSON and let a single summarizer assemble exactly the view you want. You can introduce a heavier orchestrator later if you need build graph/caching beyond what tools already provide.
|
|
2064
2156
|
|
|
2065
2157
|
If you want, I can sketch the runner’s interface (inputs/filters/concurrency) and the summary artifact format so it slots cleanly into your validation flow and UI.
|
|
@@ -2241,7 +2333,7 @@ A password is required only for the local database. CAP requires no password, be
|
|
|
2241
2333
|
|
|
2242
2334
|
It is possible to make the databases of different environments completely independent. A straightforward strategy is to equip your application with the ability to create and modify a PostgreSQL database, by making `postgres` a package dependency:
|
|
2243
2335
|
|
|
2244
|
-
```
|
|
2336
|
+
```json
|
|
2245
2337
|
"dependencies": {
|
|
2246
2338
|
"postgres": "3.4.7"
|
|
2247
2339
|
}
|
|
@@ -2276,11 +2368,11 @@ postgres=> create database spacedb owner localuser;
|
|
|
2276
2368
|
\q
|
|
2277
2369
|
```
|
|
2278
2370
|
|
|
2279
|
-
#### Set environment variables
|
|
2371
|
+
#### Set environment variables for migration
|
|
2280
2372
|
|
|
2281
2373
|
Add to `~/.bash_profile`:
|
|
2282
2374
|
|
|
2283
|
-
```
|
|
2375
|
+
```bash
|
|
2284
2376
|
# PostgreSQL
|
|
2285
2377
|
export PGHOST=localhost
|
|
2286
2378
|
export PGPORT=5432
|
|
@@ -2460,7 +2552,7 @@ describe('Database queries', () => {
|
|
|
2460
2552
|
|
|
2461
2553
|
It is suspected that some Testaro tests are unnecessarily inefficient because they iterate across elements in a page, performing a browser operation on each and then returning a result for processing by Node. For example, in `lineHeight`:
|
|
2462
2554
|
|
|
2463
|
-
```
|
|
2555
|
+
```javascript
|
|
2464
2556
|
const all = await init(100, page, 'body *', {hasText: /[^\s]/});
|
|
2465
2557
|
// For each locator:
|
|
2466
2558
|
for (const loc of all.allLocs) {
|
|
@@ -2481,7 +2573,7 @@ However, there is evidence that moving the processing of Playwright locators int
|
|
|
2481
2573
|
|
|
2482
2574
|
One proposal for a rewrite of `lineHeight`, still doing sampling, was:
|
|
2483
2575
|
|
|
2484
|
-
```
|
|
2576
|
+
```javascript
|
|
2485
2577
|
// Reports a test.
|
|
2486
2578
|
exports.reporter = async (page, report) => {
|
|
2487
2579
|
// Get data on elements with potentially invalid line heights.
|
|
@@ -2549,7 +2641,7 @@ exports.reporter = async (page, report) => {
|
|
|
2549
2641
|
|
|
2550
2642
|
To reuse a sampling (or any other) function in multiple tests, it was proposed to store the function in a file and then inject it into the page as follows:
|
|
2551
2643
|
|
|
2552
|
-
```
|
|
2644
|
+
```javascript
|
|
2553
2645
|
const getSample = require(path.join(__dirname, 'procs/sample.js')).getSample;
|
|
2554
2646
|
const initScript = `window.getSample = ${getSample.toString()};`;
|
|
2555
2647
|
await browserContext.addInitScript(initScript);
|
|
@@ -2564,7 +2656,7 @@ Option 1: Don't commit package-lock.json on the server (Recommended)
|
|
|
2564
2656
|
|
|
2565
2657
|
On the server, tell git to ignore local changes to package-lock.json:
|
|
2566
2658
|
|
|
2567
|
-
```
|
|
2659
|
+
```bash
|
|
2568
2660
|
# Assume package-lock.json is unchanged locally
|
|
2569
2661
|
git update-index --assume-unchanged package-lock.json
|
|
2570
2662
|
```
|
|
@@ -2575,7 +2667,7 @@ Option 2: Exclude package-lock.json from version control
|
|
|
2575
2667
|
|
|
2576
2668
|
If you don't need package-lock.json in your repo (since you use * for all versions in package.json), add it to .gitignore:
|
|
2577
2669
|
|
|
2578
|
-
```
|
|
2670
|
+
```bash
|
|
2579
2671
|
echo "package-lock.json" >> .gitignore
|
|
2580
2672
|
git rm --cached package-lock.json
|
|
2581
2673
|
git commit -m "Stop tracking package-lock.json"
|
|
@@ -2584,7 +2676,7 @@ git push
|
|
|
2584
2676
|
|
|
2585
2677
|
Then on the server:
|
|
2586
2678
|
|
|
2587
|
-
```
|
|
2679
|
+
```bash
|
|
2588
2680
|
git pull
|
|
2589
2681
|
npm update # Will create package-lock.json locally but git won't track it
|
|
2590
2682
|
```
|
|
@@ -2593,7 +2685,7 @@ Option 3: Proper deployment workflow (Best Practice)
|
|
|
2593
2685
|
|
|
2594
2686
|
On the server, use a deployment script instead of git pull:
|
|
2595
2687
|
|
|
2596
|
-
```
|
|
2688
|
+
```bash
|
|
2597
2689
|
#!/bin/bash
|
|
2598
2690
|
# deploy.sh
|
|
2599
2691
|
|
|
@@ -2618,7 +2710,7 @@ pm2 restart kilotest
|
|
|
2618
2710
|
|
|
2619
2711
|
Make it executable and use it:
|
|
2620
2712
|
|
|
2621
|
-
```
|
|
2713
|
+
```bash
|
|
2622
2714
|
chmod +x deploy.sh
|
|
2623
2715
|
./deploy.sh
|
|
2624
2716
|
```
|
|
@@ -2635,7 +2727,7 @@ Recommended Approach
|
|
|
2635
2727
|
|
|
2636
2728
|
Use exact versions to ensure consistency across environments:
|
|
2637
2729
|
|
|
2638
|
-
```
|
|
2730
|
+
```json
|
|
2639
2731
|
{
|
|
2640
2732
|
"dependencies": {
|
|
2641
2733
|
"dotenv": "^17.2.3",
|
|
@@ -2647,7 +2739,7 @@ Use exact versions to ensure consistency across environments:
|
|
|
2647
2739
|
|
|
2648
2740
|
Then:
|
|
2649
2741
|
|
|
2650
|
-
```
|
|
2742
|
+
```bash
|
|
2651
2743
|
# Lock versions
|
|
2652
2744
|
npm install
|
|
2653
2745
|
|
|
@@ -2670,13 +2762,13 @@ You can use npm-check-updates (ncu) to automatically update version numbers in p
|
|
|
2670
2762
|
|
|
2671
2763
|
Installation
|
|
2672
2764
|
|
|
2673
|
-
```
|
|
2765
|
+
```bash
|
|
2674
2766
|
npm install -g npm-check-updates
|
|
2675
2767
|
```
|
|
2676
2768
|
|
|
2677
2769
|
Usage
|
|
2678
2770
|
|
|
2679
|
-
```
|
|
2771
|
+
```bash
|
|
2680
2772
|
# Check what would be updated
|
|
2681
2773
|
ncu
|
|
2682
2774
|
|
|
@@ -2695,7 +2787,7 @@ git push
|
|
|
2695
2787
|
Automate with a Script
|
|
2696
2788
|
Add to your package.json:
|
|
2697
2789
|
|
|
2698
|
-
```
|
|
2790
|
+
```json
|
|
2699
2791
|
{
|
|
2700
2792
|
"scripts": {
|
|
2701
2793
|
"update-deps": "ncu -u && npm install && git add package.json package-lock.json"
|
|
@@ -2705,7 +2797,7 @@ Add to your package.json:
|
|
|
2705
2797
|
|
|
2706
2798
|
Then use:
|
|
2707
2799
|
|
|
2708
|
-
```
|
|
2800
|
+
```bash
|
|
2709
2801
|
npm run update-deps
|
|
2710
2802
|
git commit -m "chore: update dependencies"
|
|
2711
2803
|
git push
|
|
@@ -2714,7 +2806,7 @@ git push
|
|
|
2714
2806
|
More Control with Target Versions
|
|
2715
2807
|
Instead of *, use caret ranges for automatic minor/patch updates:
|
|
2716
2808
|
|
|
2717
|
-
```
|
|
2809
|
+
```json
|
|
2718
2810
|
{
|
|
2719
2811
|
"dependencies": {
|
|
2720
2812
|
"dotenv": "^17.2.3", // allows 17.x.x
|
|
@@ -2734,7 +2826,7 @@ Workflow
|
|
|
2734
2826
|
|
|
2735
2827
|
Local development:
|
|
2736
2828
|
|
|
2737
|
-
```
|
|
2829
|
+
```bash
|
|
2738
2830
|
# When you want to update dependencies
|
|
2739
2831
|
ncu -u # Update package.json to latest versions
|
|
2740
2832
|
npm install # Install and lock exact versions
|
|
@@ -2746,7 +2838,7 @@ git push
|
|
|
2746
2838
|
|
|
2747
2839
|
Server deployment:
|
|
2748
2840
|
|
|
2749
|
-
```
|
|
2841
|
+
```bash
|
|
2750
2842
|
git pull # Gets updated package.json and package-lock.json
|
|
2751
2843
|
npm ci # Installs exact locked versions
|
|
2752
2844
|
pm2 restart kilotest
|
|
@@ -2756,7 +2848,7 @@ Alternative: Use npm's Built-in Commands
|
|
|
2756
2848
|
|
|
2757
2849
|
For major version updates with interactive prompts:
|
|
2758
2850
|
|
|
2759
|
-
```
|
|
2851
|
+
```bash
|
|
2760
2852
|
# Check outdated packages
|
|
2761
2853
|
npm outdated
|
|
2762
2854
|
|
|
@@ -2776,7 +2868,7 @@ git push
|
|
|
2776
2868
|
Pre-commit Hook (Optional)
|
|
2777
2869
|
Ensure package-lock.json is always in sync:
|
|
2778
2870
|
|
|
2779
|
-
```
|
|
2871
|
+
```bash
|
|
2780
2872
|
# .husky/pre-commit or .git/hooks/pre-commit
|
|
2781
2873
|
#!/bin/sh
|
|
2782
2874
|
if git diff --cached --name-only | grep -q "package.json"; then
|
|
@@ -2789,6 +2881,7 @@ fi
|
|
|
2789
2881
|
```
|
|
2790
2882
|
|
|
2791
2883
|
Recommended Setup
|
|
2884
|
+
|
|
2792
2885
|
1. Change * to ^ ranges in package.json
|
|
2793
2886
|
2. Install npm-check-updates globally: npm i -g npm-check-updates
|
|
2794
2887
|
3. Add update script to package.json
|
|
@@ -2812,7 +2905,7 @@ The issue is that forceKillBrowsers() is killing the browser processes immediate
|
|
|
2812
2905
|
Solution: Coordinate Async and Sync Cleanup
|
|
2813
2906
|
Replace the cleanup handlers in run.js:
|
|
2814
2907
|
|
|
2815
|
-
```
|
|
2908
|
+
```javascript
|
|
2816
2909
|
// CLEANUP HANDLERS
|
|
2817
2910
|
|
|
2818
2911
|
// Track cleanup state.
|
|
@@ -2884,7 +2977,7 @@ process.on('uncaughtException', async error => {
|
|
|
2884
2977
|
|
|
2885
2978
|
And in `doActs`, replace the cleanup:
|
|
2886
2979
|
|
|
2887
|
-
```
|
|
2980
|
+
```javascript
|
|
2888
2981
|
console.log('Acts completed');
|
|
2889
2982
|
// If standardization is required:
|
|
2890
2983
|
if (['also', 'only'].includes(standard)) {
|
|
@@ -2895,7 +2988,7 @@ And in `doActs`, replace the cleanup:
|
|
|
2895
2988
|
// Delete the temporary report file.
|
|
2896
2989
|
await fs.rm(reportPath, {force: true});
|
|
2897
2990
|
return report;
|
|
2898
|
-
|
|
2991
|
+
```
|
|
2899
2992
|
|
|
2900
2993
|
Key changes:
|
|
2901
2994
|
|
|
@@ -2965,3 +3058,184 @@ const doTestAct = async () => {
|
|
|
2965
3058
|
// ...existing code...
|
|
2966
3059
|
};
|
|
2967
3060
|
```
|
|
3061
|
+
|
|
3062
|
+
## Elapsed times
|
|
3063
|
+
|
|
3064
|
+
One example of the elapsed times (in seconds) for tool execution on a relatively complex and defect-rich page:
|
|
3065
|
+
|
|
3066
|
+
```json
|
|
3067
|
+
"toolTimes": {
|
|
3068
|
+
"testaro": 132,
|
|
3069
|
+
"ibm": 25,
|
|
3070
|
+
"alfa": 13,
|
|
3071
|
+
"qualWeb": 13,
|
|
3072
|
+
"aslint": 7,
|
|
3073
|
+
"wax": 7,
|
|
3074
|
+
"axe": 6,
|
|
3075
|
+
"ed11y": 5,
|
|
3076
|
+
"nuVal": 4,
|
|
3077
|
+
"htmlcs": 3
|
|
3078
|
+
},
|
|
3079
|
+
```
|
|
3080
|
+
|
|
3081
|
+
The total is 215 seconds. The total elapsed time of the job was 318 seconds. The 103-second difference was consumed mainly by standardization.
|
|
3082
|
+
|
|
3083
|
+
Standardization and some Testaro tests are suspected of being inefficient in part by alternating many times between Node.js and the browser environment and partly by executing `getComputedStyle` many times.
|
|
3084
|
+
|
|
3085
|
+
The Testaro rule tests had the following elapsed times:
|
|
3086
|
+
|
|
3087
|
+
```json
|
|
3088
|
+
"ruleTestTimes": {
|
|
3089
|
+
"lineHeight": 22,
|
|
3090
|
+
"allCaps": 9,
|
|
3091
|
+
"focInd": 8,
|
|
3092
|
+
"miniText": 7,
|
|
3093
|
+
"focAll": 7,
|
|
3094
|
+
"hovInd": 7,
|
|
3095
|
+
"focOp": 6,
|
|
3096
|
+
"focVis": 6,
|
|
3097
|
+
"opFoc": 6,
|
|
3098
|
+
"allSlanted": 5,
|
|
3099
|
+
"targetSmall": 4,
|
|
3100
|
+
"zIndex": 4,
|
|
3101
|
+
"distortion": 3,
|
|
3102
|
+
"linkAmb": 3,
|
|
3103
|
+
"targetTiny": 3,
|
|
3104
|
+
"shoot1": 3,
|
|
3105
|
+
"shoot0": 2,
|
|
3106
|
+
"linkUl": 2,
|
|
3107
|
+
"adbID": 1,
|
|
3108
|
+
"imageLink": 1,
|
|
3109
|
+
"hover": 1,
|
|
3110
|
+
"tabNav": 1,
|
|
3111
|
+
"allHidden": 0,
|
|
3112
|
+
"altScheme": 0,
|
|
3113
|
+
"dupAtt": 0,
|
|
3114
|
+
"autocomplete": 0,
|
|
3115
|
+
"bulk": 0,
|
|
3116
|
+
"captionLoc": 0,
|
|
3117
|
+
"datalistRef": 0,
|
|
3118
|
+
"docType": 0,
|
|
3119
|
+
"embAc": 0,
|
|
3120
|
+
"headEl": 0,
|
|
3121
|
+
"headingAmb": 0,
|
|
3122
|
+
"hr": 0,
|
|
3123
|
+
"labClash": 0,
|
|
3124
|
+
"legendLoc": 0,
|
|
3125
|
+
"linkExt": 0,
|
|
3126
|
+
"linkOldAtt": 0,
|
|
3127
|
+
"linkTitle": 0,
|
|
3128
|
+
"linkTo": 0,
|
|
3129
|
+
"nonTable": 0,
|
|
3130
|
+
"optRoleSel": 0,
|
|
3131
|
+
"phOnly": 0,
|
|
3132
|
+
"pseudoP": 0,
|
|
3133
|
+
"radioSet": 0,
|
|
3134
|
+
"role": 0,
|
|
3135
|
+
"secHeading": 0,
|
|
3136
|
+
"styleDiff": 0,
|
|
3137
|
+
"textSem": 0,
|
|
3138
|
+
"titledEl": 0,
|
|
3139
|
+
"buttonMenu": 0
|
|
3140
|
+
},
|
|
3141
|
+
```
|
|
3142
|
+
|
|
3143
|
+
## Element identification
|
|
3144
|
+
|
|
3145
|
+
Suggestions by GPT-5.1 about a strategy for cross-tool element identification:
|
|
3146
|
+
|
|
3147
|
+
You’re pointing at exactly the right abstraction: **“element identity across tools”** is a harder problem than “which XPath generator is best”, and it probably deserves its own design.
|
|
3148
|
+
|
|
3149
|
+
Let me outline options briefly so you can decide how far to push XPath in [testaro](cci:7://file:///Users/pool/Users/pool/Documents/Topics/work/testaro:0:0-0:0) right now.
|
|
3150
|
+
|
|
3151
|
+
---
|
|
3152
|
+
|
|
3153
|
+
### 1. Why a single identifier is fragile
|
|
3154
|
+
|
|
3155
|
+
Any _single_ indicator can fail:
|
|
3156
|
+
|
|
3157
|
+
- **XPath / CSS selector**
|
|
3158
|
+
- Breaks if DOM structure changes between tools or between runs.
|
|
3159
|
+
- Different generators produce different-but-equivalent strings.
|
|
3160
|
+
- **Line/column**
|
|
3161
|
+
- Depends on exact source formatting; minification / templating alters it.
|
|
3162
|
+
- **Box coordinates**
|
|
3163
|
+
- Vary with layout, responsive design, fonts, zoom, user settings.
|
|
3164
|
+
- **Text content**
|
|
3165
|
+
- Collisions (same text in many places), localization, dynamic content.
|
|
3166
|
+
- **Outer/innerHTML**
|
|
3167
|
+
- Expensive to compare; small, unrelated changes break equality.
|
|
3168
|
+
|
|
3169
|
+
So, treating “XPath strings are equal” as _the_ notion of identity is brittle, especially across heterogeneous tools.
|
|
3170
|
+
|
|
3171
|
+
---
|
|
3172
|
+
|
|
3173
|
+
### 2. A fingerprint-style approach
|
|
3174
|
+
|
|
3175
|
+
A more robust way is to treat each element as having a **feature vector** and define similarity across those, rather than equality of a single field.
|
|
3176
|
+
|
|
3177
|
+
Example feature set per element:
|
|
3178
|
+
|
|
3179
|
+
- **DOM path structure**
|
|
3180
|
+
- A normalized XPath or CSS path (even if different generators format slightly differently)
|
|
3181
|
+
- Depth, tag sequence (e.g. `html/body/div/p`)
|
|
3182
|
+
- **DOM neighborhood**
|
|
3183
|
+
- Parent tag and its classes/id
|
|
3184
|
+
- Position among siblings (index, `nth-of-type`)
|
|
3185
|
+
- **Attributes**
|
|
3186
|
+
- `id`, `class` list, `name`, `role`, ARIA attributes
|
|
3187
|
+
- **Content**
|
|
3188
|
+
- Normalized text content (trimmed, collapsed whitespace, maybe truncated hash)
|
|
3189
|
+
- **Geometry (optional)**
|
|
3190
|
+
- Bounding box (x, y, width, height) rounded to a sensible grid
|
|
3191
|
+
- **Source anchors (optional)**
|
|
3192
|
+
- Line/column if available from a tool
|
|
3193
|
+
|
|
3194
|
+
Then, when correlating two reports, you’d compute some similarity score between elements from different tools based on:
|
|
3195
|
+
|
|
3196
|
+
- Exact matches where possible (`id`, normalized text if short and unique)
|
|
3197
|
+
- High-weight soft matches (same parent path + same tag + close sibling index)
|
|
3198
|
+
- Weaker signals (similar bounding boxes, overlapping text, etc.)
|
|
3199
|
+
|
|
3200
|
+
This gives you an **element matcher** whose job is: “Given element A from tool X and a page, find the best candidate(s) for that element in tool Y’s universe.”
|
|
3201
|
+
|
|
3202
|
+
XPath then becomes _one strong feature_ but not the only one.
|
|
3203
|
+
|
|
3204
|
+
---
|
|
3205
|
+
|
|
3206
|
+
### 3. What this implies for [testaro](cci:7://file:///Users/pool/Users/pool/Documents/Topics/work/testaro:0:0-0:0) now
|
|
3207
|
+
|
|
3208
|
+
Given that direction, for [testaro](cci:7://file:///Users/pool/Users/pool/Documents/Topics/work/testaro:0:0-0:0) specifically:
|
|
3209
|
+
|
|
3210
|
+
- It’s still useful to:
|
|
3211
|
+
- Generate a **reasonable, deterministic path** (your in-page XPath helper is fine).
|
|
3212
|
+
- Record **other cheap features** you already have in each rule:
|
|
3213
|
+
- tag name
|
|
3214
|
+
- `id`/`class`
|
|
3215
|
+
- trimmed text (maybe length-limited)
|
|
3216
|
+
- maybe a simple parent description
|
|
3217
|
+
|
|
3218
|
+
- You don’t need [testaro](cci:7://file:///Users/pool/Users/pool/Documents/Topics/work/testaro:0:0-0:0)’s XPath to **bit-for-bit match** `playwright-dompath`’s output _if_ later you’ll have a correlation step that:
|
|
3219
|
+
- Can re-locate [testaro](cci:7://file:///Users/pool/Users/pool/Documents/Topics/work/testaro:0:0-0:0)’s element on a fresh page using its own path/attributes.
|
|
3220
|
+
- Then, if needed, run that DOM node through `playwright-dompath` to obtain the canonical XPath for cross-tool IDs.
|
|
3221
|
+
|
|
3222
|
+
So you can:
|
|
3223
|
+
|
|
3224
|
+
1. For now, implement the **fast in-page XPath helper** for [testaro](cci:7://file:///Users/pool/Users/pool/Documents/Topics/work/testaro:0:0-0:0) and treat it as an _internal_ identifier plus context.
|
|
3225
|
+
2. Defer the question of **canonical cross-tool ID** to a dedicated “element correlation” layer that may:
|
|
3226
|
+
- Normalize XPaths/paths.
|
|
3227
|
+
- Use multiple signals, not just string equality.
|
|
3228
|
+
|
|
3229
|
+
---
|
|
3230
|
+
|
|
3231
|
+
### 4. Practical short-term guidance
|
|
3232
|
+
|
|
3233
|
+
Given your goal to move on with [testaro](cci:7://file:///Users/pool/Users/pool/Documents/Topics/work/testaro:0:0-0:0):
|
|
3234
|
+
|
|
3235
|
+
- Don’t over-invest in perfectly mimicking `playwright-dompath` inside the page.
|
|
3236
|
+
- Do:
|
|
3237
|
+
- Use your shared `addXPathScript` + `window.testaroXPath`.
|
|
3238
|
+
- Include some extra context in each [testaro](cci:7://file:///Users/pool/Users/pool/Documents/Topics/work/testaro:0:0-0:0) instance (`tagName`, `id`, `class`, short text).
|
|
3239
|
+
- Plan separately for a future **element-matching module** that consumes all tools’ outputs and decides “these instances refer to the same DOM element”.
|
|
3240
|
+
|
|
3241
|
+
That way, you get immediate performance and implementation wins in [testaro](cci:7://file:///Users/pool/Users/pool/Documents/Topics/work/testaro:0:0-0:0), without locking yourself into the fragile assumption that “XPath strings must match exactly across all tools.
|