testeranto 0.129.0 → 0.135.0

package/README.md

@@ -1,6 +1,6 @@
 # testeranto
 
-## the AI powered BDD test framework for typescript projects
+## The AI-powered BDD test framework for TypeScript projects
 
 🚧 WARNING: Testeranto is still under development and is not ready for production yet. 🚧
 
@@ -12,11 +12,7 @@ npm: [npmjs.com/package/testeranto](https://www.npmjs.com/package/testeranto)
 
 dev: [github.dev/adamwong246/testeranto](https://github.dev/adamwong246/testeranto)
 
-example test report: [chromapdx.github.io/kokomoBay](https://chromapdx.github.io/kokomoBay/testeranto/index.html)
-
-example repo: [kokomo bay](https://github.com/ChromaPDX/kokomoBay)
-
-example repo V2: [testeranto-starter](https://github.com/adamwong246/testeranto-starter)
+example repo: [testeranto-starter](https://github.com/adamwong246/testeranto-starter)
 
 ## What is testeranto?
 
@@ -27,232 +23,248 @@ example repo V2: [testeranto-starter](https://github.com/adamwong246/testeranto-
 - Testeranto connects "features" to "tests". This allows the AI to read feature documentation from external systems, like Jira.
 - Testeranto generates test results into static a website which can be deployed to github pages easily.
 
-## tech of note
+## Key Technologies
 
-- esm - Testeranto uses modern js
-- typescript - tests are functions with type parameters
-- puppeteer - provides access to both node and chrome runtimes
-- esbuild - used to quickly generate test bundles
-- aider - AI to automatically fix broken tests
-- eslint - runs upon the input files to generate a file of static analysis errors
-- tsc - runs upon the input files to generate a file of type errors
-- markdown - Markdown is used record feature files
+Testeranto builds on modern JavaScript/TypeScript tooling:
 
-## scripts
+| Technology | Purpose                                |
+| ---------- | -------------------------------------- |
+| TypeScript | Strongly-typed test definitions        |
+| Puppeteer  | Cross-runtime testing (Node & Browser) |
+| esbuild    | Fast test bundling                     |
+| Aider.ai   | AI-powered test fixing                 |
+| ESLint     | Static analysis of test files          |
+| tsc        | Type checking of test files            |
+| Markdown   | Feature documentation format           |
 
-`yarn t-init`: startup a new testeranto project
+## Quick Start
 
-`yarn t-build <someTest> <once|dev>`: build the "someTest" project once, or continuously
+1. Install testeranto:
 
-`yarn t-run <someTest> <once|dev>`: run the "someTest" project once, or continuously
+```bash
+npm install testeranto
+```
 
-`yarn t-report` Run the report server
+2. Create a test file (e.g., `rectangle.test.ts`):
 
-`yarn t-aider PATH_TO_PROMPT_FILE`: Execute a generated prompt file to fix broken tests.
+```typescript
+import { Given, When, Then } from "testeranto";
 
-## AI
-
-Testeranto generates a "prompt" alongside test results. This prompt is passed to aider as input.
+type Rectangle = { width: number; height: number };
 
+const RectangleSpec = (Suite, Given, When, Then) => [
+  Suite.Default("Rectangle tests", {
+    test1: Given.Default(
+      ["Basic rectangle operations"],
+      [When.setWidth(5), When.setHeight(10)],
+      [Then.getWidth(5), Then.getHeight(10)]
+    ),
+  }),
+];
 ```
-// input src files which can be edited by aider
-/add test/node.ts
 
-// test report files that inform aider but should not be edited
-/read testeranto/reports/allTests/node/test/node/tests.json
-/read testeranto/reports/allTests/test/node/node/lint_errors.json
-/read testeranto/reports/allTests/test/node/node/type_errors.txt
+3. Run your tests in two separate terminals:
 
-// A list of features which can inform aider.
-/load testeranto/reports/allTests/node/test/node/featurePrompt.txt
+```bash
+# Terminal 1 - Build in watch mode
+yarn t-build rectangle.test.ts dev
 
-// tell the AI what to do
-/code Fix the failing tests described in testeranto/reports/allTests/node/test/node/tests.json. Correct any type signature errors described in the files testeranto/reports/allTests/test/node/node/type_errors.txt. Implement any method which throws "Function not implemented. Resolve the lint errors described in testeranto/reports/allTests/test/node/node/lint_errors.json"
+# Terminal 2 - Run in watch mode
+yarn t-run rectangle.test.ts dev
 ```
 
-## "Features"
+### Development Workflow
 
-Testeranto connects "features" to tests. The features may be simple strings, but they can also take the form of local markdown files, or remote URLs to external feature tracking systems. For instance, this could be a jira ticket or a github issue. These features are used to inform the AI context.
+```mermaid
 
-## Platforms
+flowchart LR
 
-Testeranto runs tests in multiple runtimes. You can run the same test (more or less) in multiple contexts, but depending on your test subject, not all may be applicable. For instance, if you are testing an http node server, you'll can't use the web runtime. If your code references `document` or `window`, you must use the web style. And if you wish to capture console.logs in a node context, you should use the `pure` runtime.
+    subgraph hh["humans"]
+        direction LR
+        Human[🧑💻 ]
+    end
 
-1. Node - the test is run in node v8 via fork.
-2. Web - the test is run in chrome, in a page.
-3. Pure - the test is dynamically imported into the main thread. It will not have access to IO (console.log, etc) but it is more performant.
+    subgraph bb["AI"]
+        direction LR
+        Bot[🤖🧠 aider]
+    end
 
-## Concepts
 
-Testeranto tests take some piece of javascript as input, and wraps it in testing apparatus, and then executes that test on the given platform. You must provide this apparatus in the following form:
+    tests ---> L
+    subgraph tests
+        direction LR
+        A[Test Specification]
+        B[Test Interface]
+        C[Test Implementation]
+        K[application code]
+    end
 
-```js
-export default async <I extends IT, O extends OT, M>(
+    subgraph buildSystem
+        direction TB
+        L["t-build"]
+        M[t-run]
+        L ---> M
 
-  // the thing that is being tested.
-  input: I["iinput"],
+        M ---> N
+        M --->O
+        M --->P
+        N["BDD tests"]
+        O["Static analysis"]
+        P["Type checking"]
 
-  testSpecification: ITestSpecification<I, O>,
-  testImplementation: ITestImplementation<I, O, M>,
-  testInterface: Partial<IWebTestInterface<I>>,
-  testResourceRequirement: ITTestResourceRequest = defaultTestResourceRequirement
-) => {
+        Q["reports"]
+        N ---> Q
+        O ---> Q
+        P ---> Q
+    end
 
-  // or WebTesteranto<I, O, M> or PureTesteranto<I, O, M>
-  return new NodeTesteranto<I, O, M>(
-    input,
-    testSpecification,
-    testImplementation,
-    testResourceRequirement,
-    testInterface
-  );
-};
+    Q ---> bb
 
-```
+    buildSystem ---> bb
+    bb ---> tests
+    hh ---> tests
 
-Practically speaking, for each thing you test, you will need to implement 3 types and 4 objects.
+    %% Styling
+    style Human fill:#268bd2,stroke:#586e75,color:#fdf6e3
+    style Bot fill:#d33682,stroke:#586e75,color:#fdf6e3
 
-### type I
+    %% Layout tweaks
+    classDef column margin-right:20px
 
-this type describes the shape of the BDD test
+```
 
-```ts
-export type I = Ibdd_in<
-  null,
-  null,
-  Rectangle,
-  Rectangle,
-  Rectangle,
-  (...x) => (rectangle: Rectangle, utils: IPM) => Rectangle,
-  (rectangle: Rectangle, utils: IPM) => Rectangle
->;
+## Architecture Overview
+
+```mermaid
+flowchart TD
+    subgraph ThreePillars["Testeranto Core"]
+        Builder[Test Builder]
+        Runner[Test Runner]
+        Aider[AI Integration]
+    end
+
+    subgraph BuilderComponents[" "]
+        Specification[Specification]
+        Implementation[Implementation]
+        Interface[Interface]
+    end
+
+    subgraph Runtimes[" "]
+        Node[Node]
+        Web[Browser]
+        Pure[JS]
+    end
+
+    Builder --> Runner
+    Runner --> Aider
+    Aider --> Builder
+
+    Builder --> BuilderComponents
+    Runner --> Runtimes
+
+    style ThreePillars fill:none,stroke:#586e75
+    style Builder fill:#268bd2,stroke:#586e75,color:#fdf6e3
+    style Runner fill:#268bd2,stroke:#586e75,color:#fdf6e3
+    style Aider fill:#b58900,stroke:#586e75,color:#002b36
+    style BuilderComponents fill:#002b36,stroke:#586e75,color:#eee8d5
+    style Runtimes fill:#073642,stroke:#586e75,color:#eee8d5
+    style Specification fill:#2aa198,stroke:#586e75,color:#002b36
+    style Implementation fill:#2aa198,stroke:#586e75,color:#002b36
+    style Interface fill:#2aa198,stroke:#586e75,color:#002b36
+    style Node fill:#859900,stroke:#586e75,color:#002b36
+    style Web fill:#859900,stroke:#586e75,color:#002b36
+    style Pure fill:#859900,stroke:#586e75,color:#002b36
 ```
 
-### type O
+## Runtime Platforms
 
-this type describes the shape of the "interface"
+Testeranto runs tests in multiple runtime environments, each suited for different testing scenarios:
 
-```ts
-export type O = Ibdd_out<
-  // Suite
-  {
-    Default: [string];
-  },
-  // "Given" are initial states
-  {
-    Default;
-    WidthOfOneAndHeightOfOne;
-    WidthAndHeightOf: [number, number];
-  },
-  // "Whens" are steps which change the state of the test subject
-  {
-    HeightIsPubliclySetTo: [number];
-    WidthIsPubliclySetTo: [number];
-    setWidth: [number];
-    setHeight: [number];
-  },
-  // "Thens" are steps which make assertions of the test subject
-  {
-    AreaPlusCircumference: [number];
-    circumference: [number];
-    getWidth: [number];
-    getHeight: [number];
-    area: [number];
-    prototype: [];
-  },
-  // "Checks" are similar to "Givens"
-  {
-    Default;
-    WidthOfOneAndHeightOfOne;
-    WidthAndHeightOf: [number, number];
-  }
->;
-```
+| Runtime  | Description                                       | When To Use                                                            | Key Characteristics                                     |
+| -------- | ------------------------------------------------- | ---------------------------------------------------------------------- | ------------------------------------------------------- |
+| **Node** | Full IO access with Node.js built-in modules      | Testing backend code, Node APIs, or anything needing filesystem access | Runs in Node v8 via fork, has access to fs, crypto, etc |
+| **Web**  | DOM API access with browser capabilities          | Testing frontend code, UI interactions, or visual regression           | Runs in Chrome page, can take screenshots/recordings    |
+| **Pure** | Isolated JS runtime without external dependencies | Fast unit tests that don't need external resources                     | Dynamically imported into main thread, no IO access     |
 
-### type M (optional)
+**Key Considerations:**
 
-this type describes the modifications to the shape of the "specification". It can be used to make your BDD tests DRYer but is not necessary
+- Use **Node** for testing backend services, file operations, or anything requiring Node.js APIs
+- Use **Web** when testing browser-specific code that references `document` or `window`
+- Use **Pure** for fast, isolated unit tests where you don't need console output or external resources
 
-```ts
-export type M = {
-  givens: {
-    [K in keyof O["givens"]]: (...Iw: O["givens"][K]) => Rectangle;
-  };
-  whens: {
-    [K in keyof O["whens"]]: (
-      ...Iw: O["whens"][K]
-    ) => (rectangle: Rectangle, utils: PM) => Rectangle;
-  };
-  thens: {
-    [K in keyof O["thens"]]: (
-      ...Iw: O["thens"][K]
-    ) => (rectangle: Rectangle, utils: PM) => Rectangle;
-  };
-};
+## CLI Commands
+
+| Command                               | Description                                   |
+| ------------------------------------- | --------------------------------------------- |
+| `yarn t-init`                         | Create a new testeranto project               |
+| `yarn t-build <YOUR_TESTS> dev\|once` | Build test bundles (watch or single-run mode) |
+| `yarn t-run <YOUR_TESTS> dev\|once`   | Run tests (watch or single-run mode)          |
+| `yarn t-report`                       | Launch test report server                     |
+| `yarn t-aider`                        | Fix failing tests with AI                     |
+
+Example workflow:
+
+```bash
+# Initialize project
+yarn t-init
+
+# Write tests in test/*.test.ts
+
+# In terminal 1 - Build tests (watch mode)
+yarn t-build test/rectangle.test.ts dev
+
+# In terminal 2 - Run tests (watch mode)
+yarn t-run test/rectangle.test.ts dev
+
+# Or for single-run mode:
+yarn t-build test/rectangle.test.ts once
+yarn t-run test/rectangle.test.ts once
+
+# Get AI help with failures
+yarn t-aider
 ```
 
-### the "specification" aka ITestSpecification<I, O>
+## AI
 
-The test specification is the BDD tests logic. The specification implements BDD directives "Given", "When", and Then"
+Testeranto generates a "prompt" alongside test results. This prompt is passed to aider as input.
 
-```ts
-export const RectangleTesterantoBaseTestSpecification: ITestSpecification<
-  I,
-  O
-> = (Suite, Given, When, Then, Check) => {
-  return [
-    Suite.Default(
-      "Testing the Rectangle class",
-      {
-        // A "given" is a strict BDD test. It starts with an initial state, then executes the "whens" which update the test subject, and then executes the "thens" as a assertions.
-        test0: Given.Default(
-          // a list of features
-          ["https://api.github.com/repos/adamwong246/testeranto/issues/8"],
-          // a list of "whens"
-          [When.setWidth(4), When.setHeight(19)],
-          // a list of "thens"
-          [Then.getWidth(4), Then.getHeight(19)]
-        ),
-      },
-
-      [
-        // a "check" is a less strict style of test. Instead of lists of whens and thens, you get a function callback.
-        Check.Default("imperative style?!", [], async (rectangle) => {
-          Then.getWidth(2).thenCB(rectangle);
-          Then.getHeight(2).thenCB(rectangle);
-          When.setHeight(22).whenCB(rectangle);
-          Then.getHeight(232).thenCB(rectangle);
-        }),
-      ]
-    ),
-  ];
-};
 ```
+// input src files which can be edited by aider
+/add test/node.ts
 
-### the "interface" aka testInterface: Partial<IWebTestInterface<I>>
+// test report files that inform aider but should not be edited
+/read testeranto/reports/allTests/node/test/node/tests.json
+/read testeranto/reports/allTests/test/node/node/lint_errors.json
+/read testeranto/reports/allTests/test/node/node/type_errors.txt
 
-The test interface is code which is NOT BDD steps. The interface implements "before all", "after all", "before each", and "after each", all of which are optional. f
+// A list of features which can inform aider.
+/load testeranto/reports/allTests/node/test/node/featurePrompt.txt
 
-```ts
-export const RectangleTesterantoBaseInterface: IPartialInterface<I> = {
-  beforeEach: async (subject, i) => {
-    return i();
-  },
-  andWhen: async function (s, whenCB, tr, utils) {
-    return whenCB(s)(s, utils);
-  },
-  butThen: async (s, t, tr, pm) => {
-    return t(s, pm);
-  },
-};
+// tell the AI what to do
+/code Fix the failing tests described in testeranto/reports/allTests/node/test/node/tests.json. Correct any type signature errors described in the files testeranto/reports/allTests/test/node/node/type_errors.txt. Implement any method which throws "Function not implemented. Resolve the lint errors described in testeranto/reports/allTests/test/node/node/lint_errors.json"
 ```
 
-## the "test resource requirement" aka ITTestResourceRequest (optional)
+## "Features"
 
-The test resource requirement describes things that the test needs to run, namely network ports. It is optional, but you should add this argument if your test needs to rely upon network ports
+Testeranto connects "features" to tests. The features may be simple strings, but they can also take the form of local markdown files, or remote URLs to external feature tracking systems. For instance, this could be a jira ticket or a github issue. These features are used to inform the AI context.
 
 ```ts
-// TODO add example of test resource requirement
+import someMarkdownFile from "someMarkdownFile.md";
+
+...
+
+test0: Given.Default(
+  [
+    "https://api.github.com/repos/adamwong246/testeranto/issues/8",
+    "you can set the width and height of a Rectangle",
+    someMarkdownFile
+    ],
+
+  [When.setWidth(4), When.setHeight(19)],
+  [Then.getWidth(4), Then.getHeight(19)]
+),
+...
+
 ```
 
 ## Sidecars (COMING SOON)

package/bundle.js

@@ -2,11 +2,17 @@ import * as esbuild from 'esbuild'
 
 await esbuild.build({
   outExtension: { '.js': '.mjs' },
-  entryPoints: ['src/build.ts', 'src/run.ts', 'src/init-docs.ts', 'src/ReportServer.ts', 'src/esbuildConfigs/eslint-formatter-testeranto.ts'],
+  entryPoints: [
+    'src/build.ts',
+    'src/run.ts',
+    'src/init-docs.ts',
+    'src/ReportServer.ts',
+    'src/esbuildConfigs/eslint-formatter-testeranto.ts',
+    'src/mothership/index.ts'
+  ],
   bundle: true,
   format: "esm",
   platform: "node",
-  // outfile: 'dist/prebuild/cli.mjs'
   outdir: 'dist/prebuild',
   packages: "external",
   supported: {
@@ -18,79 +24,6 @@ await esbuild.build({
   },
 })
 
-// await esbuild.build({
-//   entryPoints: ['src/init-docs.ts'],
-//   bundle: true,
-//   format: "esm",
-//   platform: "node",
-//   // format: "node",
-//   outfile: 'dist/prebuild/init-docs.mjs',
-//   // external: ['crypto', 'os'],
-//   supported: {
-//     "dynamic-import": true,
-//   },
-
-//   banner: {
-//     js: `import { createRequire } from 'module';const require = createRequire(import.meta.url);`,
-//   },
-// })
-
-
-
-// await esbuild.build({
-//   entryPoints: ['src/cli2.ts'],
-//   bundle: true,
-//   format: "esm",
-//   platform: "node",
-//   outfile: 'dist/prebuild/cli2.mjs',
-//   packages: "external",
-//   supported: {
-//     "dynamic-import": true,
-//   },
-
-//   banner: {
-//     js: `import { createRequire } from 'module';const require = createRequire(import.meta.url);`,
-//   },
-// })
-
-
-
-
-// await esbuild.build({
-//   entryPoints: ['src/ReportServer.ts'],
-//   bundle: true,
-//   format: "esm",
-//   platform: "node",
-//   outfile: 'dist/prebuild/ReportServer.mjs',
-//   packages: "external",
-//   // supported: {
-//   //   "dynamic-import": true,
-//   // },
-
-//   // banner: {
-//   //   js: `import { createRequire } from 'module';const require = createRequire(import.meta.url);`,
-//   // },
-// })
-
-// await esbuild.build({
-//   entryPoints: ['src/esbuildConfigs/eslint-formatter-testeranto.ts'],
-//   bundle: true,
-//   format: "esm",
-//   platform: "node",
-//   outfile: 'dist/prebuild/eslint-formatter-testeranto.mjs',
-//   packages: "external",
-//   // supported: {
-//   //   "dynamic-import": true,
-//   // },
-
-//   // banner: {
-//   //   js: `import { createRequire } from 'module';const require = createRequire(import.meta.url);`,
-//   // },
-// })
-
-
-/////////////////////////
-
 await esbuild.build({
   entryPoints: ['src/ReportClient.tsx', 'src/TestReport.tsx', 'src/Project.tsx'],
   bundle: true,

package/dist/common/src/Init.js

@@ -3,6 +3,7 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
     return (mod && mod.__esModule) ? mod : { "default": mod };
 };
 Object.defineProperty(exports, "__esModule", { value: true });
+/* eslint-disable @typescript-eslint/no-unused-vars */
 const fs_1 = __importDefault(require("fs"));
 exports.default = async () => {
     [
@@ -14,6 +15,7 @@ exports.default = async () => {
         `testeranto/reports`,
         `testeranto/reports/`,
         `testeranto/features/`,
+        `testeranto/externalTests/`,
     ].forEach((f) => {
         try {
             fs_1.default.mkdirSync(`${process.cwd()}/${f}`);
@@ -25,7 +27,10 @@ exports.default = async () => {
     fs_1.default.copyFileSync(`node_modules/testeranto/dist/prebuild/Project.js`, `testeranto/Project.js`);
     fs_1.default.copyFileSync(`node_modules/testeranto/dist/prebuild/ReportClient.js`, `testeranto/ReportClient.js`);
     fs_1.default.copyFileSync(`node_modules/testeranto/dist/prebuild/Project.css`, `testeranto/Project.css`);
-    fs_1.default.copyFileSync(`node_modules/testeranto/dist/prebuild/ReportClient.css`, `testeranto/ReportClient.css`);
+    // fs.copyFileSync(
+    //   `node_modules/testeranto/dist/prebuild/ReportClient.css`,
+    //   `testeranto/ReportClient.css`
+    // );
     fs_1.default.copyFileSync(`node_modules/testeranto/dist/prebuild/TestReport.js`, `testeranto/TestReport.js`);
     fs_1.default.copyFileSync(`node_modules/testeranto/dist/prebuild/TestReport.css`, `testeranto/TestReport.css`);
 };

package/dist/common/src/Node.js

@@ -7,6 +7,7 @@ exports.NodeTesteranto = void 0;
 const core_js_1 = __importDefault(require("./lib/core.js"));
 const index_js_1 = require("./lib/index.js");
 const node_js_1 = require("./PM/node.js");
+let ipcfile;
 class NodeTesteranto extends core_js_1.default {
     constructor(input, testSpecification, testImplementation, testResourceRequirement, testInterface) {
         super(input, testSpecification, testImplementation, testResourceRequirement, testInterface, () => {
@@ -15,7 +16,7 @@ class NodeTesteranto extends core_js_1.default {
     }
     async receiveTestResourceConfig(partialTestResource) {
         const t = JSON.parse(partialTestResource);
-        const pm = new node_js_1.PM_Node(t);
+        const pm = new node_js_1.PM_Node(t, ipcfile);
         return await this.testJobs[0].receiveTestResourceConfig(pm);
         // const { failed, artifacts, logPromise, features } =
         //   await this.testJobs[0].receiveTestResourceConfig(pm);
@@ -29,15 +30,16 @@ const testeranto = async (input, testSpecification, testImplementation, testInte
     process.on("unhandledRejection", (reason, promise) => {
         console.error("Unhandled Rejection at:", promise, "reason:", reason);
         // Optionally, terminate the process or perform cleanup
+        // t.registerUncaughtPromise(reason, promise);
     });
     try {
-        console.log(process.argv);
+        ipcfile = process.argv[3];
         const f = await t.receiveTestResourceConfig(process.argv[2]);
-        console.error("goodbye node error", f.fails);
+        console.error("goodbye node with failures", f.fails);
         process.exit(f.fails);
     }
     catch (e) {
-        console.error("goodbye node error", e);
+        console.error("goodbye node with caught error", e);
         process.exit(-1);
         // fs.writeFileSync(`tests.json`, JSON.stringify(t.,
         //  null, 2));

package/dist/common/src/NodeSidecar.js

@@ -0,0 +1,15 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.NodeSidecar = void 0;
+const Sidecar_1 = require("./lib/Sidecar");
+const nodeSidecar_1 = require("./PM/nodeSidecar");
+class NodeSidecar extends Sidecar_1.Sidecar {
+    constructor(t) {
+        super();
+        this.pm = new nodeSidecar_1.PM_Node_Sidecar(t);
+        this.pm.start(this.stop).then(() => {
+            this.start(JSON.parse(process.argv[2]));
+        });
+    }
+}
+exports.NodeSidecar = NodeSidecar;