eleventy-test 1.1.0 → 2.0.0

package/README.md

@@ -1,44 +1,113 @@
 # eleventy-test
-Eleventy is hands-down my favourite static site generator. I also make [plugins for it](https://github.com/search?q=owner%3ADenperidge%20topic%3Aeleventy-plugin&type=repositories)! So [after a while of brute forcing my Eleventy testing](https://github.com/Denperidge/eleventy-auto-cache-buster/tree/14787bebe3bb73f4c6bd971196f3bec87812044f/tests), I thought an easier solution should exist. Hopefully this is that!
-- Build using multiple Eleventy versions & configurations for your tests (called scenarios) seperately. Returns the file contents per scenario output for easy access during testing.
-- Compatible with any test framework or method.
-- Automatic installation of latest/specific Eleventy version (if not found locally).
-- Aside from the Eleventy versions used to test, this package itself has **zero** dependencies when installing.
+ Multi-configuration testing for Eleventy plugins!
+ 
+- Build using **multiple Eleventy versions & configurations** for your tests (called *scenarios*) seperately
+- Returns the **file contents per scenario** output for easy access during testing, with **caching** for repeated usage.
+- Compatible with **any test framework or method**: [configure your directory and use one function](#using-the-plugin), that's it!
+- **Automatic installation** of either the **latest/specific or specific Eleventy versions** (*unless* they are found *locally*).
+- Aside from the Eleventy versions used to test, this package itself has **zero dependencies** when installing.
 
-Want to see how it is in action? For the dogfooding fans, you can see this library in action in this [library's tests](tests/)!
+Want to see how it is in action? For you [dogfooding](https://en.wikipedia.org/wiki/Eating_your_own_dog_food) fans, you can see this library in action in this [library's tests](tests/)!
 
 ## How-to
-### Use the plugin
+### Using the plugin
+Important concepts:
+- **Input** refers to an --input dir for Eleventy. For example, your tests' index.njk and stylesheet.css will be in an input dir
+- **Scenario** refers to a specific test scenario: with its own Eleventy version & config files (.eleventy.js). Optionally, a scenario can have its own input dir
+
 1. Install the plugin
-   ```bash
+   ```sh
    npm install --save-dev eleventy-test
    yarn add -D eleventy-test
    ```
-2. Create `${projectRoot}/tests/scenarios/`
-3. Create a subdirectory for each scenario you wish to set up. The directory name should start with the *exact* (1.0.2) or *major* (1) version. The format should be one of the following:
-    - `${projectRoot}/tests/scenarios/${eleventyVersion}--${label}/`
-    - `${projectRoot}/tests/scenarios/${eleventyVersion}/`
-3. Add an Eleventy configuration file to the scenario
-4. Use the buildScenarios function and its output as needed
+2. Create a `tests/` directory in your package root
+    ```sh
+    # Example structure:
+    {project root}/
+        package.json
+        tests/
+    ```
+2. Optionally, create the `tests/input/` directory: a global input that scenarios will use if they don't have their own input dir
+    ```sh
+    # Example structure:
+    {project root}/
+        package.json
+        tests/
+            input/
+                index.njk
+                stylesheet.css
+    ```
+3. Create the `tests/scenarios/` dir. This is where you'll configure your different scenarios
+    ```sh
+    # Example structure:
+    {project root}/
+        package.json
+        tests/
+            input/
+                index.njk
+                stylesheet.css
+            scenarios/
+    ```
+4. Create a subdirectory for each scenario you wish to set up:
+    - The directory name should start with the *exact* (1.0.2) or *major* (1) version of Eleventy you want to use
+    - Optionally, a label can be added
+    - The format should be one of the following (or see [this library for examples](tests/scenarios/)):
+        - `3.1.2--custom-label/`
+        - `3.0.0/`
+        - `2--example-label/`
+        - `3.1.2/`
+    ```sh
+    # Example structure:
+    {project root}/
+        package.json
+        tests/
+            input/
+                index.njk
+                stylesheet.css
+            scenarios/
+                # Will use Eleventy 3.1.2
+                3.1.2/
+                # Will use latest Eleventy 2.X
+                2--no-njk/
+    ```
+5. Add an Eleventy configuration file to each scenario, and optionally* an `input/` directory
+    ```
+    # Example structure:
+    {project root}/
+        package.json
+        tests/
+            input/
+                index.njk
+                stylesheet.css
+            scenarios/
+                3.1.2/
+                    .eleventy.js
+                2--no-njk/
+                    # This scenario will use its own input dir instead of the global one
+                    input/
+                        index.html
+                        index.css
+                    .eleventy.js
+    ```
+    *In case you don't have a global input dir configured, each scenario will need to have its own
+
+6. Within your testing framework of choice, simply run the `buildScenarios` function. This will handle installing the right Eleventy version(s) and reading + caching the file contents back to you. 
     ```js
     import { buildScenarios } from "eleventy-test";
     import test from "ava";
 
-    const resultsAsDict = await buildScenarios({
-        projectRoot: cwd(),
-        returnArray: false
+    const results = await buildScenarios({
+        returnArray: false,  // return as dict instead of array
+        enableDebug: false
     });
 
     test("Check if index.html is consistent across builds", async t => {
         t.deepEqual(
-            await results["3--example"].getFileContent("/index.html"), 
-            await results["3.0.0--identical-example"].getFileContent("/index.html"));
+            await results["3.1.2"].getFileContent("/index.html"), 
+            await results["2--no-njk"].getFileContent("/index.html"));
     });
     ```
 
-And that's it! eleventy-test will handle installing the right versons and reading (and caching) the file contents back to you.
-
-
 > Note: you might want to add `eleventy-test-out/` to your .gitignore file!
 
 ### Run/develop/test locally
@@ -48,11 +117,15 @@ git clone https://github.com/Denperidge/eleventy-test
 cd eleventy-test
 yarn install
 
-yarn watch  # Watch for changes
+yarn watch  # Build & watch for changes. Should be active during development/testing of eleventy-test
 yarn build  # Build
 yarn start  # Run built js as module (see bottom of index.ts require.main === module)
-yarn test  # Run the tests from tests/test.mjs
+yarn test  # Using the built js, run the tests from tests/test.mjs
 ```
 
+## Explanation
+### Why?
+Eleventy is hands-down my favourite static site generator. I also make [plugins for it](https://github.com/search?q=owner%3ADenperidge%20topic%3Aeleventy-plugin&type=repositories)! So [after a while of brute forcing my Eleventy testing for different setups](https://github.com/Denperidge/eleventy-auto-cache-buster/blob/14787bebe3bb73f4c6bd971196f3bec87812044f/tests/.eleventy.js), I thought an easier solution should exist. Hopefully this is that!
+
 ## License
 This project is licensed under the [MIT License](LICENSE).

package/dist/ScenarioOutput.d.ts

@@ -1,12 +1,36 @@
-export declare function recursiveFindFiles(dir: string, files?: string[]): string[];
-export default class ScenarioOutput {
+/**
+ *
+ * @param dir directory to search in
+ * @param files currently found files
+ * @returns promise for array of filepaths found in dir
+ */
+export declare function _recursiveFindFiles(dir: string, files?: string[]): Promise<string[]>;
+/**
+ * @class easy listing & reading of scenario outputs/eleventy-test-out files
+ */
+export declare class ScenarioOutput {
+    /** path to this scenario's eleventy-test-out */
     eleventyOutputDir: string;
+    /** title of this scenario */
     title: string;
-    private _files;
+    private _filepaths;
     private cache;
     constructor(pEleventyOutputDir: string, pTitle: string);
-    get files(): {
-        [key: string]: () => string;
-    };
-    getFileContent(filename: any): Promise<string>;
+    /**
+     * @param pEleventyOutputDir path to "eleventy-test-out"
+     * @param pTitle scenario title
+     */
+    static create(pEleventyOutputDir: string, pTitle: string): Promise<ScenarioOutput>;
+    /**
+    * @returns set of relative paths to output files
+     */
+    get filepaths(): Set<string>;
+    /**
+     * Gets file contents from internal cache,
+     * reading file contents into the cache if needed
+     *
+     * @param filepath relative path to file to read
+     * @returns promise for text contents
+     */
+    getFileContent(filepath: string): Promise<string>;
 }

package/dist/debug.d.ts

@@ -0,0 +1,13 @@
+/**
+ * Functions for debug logging
+ * - enable/disable debug logs: @see _setDebug
+ * - log to console if debugging is enabled: @see debug
+ */
+/**
+ * @param enabled if true, enabled debug logging
+ */
+export declare function _setDebug(enabled: boolean): void;
+/**
+ * @param message If debug is enabled (@see setDebug), log messages to console
+ */
+export declare function debug(...message: Array<String>): void;

package/dist/eleventyUtils.d.ts

@@ -1,12 +1,77 @@
-import ScenarioOutput from "./ScenarioOutput";
-export declare function determineInstalledEleventyVersions(projectRoot?: string): Promise<{
+import { ScenarioOutput } from "./ScenarioOutput";
+import { IgitHubApiTags } from "./githubApi";
+/**
+ * Asynchronously check if passed filepath exists
+ *
+ * @param filepath filepath to check
+ * @returns true if file exists, false if it doesn't
+ * @throws any non-ENOENT errors from fsPromises.stat
+ */
+export declare function _exists(filepath: string): Promise<Boolean>;
+/**
+ *
+ * @param projectRoot project root directory of which to check the node_modules/ dir.
+ * @default process.cwd()
+ * @returns promise for a dictionary in the style of {"3.0.0": "/path/to/@11ty/eleventy3.0.0"}
+ */
+export declare function _determineInstalledEleventyVersions(projectRoot?: string): Promise<{
     [key: string]: string;
 }>;
-export declare function ensureEleventyExists(projectRoot: string, eleventyVersion: string): Promise<string>;
-export declare function buildEleventy({ eleventyVersion, scenarioDir, scenarioName, projectRoot, globalInputDir }: {
-    eleventyVersion: any;
-    scenarioDir: any;
-    scenarioName: any;
+/**
+ * Return first/latest version using the major version
+ *
+ * @param majorVersion major version to lookup (example: 2)
+ * @param allVersions array including all Eleventy versions
+ * @returns latest semantic Eleventy version that is not an alpha/beta (example: 2.0.1)
+ * @throws {Error} if no major versions can be found
+ * @see _getReleasedEleventyVersions
+ * @see IgitHubApiTags
+*/
+export declare function _majorToSemanticEleventyVersion(majorVersion: string, allVersions: Array<IgitHubApiTags>): string;
+/**
+ *
+ * @param dirname directory name from the scenario
+ * @param versions array of Eleventy release tags
+ * @returns promise for a string of the extracted eleventy version; even if only a major number is provided
+ * @see _getReleasedEleventyVersions
+ * @see IgitHubApiTags
+ */
+export declare function _dirnameToEleventyVersion(dirname: string, versions: Array<IgitHubApiTags>): string;
+/**
+ *
+ * @param eleventyVersion semantic version to look for (for example: "3", "3.1.2")
+ * @param projectRoot project root directory
+ * @default process.cwd()
+ *
+ * @param filename if filename exists, use install command below (for example: "package-lock.json", "yarn.lock")
+ * @param command command to prefix to install eleventy (for example: "yarn add", "npm install")
+ * @returns promise for false/true, depending on whether eleventy install was succesful
+ */
+export declare function _installEleventyIfPkgManagerFound(eleventyVersion: string, projectRoot: string, filename: string, command: string): Promise<boolean>;
+/**
+ *
+ * @param eleventyVersion semantic version to look for (for example "3", "3.0.0")
+ * @param projectRoot project root directory
+ * @default process.cwd()
+ * @returns promise for the install directory of specified eleventy version
+ */
+export declare function _ensureEleventyExists(eleventyVersion: string, projectRoot?: string): Promise<string>;
+interface IbuildEleventyArgs {
     projectRoot?: string;
-    globalInputDir: any;
-}): Promise<ScenarioOutput>;
+    globalInputDir?: string;
+    scenarioDir: string;
+    scenarioName: string;
+}
+/**
+ * **Note:** the arguments below need to be passed in an object. @see IbuildEleventyArgs
+ *
+ * @param scenarioDir path towards directory that holds all test scenarios
+ * @param scenarioName name of the test scenario to run
+ * @param projectRoot project root directory
+ * @default process.cwd()
+ * @param globalInputDir path to the input directory to be used if scenarios do not provide their own
+ *
+ * @returns promise for the resulting @see ScenarioOutput
+ */
+export declare function buildEleventy({ projectRoot, globalInputDir, scenarioDir, scenarioName }: IbuildEleventyArgs): Promise<ScenarioOutput>;
+export {};

package/dist/githubApi.d.ts

@@ -0,0 +1,39 @@
+export declare const DEFAULT_CACHE_PATH = "tests/eleventy-test-out/cache.json";
+/**
+ *
+ * @param dataToSave variable you want to save in cache
+ * @param filepath path to json file the cache should be written to
+ * @default tests/eleventy-test-out/cache.json
+ * @returns void promise
+ */
+export declare function _cacheWrite(dataToSave: any, filepath?: string): Promise<void>;
+/**
+ * This function works like as follows:
+ * - Check for an existing cache
+ * - If the cache exists, check if its outdated (not from today)
+ * - If it's not outdated, return cache data
+ * - If the cache doesn't exist, run passed function async
+ * - Store returned passed function data in cache & return data
+ *
+ * @param fetchData async function to run to get new cache data
+ * @param filepath path to cache json file
+ * @returns promise for the cache data
+ */
+export declare function _cache(fetchData: Function, filepath?: string): Promise<any>;
+export interface IgitHubApiTags {
+    name: string;
+    zipball_url: string;
+    tarball_url: string;
+    node_id: string;
+    commit: {
+        sha: string;
+        url: string;
+    };
+}
+/**
+ * Return data from GitHub API, from the passed ?page= parameter
+ *
+ * @returns GitHub tag info. @see IgitHubApiTags
+ */
+export declare function _requestReleasedEleventyVersions(page?: number): Promise<Array<IgitHubApiTags>>;
+export declare function _getReleasedEleventyVersions(): Promise<Array<IgitHubApiTags>>;

package/dist/index.d.ts

@@ -1,10 +1,13 @@
-import ScenarioOutput from "./ScenarioOutput";
+import { ScenarioOutput } from "./ScenarioOutput";
 export * from "./eleventyUtils";
+export * from "./ScenarioOutput";
+export * from "./githubApi";
 interface IbuildScenariosArgs {
-    projectRoot: string;
+    projectRoot?: string;
     returnArray?: boolean;
     scenariosDir?: string;
     globalInputDir?: string;
+    enableDebug?: boolean;
 }
 interface IbuildScenariosArrayArgs extends IbuildScenariosArgs {
     returnArray?: true;
@@ -12,6 +15,18 @@ interface IbuildScenariosArrayArgs extends IbuildScenariosArgs {
 interface IbuildScenariosDictArgs extends IbuildScenariosArgs {
     returnArray?: false;
 }
+/**
+ * **Note:** the arguments below need to be passed in an object. @see IbuildScenariosArgs
+ * @param projectRoot project root directory
+ * @default process.cwd()
+ * @param returnArray if set to true, return array
+ * @param scenariosDir path to directory that holds all scenarios
+ * @param globalInputDir path to the input directory to be used if scenarios do not provide their own
+ * @param enableDebug enable debug logging if true
+ *
+ * @returns (if returnArray=true) return scenario outputs as array
+ * @returns (if returnArray=false) return scenario outputs in a dict, using scenario names as key
+ */
 export declare function buildScenarios(opts: IbuildScenariosArrayArgs): Promise<ScenarioOutput[]>;
 export declare function buildScenarios(opts: IbuildScenariosDictArgs): Promise<{
     [key: string]: ScenarioOutput;

package/dist/index.js

@@ -17,98 +17,261 @@ var __copyProps = (to, from, except, desc) => {
 var __toCommonJS = (mod) => __copyProps(__defProp({}, "__esModule", { value: true }), mod);
 
 // src/index.ts
-var src_exports = {};
-__export(src_exports, {
+var index_exports = {};
+__export(index_exports, {
+  DEFAULT_CACHE_PATH: () => DEFAULT_CACHE_PATH,
+  ScenarioOutput: () => ScenarioOutput,
+  _cache: () => _cache,
+  _cacheWrite: () => _cacheWrite,
+  _determineInstalledEleventyVersions: () => _determineInstalledEleventyVersions,
+  _dirnameToEleventyVersion: () => _dirnameToEleventyVersion,
+  _ensureEleventyExists: () => _ensureEleventyExists,
+  _exists: () => _exists,
+  _getReleasedEleventyVersions: () => _getReleasedEleventyVersions,
+  _installEleventyIfPkgManagerFound: () => _installEleventyIfPkgManagerFound,
+  _majorToSemanticEleventyVersion: () => _majorToSemanticEleventyVersion,
+  _recursiveFindFiles: () => _recursiveFindFiles,
+  _requestReleasedEleventyVersions: () => _requestReleasedEleventyVersions,
   buildEleventy: () => buildEleventy,
-  buildScenarios: () => buildScenarios,
-  determineInstalledEleventyVersions: () => determineInstalledEleventyVersions,
-  ensureEleventyExists: () => ensureEleventyExists
+  buildScenarios: () => buildScenarios
 });
-module.exports = __toCommonJS(src_exports);
+module.exports = __toCommonJS(index_exports);
+var import_path4 = require("path");
 var import_process2 = require("process");
-var import_path3 = require("path");
-var import_promises2 = require("fs/promises");
-var import_https = require("https");
+var import_promises4 = require("fs/promises");
 
 // src/eleventyUtils.ts
 var import_child_process = require("child_process");
-var import_fs2 = require("fs");
-var import_promises = require("fs/promises");
-var import_path2 = require("path");
+var import_promises3 = require("fs/promises");
+var import_path3 = require("path");
 var import_process = require("process");
 
+// src/debug.ts
+var debugEnabled = false;
+function _setDebug(enabled) {
+  debugEnabled = enabled;
+}
+function debug(...message) {
+  if (debugEnabled) {
+    message.forEach((msg) => console.log(msg));
+  }
+}
+
 // src/ScenarioOutput.ts
 var import_path = require("path");
-var import_fs = require("fs");
-function recursiveFindFiles(dir, files = []) {
+var import_promises = require("fs/promises");
+async function _recursiveFindFiles(dir, files = []) {
   const foundDirs = [];
-  (0, import_fs.readdirSync)(dir).forEach((name) => {
+  const entries = await (0, import_promises.readdir)(dir);
+  await Promise.all(entries.map(async (name) => {
     const path = (0, import_path.join)(dir, name);
-    const stat = (0, import_fs.lstatSync)(path);
-    if (stat.isDirectory()) {
+    const stat2 = await (0, import_promises.lstat)(path);
+    if (stat2.isDirectory()) {
       foundDirs.push(path);
-    } else if (stat.isFile()) {
+    } else if (stat2.isFile()) {
       files.push(path);
     }
-  });
-  foundDirs.forEach((dir2) => {
-    files = recursiveFindFiles(dir2, files);
-  });
+  }));
+  for (const subDir of foundDirs) {
+    files = await _recursiveFindFiles(subDir, files);
+  }
   return files;
 }
-var ScenarioOutput = class {
+var ScenarioOutput = class _ScenarioOutput {
+  /** path to this scenario's eleventy-test-out */
   eleventyOutputDir;
+  /** title of this scenario */
   title;
-  _files;
+  _filepaths;
   cache;
   constructor(pEleventyOutputDir, pTitle) {
-    this._files = {};
+    this._filepaths = /* @__PURE__ */ new Set();
     this.cache = {};
     this.title = pTitle;
     this.eleventyOutputDir = pEleventyOutputDir;
-    recursiveFindFiles(this.eleventyOutputDir).forEach((filepath) => {
-      this._files[filepath.replace(this.eleventyOutputDir, "")] = function() {
-        return (0, import_fs.readFileSync)(filepath, { encoding: "utf-8" });
-      };
-    });
   }
-  get files() {
-    return this._files;
+  /**
+   * @param pEleventyOutputDir path to "eleventy-test-out"
+   * @param pTitle scenario title
+   */
+  static async create(pEleventyOutputDir, pTitle) {
+    const instance = new _ScenarioOutput(pEleventyOutputDir, pTitle);
+    const allFiles = await _recursiveFindFiles(instance.eleventyOutputDir);
+    for (const filepath of allFiles) {
+      instance._filepaths.add(filepath.replace(instance.eleventyOutputDir, ""));
+    }
+    return instance;
   }
-  getFileContent(filename) {
-    return new Promise(async (resolve, reject) => {
-      if (!Object.keys(this.cache).includes(filename)) {
-        this.cache[filename] = this._files[filename]();
-      }
-      resolve(this.cache[filename]);
-    });
+  /**
+  * @returns set of relative paths to output files
+   */
+  get filepaths() {
+    return this._filepaths;
+  }
+  /**
+   * Gets file contents from internal cache,
+   * reading file contents into the cache if needed
+   *
+   * @param filepath relative path to file to read
+   * @returns promise for text contents
+   */
+  async getFileContent(filepath) {
+    if (!this._filepaths.has(filepath)) {
+      throw new Error(`Can't find "${filepath}" in files. Available files: ${Array.from(this._filepaths).join(", ")}`);
+    }
+    if (!(filepath in this.cache)) {
+      this.cache[filepath] = await (0, import_promises.readFile)((0, import_path.join)(this.eleventyOutputDir, filepath), { encoding: "utf-8" });
+    }
+    return this.cache[filepath];
   }
 };
 
+// src/githubApi.ts
+var import_promises2 = require("fs/promises");
+var import_path2 = require("path");
+var import_https = require("https");
+var DEFAULT_CACHE_PATH = "tests/eleventy-test-out/cache.json";
+async function _cacheWrite(dataToSave, filepath = DEFAULT_CACHE_PATH) {
+  const data = {
+    "datetime": Date.now() - 100,
+    // -100 to fix race condition for tests
+    "data": dataToSave
+  };
+  const dir = (0, import_path2.dirname)(filepath);
+  if (!await _exists(dir)) {
+    await (0, import_promises2.mkdir)(dir, { recursive: true });
+  }
+  return (0, import_promises2.writeFile)(filepath, JSON.stringify(data));
+}
+async function _cache(fetchData, filepath = DEFAULT_CACHE_PATH) {
+  return new Promise(async (resolve, reject) => {
+    if (await _exists(filepath)) {
+      const cache = JSON.parse(await (0, import_promises2.readFile)(filepath, { encoding: "utf-8" }));
+      const cacheDatetime = new Date(cache.datetime);
+      const today = /* @__PURE__ */ new Date();
+      if (cacheDatetime.toDateString() == today.toDateString()) {
+        resolve(cache.data);
+        return;
+      }
+    }
+    const cacheData = await fetchData();
+    await _cacheWrite(cacheData, filepath);
+    resolve(cacheData);
+  });
+}
+async function _requestReleasedEleventyVersions(page = 1) {
+  debug("Pulling Eleventy tags...");
+  return new Promise((resolve, reject) => {
+    (0, import_https.get)({
+      hostname: "api.github.com",
+      path: `/repos/11ty/eleventy/tags?per_page=100&page=${page}`,
+      headers: {
+        "User-Agent": "Mozilla/5.0"
+      }
+    }, (res) => {
+      let data = [];
+      res.on("data", (chunk) => {
+        data.push(chunk);
+      }).on("end", async () => {
+        debug("Parsing Eleventy tags API response...");
+        let tags = JSON.parse(
+          Buffer.concat(data).toString("utf-8")
+        );
+        resolve(tags);
+      }).on("error", (err) => {
+        throw err;
+      });
+    });
+  });
+}
+async function _getReleasedEleventyVersions() {
+  return new Promise((resolve, reject) => {
+    let page = 1;
+    let out = [];
+    const timer = setInterval(async () => {
+      const req = await _requestReleasedEleventyVersions(page);
+      page++;
+      out = out.concat(req);
+      if (req.length < 100) {
+        clearInterval(timer);
+        resolve(out);
+      }
+    }, 1500);
+  });
+}
+
 // src/eleventyUtils.ts
-async function determineInstalledEleventyVersions(projectRoot = (0, import_process.cwd)()) {
-  let eleventyPkgs = await (0, import_promises.readdir)((0, import_path2.join)(projectRoot, "node_modules/@11ty/"));
-  const eleventyRegex = new RegExp(/eleventy(\d|$)/m);
-  eleventyPkgs = eleventyPkgs.filter((name) => eleventyRegex.test(name));
-  const versions2 = {};
-  for (let i = 0; i < eleventyPkgs.length; i++) {
-    const eleventyPkg = eleventyPkgs[i];
-    const eleventyPkgDir = (0, import_path2.join)(projectRoot, "node_modules/@11ty/", eleventyPkg);
-    const version = JSON.parse(
-      await (0, import_promises.readFile)(
-        (0, import_path2.join)(eleventyPkgDir, "package.json"),
-        { encoding: "utf-8" }
-      )
-    ).version;
-    versions2[version] = eleventyPkgDir;
+async function _exists(filepath) {
+  return new Promise((resolve, reject) => {
+    (0, import_promises3.stat)(filepath).then((stats) => {
+      resolve(true);
+    }).catch((err) => {
+      if (err.code && err.code == "ENOENT") {
+        resolve(false);
+      } else {
+        reject(err);
+      }
+    });
+  });
+}
+async function _determineInstalledEleventyVersions(projectRoot = (0, import_process.cwd)()) {
+  const eleventyPkgsDir = (0, import_path3.join)(projectRoot, "node_modules/@11ty/");
+  const versions = {};
+  debug("Determining installed Eleventy versions in " + eleventyPkgsDir);
+  if (await _exists(eleventyPkgsDir)) {
+    let eleventyPkgs = await (0, import_promises3.readdir)(eleventyPkgsDir);
+    debug(`Found the following installed packages from @11ty: ${eleventyPkgs}`);
+    const eleventyRegex = new RegExp(/eleventy(\d|$)/m);
+    eleventyPkgs = eleventyPkgs.filter((name) => eleventyRegex.test(name));
+    debug(`Filtered non-main-eleventy packages. Results: ${eleventyPkgs}`);
+    for (let i = 0; i < eleventyPkgs.length; i++) {
+      const eleventyPkg = eleventyPkgs[i];
+      const eleventyPkgDir = (0, import_path3.join)(projectRoot, "node_modules/@11ty/", eleventyPkg);
+      const version = JSON.parse(
+        await (0, import_promises3.readFile)(
+          (0, import_path3.join)(eleventyPkgDir, "package.json"),
+          { encoding: "utf-8" }
+        )
+      ).version;
+      versions[version] = eleventyPkgDir;
+      debug(`Found ${version} at ${eleventyPkgDir}`);
+    }
   }
-  return versions2;
+  return versions;
 }
-async function installEleventyIfPkgManagerFound(eleventyVersion, projectRoot, filename, command) {
+function _majorToSemanticEleventyVersion(majorVersion, allVersions) {
+  for (let i = 0; i < allVersions.length; i++) {
+    const version = allVersions[i];
+    debug("Checking " + version);
+    if (!version.name.includes("-") && version.name[1] == majorVersion) {
+      const eleventyVersion = version.name.substring(1);
+      debug("Determined latest of relevant major version for: " + eleventyVersion);
+      return eleventyVersion;
+    }
+  }
+  const versions = allVersions.map((version) => {
+    return version.name;
+  });
+  throw new Error(`Couldn't determine Eleventy version from ${majorVersion}
+versions: ${versions.join(",")}`);
+}
+function _dirnameToEleventyVersion(dirname2, versions) {
+  let eleventyVersion = dirname2.includes("@") ? dirname2.substring(dirname2.lastIndexOf("@") + 1) : dirname2;
+  debug(`eleventyVersion from dirname: ${eleventyVersion}`);
+  if (eleventyVersion.length < 5) {
+    debug("eleventyVersion length is under 5, and as such not a full semantic version. Determining latest...");
+    eleventyVersion = _majorToSemanticEleventyVersion(eleventyVersion[0], versions);
+  }
+  return eleventyVersion;
+}
+async function _installEleventyIfPkgManagerFound(eleventyVersion, projectRoot = (0, import_process.cwd)(), filename, command) {
+  debug(`Attempting to find a package manager to install Eleventy ${eleventyVersion} with`);
   return new Promise(async (resolve, reject) => {
-    if ((0, import_fs2.existsSync)((0, import_path2.join)(projectRoot, filename))) {
+    if (await _exists((0, import_path3.join)(projectRoot, filename))) {
       try {
+        debug("Running Eleventy " + eleventyVersion);
         (0, import_child_process.execSync)(`${command} @11ty/eleventy${eleventyVersion}@npm:@11ty/eleventy@${eleventyVersion}`, { cwd: projectRoot });
+        debug("Done running Eleventy " + eleventyVersion);
         resolve(true);
       } catch (e) {
         throw e;
@@ -119,22 +282,23 @@ async function installEleventyIfPkgManagerFound(eleventyVersion, projectRoot, fi
     }
   });
 }
-async function ensureEleventyExists(projectRoot, eleventyVersion) {
+async function _ensureEleventyExists(eleventyVersion, projectRoot = (0, import_process.cwd)()) {
+  debug(`Ensuring Eleventy ${eleventyVersion} exists`);
   return new Promise(async (resolve, reject) => {
-    const versions2 = await determineInstalledEleventyVersions(projectRoot);
-    if (Object.keys(versions2).includes(eleventyVersion)) {
-      resolve(versions2[eleventyVersion]);
+    const versions = await _determineInstalledEleventyVersions(projectRoot);
+    if (Object.keys(versions).includes(eleventyVersion)) {
+      resolve(versions[eleventyVersion]);
     } else {
       console.log(`Eleventy version ${eleventyVersion} could not be found. Installing...`);
-      const eleventyDir = (0, import_path2.join)(projectRoot, "node_modules/@11ty/eleventy" + eleventyVersion);
-      if (await installEleventyIfPkgManagerFound(
+      const eleventyDir = (0, import_path3.join)(projectRoot, "node_modules/@11ty/eleventy" + eleventyVersion);
+      if (await _installEleventyIfPkgManagerFound(
         eleventyVersion,
         projectRoot,
         "package-lock.json",
         "npm install --save-dev"
       )) {
         resolve(eleventyDir);
-      } else if (await installEleventyIfPkgManagerFound(
+      } else if (await _installEleventyIfPkgManagerFound(
         eleventyVersion,
         projectRoot,
         "yarn.lock",
@@ -148,39 +312,47 @@ async function ensureEleventyExists(projectRoot, eleventyVersion) {
   });
 }
 async function buildEleventy({
-  eleventyVersion,
-  scenarioDir,
-  scenarioName,
   projectRoot = (0, import_process.cwd)(),
-  globalInputDir
+  globalInputDir,
+  scenarioDir,
+  scenarioName
 }) {
+  const versions = await _cache(_getReleasedEleventyVersions);
+  debug("Parsing Eleventy version of scenario " + scenarioDir);
+  let eleventyVersion = _dirnameToEleventyVersion(scenarioName, versions);
+  debug("Running buildEleventy", "scenarioDir: " + scenarioDir, "Eleventy version: " + eleventyVersion);
   return new Promise(async (resolve, reject) => {
-    const eleventyDir = await ensureEleventyExists(projectRoot, eleventyVersion);
+    debug("Finding package for Eleventy " + eleventyVersion);
+    const eleventyDir = await _ensureEleventyExists(eleventyVersion, projectRoot);
+    debug("Found package for Eleventy " + eleventyVersion);
     const bin = JSON.parse(
-      await (0, import_promises.readFile)(
-        (0, import_path2.join)(eleventyDir, "package.json"),
+      await (0, import_promises3.readFile)(
+        (0, import_path3.join)(eleventyDir, "package.json"),
         { encoding: "utf-8" }
       )
     ).bin.eleventy;
-    const pathToBin = (0, import_path2.join)(eleventyDir, bin);
-    const scenarioInputDir = (0, import_path2.join)(scenarioDir, "input");
+    const pathToBin = (0, import_path3.join)(eleventyDir, bin);
+    debug(`Found entrypoint for Eleventy ${eleventyVersion} at ${pathToBin}`);
+    const scenarioInputDir = (0, import_path3.join)(scenarioDir, "input");
     let inputDir;
-    try {
-      await (0, import_promises.access)(scenarioInputDir);
+    debug(`Checking whether to use scenario (${scenarioInputDir}) or global input (${globalInputDir})...`);
+    if (await _exists(scenarioInputDir)) {
+      debug("Using scenario input");
       inputDir = scenarioInputDir;
-    } catch {
-      try {
-        await (0, import_promises.access)(globalInputDir);
-        inputDir = globalInputDir;
-      } catch {
-      }
+    } else if (globalInputDir && await _exists(globalInputDir)) {
+      debug("Using global input dir");
+      inputDir = globalInputDir;
     }
+    debug("inputDir: " + inputDir);
     if (inputDir == void 0) {
-      throw Error("inputDir is undefined!");
+      throw new Error("inputDir is undefined! Either create a global input dir or one for the scenario specifically");
     }
-    const outputDir = (0, import_path2.join)(scenarioDir, "eleventy-test-out");
-    await (0, import_promises.rm)(outputDir, { force: true, recursive: true });
+    const outputDir = (0, import_path3.join)(scenarioDir, "eleventy-test-out");
+    debug("Cleaning old test output");
+    await (0, import_promises3.rm)(outputDir, { force: true, recursive: true });
+    debug("Cleaned");
     try {
+      debug("Creating Eleventy process...");
       const out = (0, import_child_process.fork)(
         pathToBin,
         ["--input", inputDir, "--output", outputDir],
@@ -190,7 +362,8 @@ async function buildEleventy({
         console.log(msg);
       });
       out.on("close", async (code) => {
-        resolve(new ScenarioOutput(outputDir, scenarioName));
+        debug(`Eleventy ${eleventyVersion}/${scenarioName} finished`);
+        resolve(await ScenarioOutput.create(outputDir, scenarioName));
       });
     } catch (e) {
       throw e;
@@ -199,74 +372,34 @@ async function buildEleventy({
 }
 
 // src/index.ts
-var versions;
-async function scenarioDirnameToEleventyVersion(scenarioDirname) {
-  let eleventyVersion = scenarioDirname.includes("--") ? scenarioDirname.split("--")[0] : scenarioDirname;
-  if (eleventyVersion.length < 5) {
-    const scenarioMajorVersion = scenarioDirname[0];
-    if (versions == void 0) {
-      console.log("Pulling Eleventy tags...");
-      versions = await new Promise((resolve, reject) => {
-        (0, import_https.get)({
-          hostname: "api.github.com",
-          path: "/repos/11ty/eleventy/tags",
-          headers: {
-            "User-Agent": "Mozilla/5.0"
-          }
-        }, (res) => {
-          let data = [];
-          res.on("data", (chunk) => {
-            data.push(chunk);
-          }).on("end", () => {
-            console.log("Parsing API response...");
-            resolve(
-              JSON.parse(
-                Buffer.concat(data).toString("utf-8")
-              )
-            );
-          }).on("error", (err) => {
-            throw err;
-          });
-        });
-      });
-    }
-    for (let i = 0; i < versions.length; i++) {
-      const version = versions[i];
-      if (!version.name.includes("-") && version.name[1] == scenarioMajorVersion) {
-        eleventyVersion = version.name.substring(1);
-        break;
-      }
-    }
-  }
-  return eleventyVersion;
-}
-async function buildScenarios({ projectRoot = (0, import_process2.cwd)(), returnArray = true, scenariosDir = "tests/scenarios/", globalInputDir = "tests/input" }) {
+async function buildScenarios({ projectRoot = (0, import_process2.cwd)(), returnArray = true, scenariosDir = "tests/scenarios/", globalInputDir = "tests/input", enableDebug = false }) {
+  _setDebug(enableDebug);
+  debug("If you can see this, debugging has been enabled. Starting buildScenarios");
   return new Promise(async (resolve, reject) => {
-    scenariosDir = (0, import_path3.isAbsolute)(scenariosDir) ? scenariosDir : (0, import_path3.join)(projectRoot, scenariosDir);
-    globalInputDir = (0, import_path3.isAbsolute)(globalInputDir) ? globalInputDir : (0, import_path3.join)(projectRoot, globalInputDir);
-    try {
-      await (0, import_promises2.access)(globalInputDir);
-    } catch {
-      globalInputDir = "undefined";
+    scenariosDir = (0, import_path4.isAbsolute)(scenariosDir) ? scenariosDir : (0, import_path4.join)(projectRoot, scenariosDir);
+    if (globalInputDir) {
+      globalInputDir = (0, import_path4.isAbsolute)(globalInputDir) ? globalInputDir : (0, import_path4.join)(projectRoot, globalInputDir);
     }
+    debug(`scenariosDir: ${scenariosDir}`, `globalInputDir: ${globalInputDir}`);
     try {
-      const scenarioDirs = await (0, import_promises2.readdir)(scenariosDir, { recursive: false, encoding: "utf-8" });
+      const scenarioDirs = await (0, import_promises4.readdir)(scenariosDir, { recursive: false, encoding: "utf-8" });
       const scenarioOutputs = [];
+      debug(`Found scenario dirs: ${scenarioDirs}`);
       for (let i = 0; i < scenarioDirs.length; i++) {
         const scenarioDirname = scenarioDirs[i];
-        const scenarioDir = (0, import_path3.join)(scenariosDir, scenarioDirname);
-        let scenarioEleventyVersion = await scenarioDirnameToEleventyVersion(scenarioDirname);
+        const scenarioDir = (0, import_path4.join)(scenariosDir, scenarioDirname);
         scenarioOutputs.push(await buildEleventy({
-          eleventyVersion: scenarioEleventyVersion,
-          scenarioName: scenarioDirname,
-          globalInputDir,
           projectRoot,
+          globalInputDir,
+          scenarioName: scenarioDirname,
           scenarioDir
         }));
       }
       if (returnArray) {
+        debug("Returning as array...");
         resolve(scenarioOutputs);
       } else {
+        debug("Returning as object...");
         const returnDict = {};
         scenarioOutputs.forEach((scenarioOutput) => {
           returnDict[scenarioOutput.title] = scenarioOutput;
@@ -280,13 +413,25 @@ async function buildScenarios({ projectRoot = (0, import_process2.cwd)(), return
 }
 if (require.main === module) {
   buildScenarios({
-    projectRoot: (0, import_process2.cwd)()
+    projectRoot: (0, import_process2.cwd)(),
+    enableDebug: true
   });
 }
 // Annotate the CommonJS export names for ESM import in node:
 0 && (module.exports = {
+  DEFAULT_CACHE_PATH,
+  ScenarioOutput,
+  _cache,
+  _cacheWrite,
+  _determineInstalledEleventyVersions,
+  _dirnameToEleventyVersion,
+  _ensureEleventyExists,
+  _exists,
+  _getReleasedEleventyVersions,
+  _installEleventyIfPkgManagerFound,
+  _majorToSemanticEleventyVersion,
+  _recursiveFindFiles,
+  _requestReleasedEleventyVersions,
   buildEleventy,
-  buildScenarios,
-  determineInstalledEleventyVersions,
-  ensureEleventyExists
+  buildScenarios
 });

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "eleventy-test",
-  "version": "1.1.0",
+  "version": "2.0.0",
   "description": "Multi-configuration testing for Eleventy plugins",
   "license": "MIT",
   "repository": "https://github.com/Denperidge/eleventy-test.git",
@@ -15,19 +15,23 @@
     "build": "npm-run-all --serial build:js build:types",
     "build:js": "esbuild src/index.ts --bundle --outfile=dist/index.js --platform=node",
     "build:types": "tsc --declaration src/index.ts --emitDeclarationOnly --outDir ./dist",
-    "test": "ava tests/test.mjs --timeout=90s"
+    "test": "ava tests/test.*.mjs --timeout=90s",
+    "test:unit": "ava tests/test.unit.mjs",
+    "test:system": "ava tests/test.system.mjs --timeout=90s"
   },
   "devDependencies": {
-    "@11ty/eleventy": "^3.0.0",
+    "@11ty/eleventy": "^3.1.2",
     "@11ty/eleventy1.0.2": "npm:@11ty/eleventy@1.0.2",
+    "@11ty/eleventy2": "npm:@11ty/eleventy@2",
     "@11ty/eleventy2.0.0-canary.8": "npm:@11ty/eleventy@2.0.0-canary.8",
     "@11ty/eleventy2.0.1": "npm:@11ty/eleventy@2.0.1",
-    "@types/node": "^22.10.1",
-    "ava": "^6.2.0",
-    "esbuild": "^0.24.0",
-    "jsdom": "^25.0.1",
+    "@11ty/eleventy3.1.2": "npm:@11ty/eleventy@3.1.2",
+    "@types/node": "^25.3.3",
+    "ava": "^7.0.0",
+    "esbuild": "^0.27.3",
+    "jsdom": "^28.1.0",
     "npm-run-all": "^4.1.5",
-    "typescript": "^5.7.2"
+    "typescript": "^5.9.3"
   },
   "dependencies": {}
 }