mocha-distributed 0.8.1 → 0.9.0

package/README.md

@@ -1,160 +1,254 @@
 # mocha-distributed
 
+Run mocha tests faster.
+
+Speed up your mocha tests by running them in parallel in multiple machines all
+at once without changing a single line of code. You only need a redis server.
+
+## Purpose
+
 The aim of this project is to provide a simple way of running distributed mocha
-tests without having to change too many lines of code, nor having to decide
-what to run where.
+tests without having to change any line of code, nor having to decide
+what to run where. Tests spread automatically according to the nodes you have.
+
+The concept is very simple, basically you spawn as many runners as you wish
+on as many nodes as you wish, and each node decides whether they should run
+a test or the test has already been executed or is being executed somewhere
+else.
 
 It does not matter if you run the tests in one machine as subprocesses or in
 many machines with multiple processes each.
 
-Hopefully you will only need to include a single line of code on each of your
-mocha files in order for them to run in parallel.
+Because you don't need to change a single line of code, which means that you
+can still run mocha locally as usual.
 
-Your test files will still be 100% compatible with mocha, so you can run them
-without any side-effects locally, using mocha, as if nothing was changed.
+## Quick start
 
-A idea for the future is to create a mocha-compatible runner so that you don't
-even have to change source files in any way. But let's go one step at a time.
+You don't need to change a single line of code on your tests, this project uses
+mocha hooks in order to work, so the only thing you'll need to do in preparation
+is:
 
-## How it works
+  ```bash
+  $ npm install -s mocha-distributed
+  ```
 
-### Brief example
+Make sure you have a redis running somewhere with IP visibility from the machine
+or machines where you want to run the tests on. 
 
-First, add the following line in each of your test files you want to distribute.
-If you try to run the tests without this line, the tests will run on ALL machines,
-including the master node.
+Finally, on each of the runners just run:
 
-  ```javascript
-  require('mocha-distributed');
+  ```bash
+    $ export MOCHA_DISTRIBUTED_EXECUTION_ID="execution__2021-01-01__20:10" 
+    $ export MOCHA_DISTRIBUTED="redis://redis.address" 
+    $ mocha --require mocha-distributed test/**/*.js
   ```
 
-Then, if you want to run mocha in one computer (e.g your dev computer):
+There are several environment variables that allow you to control the behaviour
+of distributed tests, but this is the simplest way to launch them.
 
-  ```bash
-  $ mocha test/**/*.js
-  ```
+MOCHA_DISTRIBUTED is the one holding the redis address, this is the only 
+requirement to make mocha-distributed work.
 
-To run mocha distributed:
+MOCHA_DISTRIBUTED_EXECUTION_ID is the other variable you want to pay attention
+to. Make sure you use a different value for each group of runners every time
+you launch a test. This variable is what makes possible to make a runner know
+whether a test has already been executed or not by other of their peers.
 
-  - Execute as master in only one machine (imagine it is running on 1.2.3.4 IP address):
+## Environment Variables
 
-    ```bash
-    $ MOCHA_DISTRIBUTED="master" mocha test/**/*.js
-    ```
+  - **MOCHA_DISTRIBUTED** (required)
 
-  - Execute runners on one or a thousand machines/processes as:
+    Right now this variable is the one used to specify the node that will hold
+    information about tests being run. This project only supports redis right
+    now. This variable can take the form:
+    
+      redis[s]://[[username][:password]@][host][:port]
 
-    ```bash
-    $ MOCHA_DISTRIBUTED="1.2.3.4" mocha test/**/*.js
-    ```
+    Please make sure it has visibility to the desired redis server.
 
+  - **MOCHA_DISTRIBUTED_EXECUTION_ID** (required)
 
+    Make sure this value is different every time you launch your tests. You can
+    use any string here, but it should be different across test executions or
+    your tests will just be skipped after the second execution.
 
-### Conceptual overview
+    Execution ID is used in order to differentiate different runs of the same
+    tests among parallel executions. If you launch 10 instances and you want
+    tests to be distributed among them, all need to have the same value for this
+    variable, otherwise each of them will run all the tests on its own.
 
-The concept is very simple, this module hooks all mocha calls and does some magic
-to allow running tests across machines without you having to decide what runs
-where, or splitting tests beforehand, etc...
+    Reusing this variable in different executions will cause your tests to be
+    skipped.
+
+    Use a random uuid or other random value, a kubernetes job_name, your 
+    build system job id, ...
+
+  - **MOCHA_DISTRIBUTED_GRANULARITY** = test
+  
+    - test (default)
+      Potentially all tests can be executed by any runner in any order. This
+      is the default, but if you have trouble running your tests in parallel
+      please use "suite" instead
+
+    - suite (safest)
+
+      Launch all tests from the same suite in the same runner. This prevents
+      some parallelization errors if your tests are not prepared for full
+      paralelization.
+
+  - **MOCHA_DISTRIBUTED_RUNNER_ID** = random-id
+
+    By default this value is initialized automatically with a different random
+    string in each machine, BUT you can override this in case you need it for
+    whatever reason, although in theory you probably shouldn't.
+
+  - **MOCHA_DISTRIBUTED_EXPIRATION_TIME** = 86400
+
+    Configures to how long the data is kept in redis before it expires (in 
+    seconds). The amount of data in redis is minimal, so you probably don't want
+    to play with it.
+
+    It might be helpful to increase it though, if you want to build some sort of
+    reporting on top of it, because you can directly explore test results in
+    redis. See Tests results in Redis for more info.
 
-To distribute tests you only need to create several processess across one
-or more machines (system won't care), and set one of them as the master,
-and the rest as the runners. Only one master allowed.
 
-Runners connect to the master and for each suite they ask whether they are
-the 'owners' to run the tests on that suite or not. If they are, they run it.
-If they are not, they just skip the tests and continue running the next suite.
+  - **MOCHA_DISTRIBUTED_VERBOSE** = false
+    - false (default)
+      Avoid printing verbose information
 
-For simplicity this is all granularity you'll get for now. If you need two
-suites to run one after another on the same machine, then create a suite
-that encloses those.
+    - true
+      Prints some extra information about the variables, the server, ...
+      that might be useful for debugging issues and/or informational.
 
-All test results (with error information) are sent to the master, and the master
-will display the output of all tests.
+## Reading test results from Redis
 
-If you like to save test results, etc... run the master with the right mocha
-parameters. Using those parameters on the runners won't hurt either.
+All runners write the test result in JSON format in a specific redis list.
 
-### How to run in practice
+The list is basically the execution ID from the variable 
+MOCHA_DISTRIBUTED_EXECUTION_ID concatenated to ':test_result'
 
-There is a magic environment variable called MOCHA_DISTRIBUTED.
+For example, if you are using: MOCHA_DISTRIBUTED_EXECUTION_ID="abcdefg"
 
-When unset or empty, this module does nothing at all. Mocha runs normally,
-as if you would have not installed this module.
+Then the key you should look at in redis will be "abcdefg:test_result"
 
-When set to the special keyword 'master', the mocha will automatically create
-an HTTP server and listen to other processes or machines to connect to it and
-ask/inform about running the tests.
+You can access this list and explore the result of all tests. Each item
+on the list will contain information about the test suite, test id, ...
+test name, if it timed out or not, duration of the test, result of the test,
+if there were any errors, ... all that info is extracted from mocha itself.
 
-For the runners, you would need to set MOCHA_DISTRIBUTED variable with the IP
-of the master computer that is running the test. If you are running all the
-tests in multiple processes you can set it to 127.0.0.1, otherwise it should
-be the IP of that machine in your private network, or the public IP if the
-machines are distributed around the world.
+You will see something like this on each of the items of the list:
 
-You can also append the port to both the master and the runners, but in
-that case, the port must match.
+  ```json
+  {
+    "id": [
+      "suite-1-async",
+      "test-1.1-async"
+    ],
+    "type": "test",
+    "title": "test-1.1-async",
+    "timedOut": false,
+    "duration": 502,
+    "file": "/home/psanchez/github/mocha-distributed/example/suite-1.js",
+    "state": "passed",
+    "speed": "slow",
+    "err": 0
+  }```
+
+The JSON formatting will differ since it is saved in a single line.
+
+Apart of test_result you can also access passed_count and failed_count
 
 ## Examples
 
-### Run tests in one machine and one process
+### Environment-agnostic
 
-Just don't use the MOCHA_DISTRIBUTED variable, or set it to empty string.
+Make sure at least the following variables are set:
 
   ```bash
-  $ mocha test/**/*.js
+  MOCHA_DISTRIBUTED="redis://1.2.3.4"
+  MOCHA_DISTRIBUTED_EXECUTION_ID="a5ce4d8a-5b06-4ec8-aea2-37d7e4b2ffe1"
   ```
 
-### Run tests in one machine, multiple processes
+Again, execution ID should be a different random number each time you want to
+launch tests in parallel.
 
-To keep things simple, do something like this:
+Example:
 
   ```bash
-  $ MOCHA_DISTRIBUTED="master" mocha test/**/*.js
-  $ MOCHA_DISTRIBUTED="localhost" mocha test/**/*.js > /dev/null &
-  $ MOCHA_DISTRIBUTED="localhost" mocha test/**/*.js > /dev/null &
-  ...
-  $ MOCHA_DISTRIBUTED="localhost" mocha test/**/*.js > /dev/null &
+  $ mocha --require mocha-distributed test/**/*.js
   ```
 
-Run as many processes as you'd like
+Of course, this assumes you have already installed mocha-distributed.
 
-### Run tests in several processes across several machines
+### Run tests in parallel in the same machine
 
-On one machine do:
+To keep things simple, do something like this:
 
   ```bash
-  $ MOCHA_DISTRIBUTED="master" mocha test/**/*.js
+  $ MOCHA_DISTRIBUTED_EXECUTION_ID=`uuidgen`
+  $ MOCHA_DISTRIBUTED="redis://redis-server"
+
+  $ mocha --require mocha-distributed test/**/*.js > output01.txt &
+  $ mocha --require mocha-distributed test/**/*.js > output02.txt &
+  ...
+  $ mocha --require mocha-distributed test/**/*.js > output0N.txt &
   ```
 
-You can also run some runners in that machine if you wish (see previous example).
+Run as many processes as you'd like.
 
-Figure out the IP address of the master. For this example let's say the master
-IP address is 1.2.3.4. Now on the rest of machines, just do:
+### Using kubernetes parallel jobs to launch tests
 
-  ```bash
-  $ MOCHA_DISTRIBUTED="1.2.3.4" mocha test/**/*.js > /dev/null &
-  $ MOCHA_DISTRIBUTED="1.2.3.4" mocha test/**/*.js > /dev/null &
-  ...
-  $ MOCHA_DISTRIBUTED="1.2.3.4" mocha test/**/*.js > /dev/null &
-  ```
+If you plan to use kubernetes to launch parallel jobs, make sure the backoff 
+limit is set to 1, so it does not retry the job after it fails, and make sure
+you set execution ID to a different value each time (but common across all
+parallel executions).
+
+The easiest is to use the job ID (not the pod ID). You can do that by exposing
+pod metadata information as environment variables.
+
+See https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/
+
+### Conceptual overview
+
+The concept is very simple, this module hooks all mocha calls and does some magic
+to allow running tests across machines without you having to decide what runs
+where, or splitting tests beforehand, etc...
+
+To distribute tests you only need to create several processess across one
+or more machines (this method won't care how you spawn your runners), and either
+set one of them as the master or use a redis database, and launch as many runners
+as you wish.
+
+Each runners connects to the redis instance and for each suite or test,
+depending on the granularity, they ask whether they are the 'owners' to run the
+tests on that suite or not. If they are, they run it. If they are not, they just
+skip the tests and continue running the next suite/tests.
+
+### Caveats
+
+When running with redis, all tests are executed by independent runners, which
+means you need to take a look at the output of all the runners and see which
+ones were skipped and which ones were executed for you to see if some of those
+executed failed.
 
-Again, spawn as many processes as you'd like.
+Also the exit code of the different mocha runners will differ. The
+ones whose tests fail, will return an error, and the ones whose tests work well
+or have been skipped will return 0.
 
 ## Build systems
 
 ### jenkins, bamboo, circle-ci, gitlab, travis...
 
-If you use jenkins, bamboo or any other build system, only one runner should
-be defined as the master. The master never runs tests, only waits.
+If you use jenkins, bamboo or any other build system, make sure
+one redis is installed somewhere and all runners can access to it.
 
-You should create more processes or launch more docker or kubernetes instances
-or spread test on several nodes... do it as you wish, but for each of the
-runners that you create, make sure they have visibility to the master (e.g
-make sure you can send a ping from all the runners to the master).
+You can create as many processes processes or launch as docker or kubernetes
+instances as you wish, but for each of the runners that you create, make sure
+they have visibility to the redis.
 
-In case you run multiple masters on the same machine, make sure you setup
-a different port each time, otherwise runner's from different projects will
-inform the wrong master.
+You can use the project name and build ID or job id as the execution ID for
+mocha-distributed. Use something unique among the builds of all your projects.
 
 ## MIT License
 

package/docker-compose.yml

@@ -0,0 +1,18 @@
+version: "3.7"
+
+services:
+
+  redis:
+    image: redis:latest
+    ports:
+      - 6379:6379
+
+  redis-commander:
+    image: rediscommander/redis-commander:latest
+    environment:
+      - REDIS_HOSTS=local:redis:6379
+    ports:
+      - 8081:8081
+    depends_on:
+      - redis
+

package/index.js

@@ -1,49 +1,157 @@
+const redis = require("redis");
+const crypto = require("crypto");
+
+const GRANULARITY = {
+  TEST: "test",
+  SUITE: "suite",
+};
+
+// Initialize variables from environment
+const g_redisAddress = process.env.MOCHA_DISTRIBUTED || "";
+const g_testExecutionId = process.env.MOCHA_DISTRIBUTED_EXECUTION_ID || "";
+const g_expirationTime =
+  process.env.MOCHA_DISTRIBUTED_EXPIRATION_TIME || `${24 * 3600}`;
+
+// Generate a unique random id for this runner (with almost 100% certainty
+// to be different on any machine/environment).
+const _randomRunnerBuf = Buffer.alloc(16);
+const _randomRunnerId = crypto.randomFillSync(_randomRunnerBuf).toString("hex");
+const g_runnerId = process.env.MOCHA_DISTRIBUTED_RUNNER_ID || _randomRunnerId;
+let g_granularity =
+  process.env.MOCHA_DISTRIBUTED_GRANULARITY || GRANULARITY.TEST;
+const g_mochaVerbose = process.env.MOCHA_DISTRIBUTED_VERBOSE === "true";
+
+if (g_granularity !== GRANULARITY.TEST) {
+  g_granularity = GRANULARITY.SUITE;
+}
+
+let g_redis = null;
+
 // -----------------------------------------------------------------------------
-// index.js
+// getTestPath
 //
-// Copyright(c) 2019 Pau Sanchez - MIT License
-// -----------------------------------------------------------------------------
-
-// Three scenarios for MOCHA_DISTRIBUTED:
-//   - master (MOCHA_DISTRIBUTED = 'master[:PORT]')
-//     Runs the server as master node, waiting for connections and waiting
-//     others to run the tests. For simplicity, this node won't execute
-//     any tests, but gather the result of all of them.
+// Returns an array with the test suites and test name from a test context
+// as found in the hooks
 //
-//   - runner (MOCHA_DISTRIBUTED='<master-address>[:PORT]'  IP/DNS of the master node)
-//     IP/DNS of the master node (a.k.a runner), this node will get a unique
-//     name from the master, and will ask, before each suite or orphaned test
-//     if it needs to run it or not
+// Example:
 //
-//   - Empty or not defined
-//     Runs mocha normally, we don't redefine any variable at all.
-//     Everything will run locally, if spawned on many machines or processes
-//     tests will run on all of them as if this module did not exist.
-const master = require('./master.js');
-const runner = require('./runner.js');
-
-// Initialize mode & port from environment variable MOCHA_DISTRIBUTED
-const DEFAULT_PORT = 12421;
-let mode = (process.env.MOCHA_DISTRIBUTED || '').toLowerCase();
-let port = DEFAULT_PORT;
-
-if (mode.indexOf (':') >= 0) {
-  const splitted = mode.split(':');
-  mode = splitted[0];
-  port = parseInt (splitted[1], 10);
-}
+//    >>> getTestPath(ctxt)
+//
+// -----------------------------------------------------------------------------
+function getTestPath(testContext) {
+  const path = [testContext.title];
 
-// let's get the party started
-if (!mode) {
-  // run as normal mocha
-}
-else if (mode === 'master') {
-  master (port);
-}
-else {
-  const masterAddress = mode;
-  runner (masterAddress, port);
+  while (!testContext.root && testContext.parent) {
+    testContext = testContext.parent;
+
+    if (testContext && !testContext.root) {
+      path.push(testContext.title);
+    }
+  }
+
+  return path.reverse();
 }
 
-// remove from cache, so it is always reinitialized
-delete require.cache[require.resolve('./index.js')];
+// -----------------------------------------------------------------------------
+// Initialize redis once before the tests
+// -----------------------------------------------------------------------------
+exports.mochaGlobalSetup = async function () {
+  if (g_mochaVerbose) {
+    const redisNoCredentials = g_redisAddress.replace(
+      /\/\/[^@]*@/,
+      "//***:***@"
+    );
+    console.log("---------------------------------------------------");
+    console.log(" Mocha Distributed");
+    console.log("   - Runner Id                :", g_runnerId);
+    console.log("   - Redis Address            :", redisNoCredentials);
+    console.log("   - Execution Id             :", g_testExecutionId);
+    console.log("   - Data Expiration Time     :", g_expirationTime);
+    console.log("   - Test Parallel Granularity:", g_granularity);
+    console.log("---------------------------------------------------");
+  }
+
+  if (!g_redisAddress || !g_testExecutionId) {
+    console.log (g_redisAddress, g_testExecutionId)
+    console.error(
+      "You need to set at least the following environment variables:\n" +
+        "  - MOCHA_DISTRIBUTED\n" +
+        "  - MOCHA_DISTRIBUTED_EXECUTION_ID\n"
+    );
+    process.exit(-1);
+  }
+
+  g_redis = redis.createClient({ url: g_redisAddress });
+  g_redis.on("error", (err) => {
+    console.log("Redis Client Error", err);
+    console.log("Closing application!");
+    process.exit(-1);
+  });
+  await g_redis.connect();
+};
+
+// -----------------------------------------------------------------------------
+// Quit from redis
+// -----------------------------------------------------------------------------
+exports.mochaGlobalTeardown = async function () {
+  if (g_redis) {
+    await g_redis.quit();
+  }
+};
+
+// -----------------------------------------------------------------------------
+// Hook tests
+//
+// Please note that we run skip before each test if the ownership of it has
+// already been defined by another runner.
+// -----------------------------------------------------------------------------
+exports.mochaHooks = {
+  beforeEach: async function () {
+    const testPath = getTestPath(this.currentTest);
+    const testKeyFullPath = `${g_testExecutionId}:${testPath.join(":")}`;
+    const testKeySuite = `${g_testExecutionId}:${testPath[0]}`;
+
+    const testKey =
+      g_granularity === GRANULARITY.TEST ? testKeyFullPath : testKeySuite;
+
+    // Atomically set/get the runner id associated to this test. Only the first
+    // runner to get there will set the value to its own runner id.
+    const [_, assignedRunnerId] = await g_redis
+      .multi()
+      .set(testKey, g_runnerId, { EX: g_expirationTime, NX: true })
+      .get(testKey)
+      .exec();
+
+    if (assignedRunnerId !== g_runnerId) {
+      this.currentTest.title += " (skipped by mocha_distributted)";
+      this.skip();
+    }
+  },
+  afterEach(done) {
+    const SKIPPED = "pending";
+
+    // Save all data in redis in a way it can be retrieved and aggregated
+    // easily for all test by an external reporter
+    if (this.currentTest.state !== SKIPPED) {
+      const testResult = {
+        id: getTestPath(this.currentTest),
+        type: this.currentTest.type,
+        title: this.currentTest.title,
+        timedOut: this.currentTest.timedOut,
+        duration: this.currentTest.duration,
+        file: this.currentTest.file,
+        state: this.currentTest.state,
+        speed: this.currentTest.speed,
+        err: this.currentTest.err | null,
+      };
+
+      // save as single line on purpose
+      const key = `${g_testExecutionId}:test_result`;
+      g_redis.rPush(key, JSON.stringify(testResult));
+      g_redis.expire(key, g_expirationTime);
+      g_redis.incr(`${g_testExecutionId}:${this.currentTest.state}_count`);
+    }
+
+    done();
+  },
+};

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mocha-distributed",
-  "version": "0.8.1",
+  "version": "0.9.0",
   "description": "Run multiple mocha suites and tests in parallel, from different processes or different machines",
   "main": "index.js",
   "scripts": {
@@ -27,10 +27,10 @@
   "author": "Pau Sanchez",
   "license": "MIT",
   "dependencies": {
-    "axios": "^0.18.0"
+    "redis": "^4.0.1"
   },
   "devDependencies": {
     "chai": "^4.2.0",
-    "mocha": "^5.2.0"
+    "mocha": "^9.1.3"
   }
 }

package/constants.js

@@ -1,33 +0,0 @@
-// -----------------------------------------------------------------------------
-// constants.js
-//
-// Copyright(c) 2019 Pau Sanchez - MIT License
-// -----------------------------------------------------------------------------
-
-const RUNNER_ID_PREFIX = 'runner-';
-
-const TEST_STATUS_RUNNING = 'running';
-const TEST_STATUS_SUCCESS = 'success';
-const TEST_STATUS_FAILED  = 'failed';
-
-const TEST_PATH_SEPARATOR = '>>>';
-
-const EVENT_FINISHED = 'finished';
-
-const ERRORS = {
-  INVALID_REQUEST : 'INVALID_REQUEST',
-  INVALID_TEST_ID : 'INVALID_TEST_ID',
-  INVALID_REQUEST_ACTION : 'INVALID_REQUEST_ACTION',
-  INVALID_RUNNER_OWNERSHIP : 'INVALID_RUNNER_OWNERSHIP',
-  ALREADY_RUNNING : 'ALREADY_RUNNING',
-};
-
-module.exports = {
-  RUNNER_ID_PREFIX,
-  TEST_STATUS_RUNNING,
-  TEST_STATUS_SUCCESS,
-  TEST_STATUS_FAILED,
-  TEST_PATH_SEPARATOR,
-  EVENT_FINISHED,
-  ERRORS
-};

package/master-mocha-bindings.js

@@ -1,159 +0,0 @@
-// -----------------------------------------------------------------------------
-// runner-mocha-bindings.js
-//
-// NOTE: mocha first does a first pass executing all describe/it/... and then
-//       when it has gathered all information, it runs the functions associated
-//       to each. What we do is we hook our method in the middle and call
-//       or not, the original test method.
-//
-// Copyright(c) 2019 Pau Sanchez - MIT License
-// -----------------------------------------------------------------------------
-const constants = require('./constants.js');
-
-let g_mochaMethods = {
-  describe   : global.describe || null,
-  it         : global.it || null,
-  before     : global.before || null,
-  beforeEach : global.beforeEach || null,
-  after      : global.after || null,
-  afterEach  : global.afterEach || null
-};
-
-// contains the actual test path such as suite.title > suite.title > it.title
-let g_testPath = [];
-let g_testResults = new Map();
-let g_testEventEmitter = null;
-
-// -----------------------------------------------------------------------------
-// setEventEmitter
-//
-// To communicate with the mocha bindings
-// -----------------------------------------------------------------------------
-function setEventEmitter (testEventEmitter) {
-  g_testEventEmitter = testEventEmitter;
-}
-
-// -----------------------------------------------------------------------------
-// describe
-//
-// Custom version to describe a test, with same signature
-// -----------------------------------------------------------------------------
-async function describe (title, fn) {
-  g_testPath.push (title);
-
-  // TODO: handle timeouts!!
-  const result = g_mochaMethods.describe (title, fn);
-  g_testPath.pop();
-
-  return result;
-}
-
-// TODO:
-// describe.skip = async function (title, fn) { }
-// describe.once = async function (title, fn) { }
-
-// -----------------------------------------------------------------------------
-// it
-// -----------------------------------------------------------------------------
-async function it (title, fn) {
-  // store current path for when the 'it' function executes
-  g_testPath.push (title);
-  const testPath = g_testPath.slice(0);
-  const testId   = testPath.join(constants.TEST_PATH_SEPARATOR);
-  const listenerId = constants.EVENT_FINISHED + ':' + testId;
-  g_testPath.pop();
-
-  // TODO: manage timeouts
-
-  // Two listeners are created, once beforehand, and other in runtime.
-  //
-  // Runtime one will wait the master to receive the status... but since the
-  // master node runs serially, it can also happen that another runner executes
-  // the tests before the master can initialize the second event listener, and
-  // we use this one to capture the result beforehand.
-  g_testEventEmitter.once (
-    listenerId,
-    function (test) {
-      g_testResults.set (test.id, test);
-    }
-  );
-
-  // define our own function hook to wait until the test finishes
-  return g_mochaMethods.it (title, async function () {
-    await new Promise(function (resolve, reject) {
-      // helper function to avoid repeating the code twice
-      function helperProcessTestResult(test) {
-        if (test.status === constants.TEST_STATUS_FAILED) {
-          reject (test.error);
-          return;
-        }
-
-        resolve();
-      }
-
-      if (g_testResults.has(testId)) {
-        const test = g_testResults.get (testId);
-        helperProcessTestResult (test);
-        return;
-      }
-
-      // first event-listener has not triggered, so let's install another
-      // event listener to resolve as quickly as possible
-      g_testEventEmitter.once (listenerId, helperProcessTestResult);
-    });
-  });
-}
-
-// -----------------------------------------------------------------------------
-// execHookMochaMethod
-//
-// Method used accross all mocha hooks (before, beforeEach, after, afterEach)
-// in order to query the master server if it needs to be executed or not.
-// -----------------------------------------------------------------------------
-function execHookMochaMethod (title, orgMochaMethod, fn) {
-  g_testPath.push (title);
-  const testPath = g_testPath.slice(0);
-  g_testPath.pop();
-
-  return orgMochaMethod (async function () {
-    // TODO: wait
-  });
-}
-
-// -----------------------------------------------------------------------------
-// before
-// -----------------------------------------------------------------------------
-async function before (fn) {
-  return execHookMochaMethod (':before', g_mochaMethods.before, fn);
-}
-
-// -----------------------------------------------------------------------------
-// beforeEach
-// -----------------------------------------------------------------------------
-async function beforeEach (fn) {
-  return execHookMochaMethod (':beforeEach', g_mochaMethods.beforeEach, fn);
-}
-
-// -----------------------------------------------------------------------------
-// after
-// -----------------------------------------------------------------------------
-async function after (fn) {
-  return execHookMochaMethod (':after', g_mochaMethods.after, fn);
-}
-
-// -----------------------------------------------------------------------------
-// afterEach
-// -----------------------------------------------------------------------------
-async function afterEach (fn) {
-  return execHookMochaMethod (':afterEach', g_mochaMethods.after, fn);
-}
-
-module.exports = {
-  setEventEmitter,
-  describe,
-  it,
-  before,
-  beforeEach,
-  after,
-  afterEach
-};

package/master-server.js

@@ -1,194 +0,0 @@
-// -----------------------------------------------------------------------------
-// master-mocha-bindings.js
-//
-// Copyright(c) 2019 Pau Sanchez - MIT License
-// -----------------------------------------------------------------------------
-const url = require('url');
-const querystring = require('querystring');
-const constants = require('./constants.js');
-
-let g_testsPerRunner = new Map();
-let g_testEventEmitter = null;
-
-// -----------------------------------------------------------------------------
-// setEventEmitter
-//
-// To communicate with the mocha bindings
-// -----------------------------------------------------------------------------
-function setEventEmitter (testEventEmitter) {
-  g_testEventEmitter = testEventEmitter;
-}
-
-// -----------------------------------------------------------------------------
-// handleRunnerShouldRun
-//
-// REST: /runner/<runner-id>/should-run?test=xxxxx
-//
-// Asks whether the client should run given test
-// -----------------------------------------------------------------------------
-function handleRunnerShouldRun (req, res) {
-  const queryObj = querystring.parse(req.route.query);
-  const splitted = req.route.pathname.split('/');
-  const runnerId = splitted[2];
-  const action = splitted[3];
-
-  const testId = queryObj.test || null;
-
-  // is this test already assigned to somebody else?
-  if (!testId) {
-    res.write (
-      JSON.stringify ({
-        answer : 'skip',
-        reason : constants.ERRORS.INVALID_TEST_ID
-      })
-    )
-    return;
-  }
-
-  if (g_testsPerRunner.has(testId)) {
-    const runner = g_testsPerRunner.get (testId).runner;
-
-    // is it a retry? e.g. same runner...
-    if (runnerId === runner) {
-      g_testsPerRunner.get (testId).retries ++;
-      res.write (JSON.stringify ({ answer : 'run' }));
-      return;
-    }
-
-    // otherwise, it is already running :)
-    res.write (
-      JSON.stringify ({
-        answer : 'skip',
-        reason : constants.ERRORS.ALREADY_RUNNING,
-        runner : runner
-      })
-    );
-    return;
-  }
-
-  // make this runner the owner of running this test
-  g_testsPerRunner.set (testId, {
-    id     : testId,
-    runner : runnerId,
-    status : constants.TEST_STATUS_RUNNING,
-    retries: 0,
-    start  : Date.now(),
-    end    : false,
-    error  : null
-  });
-
-  res.write (
-    JSON.stringify ({
-      answer : 'run'
-    })
-  );
-}
-
-// -----------------------------------------------------------------------------
-// handleRunnerResult
-//
-// REST: /runner/<runner-id>/result?test=xxxxx&status=success|error
-//
-// Informs the result of running given test
-// -----------------------------------------------------------------------------
-function handleRunnerResult (req, res) {
-  const queryObj = querystring.parse(req.route.query);
-  const splitted = req.route.pathname.split('/');
-  const runnerId = splitted[2];
-  const action = splitted[3];
-
-  const testId = queryObj.test || null;
-  if (!testId || !g_testsPerRunner.has(testId)) {
-    res.write (
-      JSON.stringify ({
-        error : constants.ERRORS.INVALID_TEST_ID
-      })
-    )
-    return;
-  }
-
-  const test = g_testsPerRunner.get (testId);
-  if (test.runner !== runnerId) {
-    res.write (
-      JSON.stringify ({
-        error : constants.ERRORS.INVALID_RUNNER_OWNERSHIP
-      })
-    )
-    return;
-  }
-
-  let status = queryObj.status || constants.TEST_STATUS_FAILED;
-  if (status !== constants.TEST_STATUS_SUCCESS) {
-    status = constants.TEST_STATUS_FAILED;
-  }
-
-  test.status = status;
-  test.error  = JSON.parse(querystring.unescape (queryObj.error || 'null'));
-
-  res.write ( JSON.stringify ({ 'status' : status }) );
-
-  // Emit an event-finished for given test ID, we use the ID because that
-  // should be unique.
-  g_testEventEmitter.emit (constants.EVENT_FINISHED + ':' + testId, test);
-}
-
-// -----------------------------------------------------------------------------
-// handleRunners
-//
-// REST: /runner/<runner-id>/*
-//
-// Handles all requests from all runners (registration, queries, ...)
-// -----------------------------------------------------------------------------
-function handleRunners (req, res) {
-  const queryObj = querystring.parse(req.route.query);
-  const splitted = req.route.pathname.split('/');
-  const runnerId = splitted[2];
-  const action   = splitted[3];
-
-  // TODO: check that runner is registered
-  if (!runnerId || !action) {
-    res.end();
-    return;
-  }
-
-  switch (action) {
-    case 'should-run': handleRunnerShouldRun (req, res); break;
-    case 'result':     handleRunnerResult(req, res); break;
-    default:
-      console.error ("Invalid action: " + action + "(" + req.url + ")");
-      res.write (
-        JSON.stringify ({ error : constants.ERRORS.INVALID_REQUEST_ACTION })
-      )
-      break;
-  }
-  res.end();
-}
-
-// -----------------------------------------------------------------------------
-// mainServerHandler
-//
-// REST: /*
-//
-// Handles all requests to this master server
-// -----------------------------------------------------------------------------
-function mainServerHandler (req, res) {
-  res.writeHead(200, {'Content-Type': 'application/javascript'});
-  req.route = url.parse(req.url);
-
-  // REST: /runner/<runner-id>/*
-  //
-  // execute actions from given runner
-  if (req.route.pathname.startsWith (`/runner/`)) {
-    handleRunners (req, res);
-  }
-  else {
-    res.write(JSON.stringify ({ error : constants.ERRORS.INVALID_REQUEST }));
-    res.end();
-  }
-}
-
-
-module.exports = {
-  setEventEmitter,
-  mainServerHandler
-};

package/master.js

@@ -1,47 +0,0 @@
-// -----------------------------------------------------------------------------
-// master.js
-//
-// Copyright(c) 2019 Pau Sanchez - MIT License
-// -----------------------------------------------------------------------------
-const http = require('http');
-const EventEmitter = require('events');
-const masterServer = require ('./master-server.js');
-const masterMochaBindings = require ('./master-mocha-bindings.js');
-
-// used to notify master that a test has been executed
-class TestEmitter extends EventEmitter {};
-
-let g_server = null;
-let g_eventEmitter = null;
-
-// -----------------------------------------------------------------------------
-// master
-//
-// Initializes the master server, redefines variables, etc...
-// -----------------------------------------------------------------------------
-function master (port) {
-
-  // Checking server because this master function might be called once
-  // per every mocha file, and we only want to initialize once.
-  if (g_server === null) {
-    g_eventEmitter = new TestEmitter();
-
-    masterServer.setEventEmitter(g_eventEmitter);
-    masterMochaBindings.setEventEmitter (g_eventEmitter);
-
-    g_server = http.createServer(masterServer.mainServerHandler);
-    g_server.listen(port, function() {
-      console.log("# Mocha distributed master listening at port:", port, "\n");
-    });
-  }
-
-  // hook all mocha methods
-  global.describe   = masterMochaBindings.describe;
-  global.it         = masterMochaBindings.it;
-  global.before     = masterMochaBindings.before;
-  global.beforeEach = masterMochaBindings.beforeEach;
-  global.after      = masterMochaBindings.after;
-  global.afterEach  = masterMochaBindings.afterEach;
-}
-
-module.exports = master;

package/runner-mocha-bindings.js

@@ -1,212 +0,0 @@
-// -----------------------------------------------------------------------------
-// runner-mocha-bindings.js
-//
-// NOTE: mocha first does a first pass executing all describe/it/... and then
-//       when it has gathered all information, it runs the functions associated
-//       to each. What we do is we hook our method in the middle and call
-//       or not, the original test method.
-//
-// Copyright(c) 2019 Pau Sanchez - MIT License
-// -----------------------------------------------------------------------------
-const axios = require('axios');
-const querystring = require('querystring');
-const crypto = require('crypto');
-
-const constants = require('./constants.js');
-
-let g_masterAddress = null;
-
-// Generate a unique random id for this runner (with almost 100% certainty
-// to be different on any machine/environment).
-const g_runnerIdBuffer = Buffer.alloc(16);
-const g_runnerId = crypto.randomFillSync(g_runnerIdBuffer).toString('hex');
-
-let g_mochaMethods = {
-  describe   : global.describe || null,
-  it         : global.it || null,
-  before     : global.before || null,
-  beforeEach : global.beforeEach || null,
-  after      : global.after || null,
-  afterEach  : global.afterEach || null
-};
-
-
-// contains the actual test path such as suite.title > suite.title > it.title
-let g_testPath = [];
-
-// -----------------------------------------------------------------------------
-// askMasterIfShouldRun
-//
-// Asks master server if the current runner should run given suite.
-//
-// Right now we only ask for suites because it seems the safest approach.
-//
-// Returns:
-//  - true: the caller should run the test
-//  - false: the caller should not run the test
-//  - null: connectivity issue with the master, it depends on the caller to
-//    decide what to do
-// -----------------------------------------------------------------------------
-async function askMasterIfShouldRun (testPath) {
-  const testSuite = querystring.escape(testPath[0]);
-  try {
-    const r = await axios.get (`http://${g_masterAddress}/runner/${g_runnerId}/should-run?test=${testSuite}`);
-    const ok = (r.data.answer === 'run');
-
-    // we ask for given test, so we can save the result properly
-    // TODO: this can be removed if the server accepts runners
-    if (ok) {
-      const path = querystring.escape(testPath.join(constants.TEST_PATH_SEPARATOR));
-      const r = await axios.get (`http://${g_masterAddress}/runner/${g_runnerId}/should-run?test=${path}`);
-    }
-    return ok;
-  }
-  catch (e) {
-    // ignore connection errors
-    // the master could have died because it already finished
-  }
-
-  // skip (should not run) if there is any kind of error with master
-  return null;
-}
-
-// -----------------------------------------------------------------------------
-// sendTestResultToMaster
-// -----------------------------------------------------------------------------
-async function sendTestResultToMaster (testPath, status, error = null) {
-  // TODO: use POST
-  const path = querystring.escape(testPath.join(constants.TEST_PATH_SEPARATOR));
-  let serror = querystring.escape(JSON.stringify(error || null));
-
-  try {
-    const r = await axios.get (
-      `http://${g_masterAddress}/runner/${g_runnerId}/result?test=${path}&status=${status}&error=${serror}`
-    );
-  }
-  catch (e) {
-    // ignore connection errors
-    // the master could have died because it already finished
-  }
-}
-
-// -----------------------------------------------------------------------------
-// init
-//
-// Initialize this module
-// -----------------------------------------------------------------------------
-function init (masterAddress){
-  g_masterAddress = masterAddress;
-}
-
-// -----------------------------------------------------------------------------
-// describe
-//
-// Custom version to describe a test, with same signature
-// -----------------------------------------------------------------------------
-async function describe (title, fn) {
-  g_testPath.push (title);
-  const result = g_mochaMethods.describe (title, fn);
-  g_testPath.pop();
-
-  return result;
-}
-
-// TODO:
-// describe.skip = async function (title, fn) { }
-// describe.once = async function (title, fn) { }
-
-
-// -----------------------------------------------------------------------------
-// it
-// -----------------------------------------------------------------------------
-async function it (title, fn) {
-  // store current path for when the 'it' function executes
-  g_testPath.push (title);
-  const testPath = g_testPath.slice(0);
-  g_testPath.pop();
-
-  // define our own function hook
-  return g_mochaMethods.it (title, async function () {
-    if (!await askMasterIfShouldRun (testPath)) {
-      this.skip();
-      return;
-    }
-
-    try {
-      const testResult = await fn();
-      await sendTestResultToMaster (testPath, constants.TEST_STATUS_SUCCESS);
-      return testResult;
-    }
-    catch (e) {
-      await sendTestResultToMaster (testPath, constants.TEST_STATUS_FAILED, e);
-      throw e;
-    }
-  });
-}
-
-// -----------------------------------------------------------------------------
-// execHookMochaMethod
-//
-// Method used accross all mocha hooks (before, beforeEach, after, afterEach)
-// in order to query the master server if it needs to be executed or not.
-// -----------------------------------------------------------------------------
-function execHookMochaMethod (title, orgMochaMethod, fn) {
-  g_testPath.push (title);
-  const testPath = g_testPath.slice(0);
-  g_testPath.pop();
-
-  return orgMochaMethod (async function () {
-    if (!await askMasterIfShouldRun (testPath)) {
-      this.skip();
-      return;
-    }
-
-    try {
-      const testResult = await fn();
-      await sendTestResultToMaster (testPath, constants.TEST_STATUS_SUCCESS);
-      return testResult;
-    }
-    catch (e) {
-      await sendTestResultToMaster (testPath, constants.TEST_STATUS_FAILED, e);
-      throw e;
-    }
-  });
-}
-
-// -----------------------------------------------------------------------------
-// before
-// -----------------------------------------------------------------------------
-async function before (fn) {
-  return execHookMochaMethod (':before', g_mochaMethods.before, fn);
-}
-
-// -----------------------------------------------------------------------------
-// beforeEach
-// -----------------------------------------------------------------------------
-async function beforeEach (fn) {
-  return execHookMochaMethod (':beforeEach', g_mochaMethods.beforeEach, fn);
-}
-
-// -----------------------------------------------------------------------------
-// after
-// -----------------------------------------------------------------------------
-async function after (fn) {
-  return execHookMochaMethod (':after', g_mochaMethods.after, fn);
-}
-
-// -----------------------------------------------------------------------------
-// afterEach
-// -----------------------------------------------------------------------------
-async function afterEach (fn) {
-  return execHookMochaMethod (':afterEach', g_mochaMethods.after, fn);
-}
-
-module.exports = {
-  init,
-  describe,
-  it,
-  before,
-  beforeEach,
-  after,
-  afterEach
-};

package/runner.js

@@ -1,20 +0,0 @@
-// -----------------------------------------------------------------------------
-// runner.js
-//
-// Copyright(c) 2019 Pau Sanchez - MIT License
-// -----------------------------------------------------------------------------
-const runnerMochaBindings = require ('./runner-mocha-bindings.js');
-
-function runner (masterAddress, port) {
-  runnerMochaBindings.init (masterAddress + ':' + port);
-
-  // hook all mocha methods
-  global.describe   = runnerMochaBindings.describe;
-  global.it         = runnerMochaBindings.it;
-  global.before     = runnerMochaBindings.before;
-  global.beforeEach = runnerMochaBindings.beforeEach;
-  global.after      = runnerMochaBindings.after;
-  global.afterEach  = runnerMochaBindings.afterEach;
-}
-
-module.exports = runner;