testem 3.19.0 → 3.20.0

package/README.md

@@ -3,17 +3,19 @@ Got Scripts? Test’em!
 
 [![Build Status](https://github.com/testem/testem/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/testem/testem/actions/workflows/ci.yml?query=branch%3Amaster) [![npm version](https://badge.fury.io/js/testem.svg)](http://badge.fury.io/js/testem)
 
+Testem is a **JavaScript test runner** that runs your tests in **real desktop browsers**—Chrome, Firefox, Safari, Edge, and others you launch—so your specs execute in the same browser engines and DOM your users get, not a pretend environment. It also runs tests in **[Node](http://nodejs.org/)**, [PhantomJS](http://phantomjs.org/), or any launcher you configure. It is **framework-agnostic** and aimed at **any kind of tests** you want to run: unit, integration, end-to-end style suites, or custom setups—you pick the style; Testem wires it to the browser or process.
+
 Unit testing in Javascript can be tedious and painful, but Testem makes it so easy that you will actually *want* to write tests.
 
 Features
 --------
 
-* Test-framework agnostic. Support for
+* Test-framework agnostic — designed for **any kind of tests** that fit your project (unit, integration, custom runners, etc.), not a single prescribed style. Support for
     - [Jasmine](http://jasmine.github.io/)
     - [QUnit](http://qunitjs.com/)
     - [Mocha](http://mochajs.org/)
     - Others, through custom test framework adapters.
-* Run tests in all major browsers as well as [Node](http://nodejs.org) and [PhantomJS](http://phantomjs.org/)
+* Run tests in **all** major **real** browsers (your tests load and run in the actual browser) as well as [Node](http://nodejs.org) and [PhantomJS](http://phantomjs.org/)
 * Two distinct use-cases:
     - Test-Driven-Development(TDD) — designed to streamline the TDD workflow
     - Continuous Integration(CI) — designed to work well with popular CI servers like Jenkins or Teamcity
@@ -27,6 +29,8 @@ Features
     - JSHint/JSLint
     - everything else
 
+**Internet Explorer** and **PhantomJS** are still supported as launchers, but we document them as **deprecated** targets: keeping them viable through transpilation and polyfills is likely to get more difficult over time, so prefer evergreen browsers or Node for new projects. See the [configuration reference](docs/config_file.md) for how we categorize browsers.
+
 Screencasts
 -----------
 
@@ -61,7 +65,7 @@ You will see a terminal-based interface which looks like this
 
 ![Initial interface](https://github.com/testem/testem/raw/master/images/initial.png)
 
-Now open your browser and go to the specified URL. You should now see
+Now open a **real browser** (the URL Testem prints is a normal page in Chrome, Firefox, Safari, etc.) and go to the specified URL. You should now see
 
 ![Zero of zero](https://github.com/testem/testem/raw/master/images/zeros.png)
 
@@ -107,6 +111,28 @@ In development mode, Testem has a text-based graphical user interface which uses
 * d : half a page down target text panel
 * u : half a page up target text panel
 
+### File watching
+
+In development mode, Testem watches your project directory for changes and re-runs tests when a
+relevant file is added, edited, or removed. Watching is implemented with
+[chokidar](https://github.com/paulmillr/chokidar) (v5).
+
+* **`src_files`** — Glob patterns for source files whose changes should trigger a run (defaults to
+  `*.js` when unset). This is the main *watch list*.
+* **`watch_files`** — Optional; if set, these patterns are watched instead of defaulting to
+  `src_files` (see `docs/config_file.md`).
+* **`src_files_ignore`** — Patterns to exclude from the watch policy (e.g. `node_modules`).
+* **`disable_watching`** — Set to `true` to turn off the file watcher entirely.
+
+Testem watches the **current working directory** and applies your include/ignore patterns to
+events from the watcher. You do not need to list every file explicitly; globs and ignores follow
+the same policy as in the config reference.
+
+**Troubleshooting:** On some setups (Docker, network filesystems, VMs), native `fs.watch` can be
+flaky. Chokidar supports environment variables such as `CHOKIDAR_USE_POLLING=1` (force polling)
+and `CHOKIDAR_INTERVAL` (polling interval in ms). See the
+[chokidar readme](https://github.com/paulmillr/chokidar) for details.
+
 ### Command line options
 
 To see all command line options
@@ -134,9 +160,7 @@ Will print them out. The output might look like
 
     $ testem launchers
     Browsers available on this system:
-    IE7
-    IE8
-    IE9
+    IE11
     Chrome
     Firefox
     Safari
@@ -144,8 +168,6 @@ Will print them out. The output might look like
     Opera
     PhantomJS
 
-Did you notice that this system has IE versions 7-9? Yes, actually it has only IE9 installed, but Testem uses IE's compatibility mode feature to emulate IE 7 and 8.
-
 When you run `testem ci` to run tests, it outputs the results in the [TAP](http://testanything.org/) format by default, which looks like
 
     ok 1 Chrome 16.0 - hello should say hello.
@@ -455,7 +477,8 @@ If you need to run a preprocessor (or indeed any shell command before the start
 
     "before_tests": "coffee -c *.coffee"
 
-And Testem will run it before each test run. For file watching, you may still use the `src_files` option
+And Testem will run it before each test run. Point **`src_files`** at the sources you want
+watched (see **File watching** under Development Mode above).
 
 ```javascript
 "src_files": [

package/lib/api.js

@@ -1,6 +1,6 @@
 
 
-const log = require('npmlog');
+const log = require('./log');
 const Config = require('./config');
 const EventEmitter = require('events').EventEmitter;
 const fs = require('fs');

package/lib/app.js

@@ -1,9 +1,7 @@
-
-
 const EventEmitter = require('events').EventEmitter;
-const Bluebird = require('bluebird');
+const { fromCallback, using, disposer, mapLimit } = require('./utils/promises');
 const Path = require('path');
-const log = require('npmlog');
+const log = require('./log');
 const StyledString = require('styled_string');
 const _ = require('lodash');
 
@@ -40,7 +38,7 @@ module.exports = class App extends EventEmitter {
 
     let alreadyExit = false;
 
-    this.cleanExit = err => {
+    this.cleanExit = (err) => {
       if (!alreadyExit) {
         alreadyExit = true;
 
@@ -52,7 +50,9 @@ module.exports = class App extends EventEmitter {
 
         if (this.testSuiteTimedOut === true) {
           let timeoutSeconds = this.testSuiteTimeout.timeout;
-          err = new Error(`Test suite execution has timed out (config.timeout = ${timeoutSeconds} seconds). Terminated all test runners.`);
+          err = new Error(
+            `Test suite execution has timed out (config.timeout = ${timeoutSeconds} seconds). Terminated all test runners.`,
+          );
           exitCode = 1;
         }
 
@@ -64,53 +64,63 @@ module.exports = class App extends EventEmitter {
   start(cb) {
     log.info('Starting ' + this.config.appMode);
 
-    return Bluebird.using(SignalListeners.with(), signalListeners => {
-      signalListeners.on('signal', err => this.exit(err));
+    return using(SignalListeners.with(), (signalListeners) => {
+      signalListeners.on('signal', (err) => this.exit(err));
 
-      return Bluebird.using(Reporter.with(this, this.stdoutStream, this.reportFileName), reporter => {
-        this.reporter = reporter;
+      return using(
+        Reporter.with(this, this.stdoutStream, this.reportFileName),
+        (reporter) => {
+          this.reporter = reporter;
 
-        return Bluebird.using(this.fileWatch(), () => {
-          return Bluebird.using(this.getServer(), () => {
-            return Bluebird.using(this.getRunners(), () => {
-              return Bluebird.using(this.runHook('on_start'), () => {
-                let w = this.waitForTests();
+          return using(this.fileWatch(), () => {
+            return using(this.getServer(), () => {
+              return using(this.getRunners(), () => {
+                return using(this.runHook('on_start'), () => {
+                  let w = this.waitForTests();
 
-                if (cb) {
-                  cb();
-                }
+                  if (cb) {
+                    cb();
+                  }
 
-                return w;
-              }).then(() => {
-                log.info('Stopping ' + this.config.appMode);
+                  return w;
+                })
+                  .then(() => {
+                    log.info('Stopping ' + this.config.appMode);
 
-                this.emit('tests-finish');
+                    this.emit('tests-finish');
 
-                return Bluebird.using(this.runHook('on_exit'), () => {});
-              }).catch(error => {
-                log.error(error);
-                log.info('Stopping ' + this.config.appMode);
+                    return using(this.runHook('on_exit'), () => {});
+                  })
+                  .catch((error) => {
+                    log.error(error);
+                    log.info('Stopping ' + this.config.appMode);
 
-                this.emit('tests-error');
+                    this.emit('tests-error');
 
-                return new Bluebird((resolve, reject) => {
-                  Bluebird.using(this.runHook('on_exit'), () => {}).then(() => {
-                    reject(error);
+                    return new Promise((resolve, reject) => {
+                      using(this.runHook('on_exit'), () => {}).then(() => {
+                        reject(error);
+                      });
+                    });
                   });
-                });
               });
             });
           });
-        });
-      });
-    }).asCallback(this.cleanExit);
+        },
+      );
+    }).then(
+      () => this.cleanExit(),
+      (err) => this.cleanExit(err),
+    );
   }
 
   waitForTests() {
     log.info('Waiting for tests.');
 
     if (this.exited) {
-      return Bluebird.reject(this.exitErr || new Error('Testem exited before running any tests.'));
+      return Promise.reject(
+        this.exitErr || new Error('Testem exited before running any tests.'),
+      );
     }
 
     let run = this.triggerRun('Start');
@@ -119,7 +129,7 @@ module.exports = class App extends EventEmitter {
       run.then(() => this.exit());
     }
 
-    return new Bluebird.Promise((resolve, reject) => {
+    return new Promise((resolve, reject) => {
       this.on('testFinish', resolve);
       this.on('testError', reject);
     });
@@ -133,38 +143,42 @@ module.exports = class App extends EventEmitter {
     }
     this.restarting = true;
 
-    return this.stopCurrentRun().catch(this.exit.bind(this)).then(() => {
-      this.restarting = false;
+    return this.stopCurrentRun()
+      .catch(this.exit.bind(this))
+      .then(() => {
+        this.restarting = false;
 
-      return this.runTests();
-    });
+        return this.runTests();
+      });
   }
 
   stopCurrentRun() {
     if (!this.currentRun) {
-      return Bluebird.resolve();
+      return Promise.resolve();
     }
 
-    return Bluebird.all([ this.stopRunners(), this.currentRun ]);
+    return Promise.all([this.stopRunners(), this.currentRun]);
   }
 
   runTests() {
     if (this.paused) {
-      return Bluebird.resolve();
+      return Promise.resolve();
     }
 
     log.info('Running tests...');
 
     this.reporter.onStart('testem', { launcherId: 0 });
 
-    return Bluebird.using(this.runHook('before_tests'), () => {
-      return Bluebird.using(RunTimeout.with(this.config.get('timeout')), timeout => {
+    return using(this.runHook('before_tests'), () => {
+      return using(RunTimeout.with(this.config.get('timeout')), (timeout) => {
         this.testSuiteTimeout = timeout;
 
         timeout.on('timeout', () => {
           let timeoutSeconds = timeout.timeout;
 
-          log.info(`Test suite execution has timed out (config.timeout = ${timeoutSeconds} seconds). Terminating all test runners`);
+          log.info(
+            `Test suite execution has timed out (config.timeout = ${timeoutSeconds} seconds). Terminating all test runners`,
+          );
           this.testSuiteTimedOut = true;
           this.killRunners();
         });
@@ -176,25 +190,27 @@ module.exports = class App extends EventEmitter {
 
         return this.currentRun;
       }).then(() => {
-        return Bluebird.using(this.runHook('after_tests'), () => {});
+        return using(this.runHook('after_tests'), () => {});
       });
-    }).catch(err => {
-      if (err.hideFromReporter) {
-        return;
-      }
-
-      let result = {
-        failed: 1,
-        passed: 0,
-        name: 'testem',
-        launcherId: 0,
-        error: {
-          message: err.toString()
+    })
+      .catch((err) => {
+        if (err.hideFromReporter) {
+          return;
         }
-      };
 
-      this.reporter.report('testem', result);
-    }).finally(() => this.reporter.onEnd('testem', { launcherId: 0 }));
+        let result = {
+          failed: 1,
+          passed: 0,
+          name: 'testem',
+          launcherId: 0,
+          error: {
+            message: err.toString(),
+          },
+        };
+
+        this.reporter.report('testem', result);
+      })
+      .finally(() => this.reporter.onEnd('testem', { launcherId: 0 }));
   }
 
   exit(err, cb) {
@@ -221,7 +237,7 @@ module.exports = class App extends EventEmitter {
     return;
   }
 
-  startServer(callback) {
+  startServer() {
     log.info('Starting server');
     this.server = new Server(this.config);
     this.server.on('file-requested', this.onFileRequested.bind(this));
@@ -229,16 +245,29 @@ module.exports = class App extends EventEmitter {
     this.server.on('browser-relogin', this.onBrowserRelogin.bind(this));
     this.server.on('server-error', this.onServerError.bind(this));
 
-    return this.server.start().asCallback(callback);
+    return this.server.start();
   }
 
   getServer() {
-    return this.startServer().disposer(() => this.stopServer());
+    return disposer(Promise.resolve(this.startServer()), () =>
+      this.stopServer(),
+    );
   }
 
-  onFileRequested(filepath) {
+  /**
+   * When the server requests a file, add it to the file watcher if it is not already being watched.
+   * If the file watcher is disabled, do nothing.
+   *
+   * @param {string} filepath
+   * @returns {Promise<void>}
+   */
+  async onFileRequested(filepath) {
     if (this.fileWatcher && !this.config.get('serve_files')) {
-      this.fileWatcher.add(filepath);
+      try {
+        await this.fileWatcher.add(filepath);
+      } catch (err) {
+        log.error(err);
+      }
     }
   }
 
@@ -251,18 +280,30 @@ module.exports = class App extends EventEmitter {
   }
 
   onBrowserLogin(browserName, id, socket) {
-    let browser = _.find(this.runners, runner => {
-      return runner.launcherId === id && (!runner.socket || !runner.socket.connected);
+    let browser = _.find(this.runners, (runner) => {
+      return (
+        runner.launcherId === id && (!runner.socket || !runner.socket.connected)
+      );
     });
 
     if (!browser) {
-      let launcher = new LauncherFactory(browserName, {
-        id: id,
-        protocol: 'browser'
-      }, this.config).create();
+      let launcher = new LauncherFactory(
+        browserName,
+        {
+          id: id,
+          protocol: 'browser',
+        },
+        this.config,
+      ).create();
       const singleRun = this.config.get('single_run');
 
-      browser = new BrowserTestRunner(launcher, this.reporter, this.runnerIndex++, singleRun, this.config);
+      browser = new BrowserTestRunner(
+        launcher,
+        this.reporter,
+        this.runnerIndex++,
+        singleRun,
+        this.config,
+      );
       this.addRunner(browser);
     }
 
@@ -270,10 +311,12 @@ module.exports = class App extends EventEmitter {
   }
 
   onBrowserRelogin(browserName, id, socket) {
-    let browser = _.find(this.runners, runner => {
+    let browser = _.find(this.runners, (runner) => {
       // a browser relogin can happen if a client socket was disconnected, which may not be reflected in runner.socket's connected state
       // or if the socket was nulled by 'onDisconnect'
-      return runner.launcherId === id && (runner.socket || runner.socket === null);
+      return (
+        runner.launcherId === id && (runner.socket || runner.socket === null)
+      );
     });
 
     if (!browser) {
@@ -293,55 +336,74 @@ module.exports = class App extends EventEmitter {
     this.emit('runnerAdded', runner);
   }
 
+  // Watcher teardown is this disposer's cleanup (using() awaits it on success and on setup failure).
   fileWatch() {
-    return this.configureFileWatch().disposer(() => {});
+    return disposer(Promise.resolve(this.configureFileWatch()), () => {
+      if (this.fileWatcher) {
+        const w = this.fileWatcher;
+        this.fileWatcher = undefined;
+        return w.close();
+      }
+    });
   }
 
-  configureFileWatch(cb) {
+  configureFileWatch() {
     if (this.config.get('disable_watching')) {
-      return Bluebird.resolve().asCallback(cb);
+      return Promise.resolve();
     }
 
-    this.fileWatcher = new FileWatcher(this.config);
-    this.fileWatcher.on('fileChanged', filepath => {
-      log.info(filepath + ' changed (' + (this.disableFileWatch ? 'disabled' : 'enabled') + ').');
-      if (this.disableFileWatch || this.paused) {
-        return;
-      }
-      let configFile = this.config.get('file');
-      if ((configFile && filepath === Path.resolve(configFile)) ||
-        (this.config.isCwdMode() && filepath === process.cwd())) {
-        // config changed
-        this.configure(() => {
-          this.triggerRun('Config changed');
-        });
-      } else {
-        Bluebird.using(this.runHook('on_change', {file: filepath}), () => {
-          this.triggerRun('File changed: ' + filepath);
-        });
-      }
-    });
-    this.fileWatcher.on('EMFILE', () => {
-      let view = this.view;
-      let text = [
-        'The file watcher received a EMFILE system error, which means that ',
-        'it has hit the maximum number of files that can be open at a time. ',
-        'Luckily, you can increase this limit as a workaround. See the directions below \n \n',
-        'Linux: http://stackoverflow.com/a/34645/5304\n',
-        'Mac OS: http://serverfault.com/a/15575/47234'
-      ].join('');
-      view.setErrorPopupMessage(new StyledString(text + '\n ').foreground('megenta'));
+    return FileWatcher.create(this.config).then((fw) => {
+      this.fileWatcher = fw;
+      this.fileWatcher.on('fileChanged', (filepath) => {
+        log.info(
+          filepath +
+            ' changed (' +
+            (this.disableFileWatch ? 'disabled' : 'enabled') +
+            ').',
+        );
+        if (this.disableFileWatch || this.paused) {
+          return;
+        }
+        let configFile = this.config.get('file');
+        if (
+          (configFile && filepath === Path.resolve(configFile)) ||
+          (this.config.isCwdMode() && filepath === process.cwd())
+        ) {
+          // config changed
+          this.configure(() => {
+            this.triggerRun('Config changed');
+          });
+        } else {
+          using(this.runHook('on_change', { file: filepath }), () => {
+            this.triggerRun('File changed: ' + filepath);
+          });
+        }
+      });
+      this.fileWatcher.on('EMFILE', () => {
+        let view = this.view;
+        let text = [
+          'The file watcher received a EMFILE system error, which means that ',
+          'it has hit the maximum number of files that can be open at a time. ',
+          'Luckily, you can increase this limit as a workaround. See the directions below \n \n',
+          'Linux: http://stackoverflow.com/a/34645/5304\n',
+          'Mac OS: http://serverfault.com/a/15575/47234',
+        ].join('');
+        view.setErrorPopupMessage(
+          new StyledString(text + '\n ').foreground('megenta'),
+        );
+      });
     });
-
-    return Bluebird.resolve().asCallback(cb);
   }
 
   getRunners() {
-    return Bluebird.fromCallback(callback => {
-      this.createRunners(callback);
-    }).disposer(() => {
-      return this.killRunners();
-    });
+    return disposer(
+      fromCallback((callback) => {
+        this.createRunners(callback);
+      }),
+      () => {
+        return this.killRunners();
+      },
+    );
   }
 
   createRunners(callback) {
@@ -374,56 +436,55 @@ module.exports = class App extends EventEmitter {
       case 'tap':
         return TapProcessTestRunner;
       default:
-        throw new Error('Don\'t know about ' + protocol + ' protocol.');
+        throw new Error('Unknown protocol: ' + protocol);
     }
   }
 
   createTestRunner(launcher, reporter) {
     let singleRun = this.config.get('single_run');
 
-    return new (this.getRunnerFactory(launcher))(launcher, reporter, this.runnerIndex++, singleRun, this.config);
+    return new (this.getRunnerFactory(launcher))(
+      launcher,
+      reporter,
+      this.runnerIndex++,
+      singleRun,
+      this.config,
+    );
   }
 
   withTestTimeout() {
-    return this.startClock().disposer(() => {
+    return disposer(this.startClock(), () => {
       return this.cancelExistingTimeout();
     });
   }
 
   singleRun(timeout) {
     let limit = this.config.get('parallel');
+    let concurrency = limit && limit >= 1 ? parseInt(limit) : Infinity;
 
-    let options = {};
-
-    if (limit && limit >= 1) {
-      options.concurrency = parseInt(limit);
-    } else {
-      options.concurrency = Infinity;
-    }
-
-    return Bluebird.map(this.runners, (runner) => {
+    return mapLimit(this.runners, concurrency, (runner) => {
       if (this.exited) {
         let e = new Error('Run canceled.');
         e.hideFromReporter = true;
-        return Bluebird.reject(e);
+        return Promise.reject(e);
       }
       if (this.restarting) {
-        return Bluebird.resolve();
+        return Promise.resolve();
       }
       return timeout.try(() => runner.start());
-    }, options);
+    });
   }
 
   wrapUp(err) {
     this.exit(err);
   }
 
-  stopServer(callback) {
+  stopServer() {
     if (!this.server) {
-      return Bluebird.resolve().asCallback(callback);
+      return Promise.resolve();
     }
 
-    return this.server.stop().asCallback(callback);
+    return this.server.stop();
   }
 
   getExitCode() {
@@ -442,20 +503,22 @@ module.exports = class App extends EventEmitter {
   }
 
   stopRunners() {
-    return Bluebird.each(this.runners, runner => {
-      if (typeof runner.stop === 'function') {
-        return runner.stop();
-      }
+    return Promise.all(
+      this.runners.map((runner) => {
+        if (typeof runner.stop === 'function') {
+          return runner.stop();
+        }
 
-      return runner.exit();
-    });
+        return runner.exit();
+      }),
+    );
   }
 
   killRunners() {
-    return Bluebird.each(this.runners, runner => runner.exit());
+    return Promise.all(this.runners.map((runner) => runner.exit()));
   }
 
   launchers() {
-    return this.runners.map(runner => runner.launcher);
+    return this.runners.map((runner) => runner.launcher);
   }
 };