taskchef 3.0.1 → 3.0.2

package/node_modules/proper-lockfile/README.md

@@ -0,0 +1,183 @@
+# proper-lockfile
+
+[![NPM version][npm-image]][npm-url] [![Downloads][downloads-image]][npm-url] [![Build Status][travis-image]][travis-url] [![Coverage Status][codecov-image]][codecov-url] [![Dependency status][david-dm-image]][david-dm-url] [![Dev Dependency status][david-dm-dev-image]][david-dm-dev-url]
+
+[npm-url]:https://npmjs.org/package/proper-lockfile
+[downloads-image]:https://img.shields.io/npm/dm/proper-lockfile.svg
+[npm-image]:https://img.shields.io/npm/v/proper-lockfile.svg
+[travis-url]:https://travis-ci.org/moxystudio/node-proper-lockfile
+[travis-image]:https://img.shields.io/travis/moxystudio/node-proper-lockfile/master.svg
+[codecov-url]:https://codecov.io/gh/moxystudio/node-proper-lockfile
+[codecov-image]:https://img.shields.io/codecov/c/github/moxystudio/node-proper-lockfile/master.svg
+[david-dm-url]:https://david-dm.org/moxystudio/node-proper-lockfile
+[david-dm-image]:https://img.shields.io/david/moxystudio/node-proper-lockfile.svg
+[david-dm-dev-url]:https://david-dm.org/moxystudio/node-proper-lockfile?type=dev
+[david-dm-dev-image]:https://img.shields.io/david/dev/moxystudio/node-proper-lockfile.svg
+
+An inter-process and inter-machine lockfile utility that works on a local or network file system.
+
+
+## Installation
+
+`$ npm install proper-lockfile`
+
+
+## Design
+
+There are various ways to achieve [file locking](http://en.wikipedia.org/wiki/File_locking).
+
+This library utilizes the `mkdir` strategy which works atomically on any kind of file system, even network based ones.
+The lockfile path is based on the file path you are trying to lock by suffixing it with `.lock`.
+
+When a lock is successfully acquired, the lockfile's `mtime` (modified time) is periodically updated to prevent staleness. This allows to effectively check if a lock is stale by checking its `mtime` against a stale threshold. If the update of the mtime fails several times, the lock might be compromised. The `mtime` is [supported](http://en.wikipedia.org/wiki/Comparison_of_file_systems) in almost every `filesystem`.
+
+
+### Comparison
+
+This library is similar to [lockfile](https://github.com/isaacs/lockfile) but the latter has some drawbacks:
+
+- It relies on `open` with `O_EXCL` flag which has problems in network file systems. `proper-lockfile` uses `mkdir` which doesn't have this issue.
+
+> O_EXCL is broken on NFS file systems; programs which rely on it for performing locking tasks will contain a race condition.
+
+- The lockfile staleness check is done via `ctime` (creation time) which is unsuitable for long running processes. `proper-lockfile` constantly updates lockfiles `mtime` to do proper staleness check.
+
+- It does not check if the lockfile was compromised which can lead to undesirable situations. `proper-lockfile` checks the lockfile when updating the `mtime`.
+
+- It has a default value of `0` for the stale option which isn't good because any crash or process kill that the package can't handle gracefully will leave the lock active forever.
+
+
+### Compromised
+
+`proper-lockfile` does not detect cases in which:
+
+- A `lockfile` is manually removed and someone else acquires the lock right after
+- Different `stale`/`update` values are being used for the same file, possibly causing two locks to be acquired on the same file
+
+`proper-lockfile` detects cases in which:
+
+- Updates to the `lockfile` fail
+- Updates take longer than expected, possibly causing the lock to become stale for a certain amount of time
+
+
+As you see, the first two are a consequence of bad usage. Technically, it was possible to detect the first two but it would introduce complexity and eventual race conditions.
+
+
+## Usage
+
+### .lock(file, [options])
+
+Tries to acquire a lock on `file` or rejects the promise on error.
+
+If the lock succeeds, a `release` function is provided that should be called when you want to release the lock. The `release` function also rejects the promise on error (e.g. when the lock was already compromised).
+
+Available options:
+
+- `stale`: Duration in milliseconds in which the lock is considered stale, defaults to `10000` (minimum value is `5000`)
+- `update`: The interval in milliseconds in which the lockfile's `mtime` will be updated, defaults to `stale/2` (minimum value is `1000`, maximum value is `stale/2`)
+- `retries`: The number of retries or a [retry](https://www.npmjs.org/package/retry) options object, defaults to `0`
+- `realpath`: Resolve symlinks using realpath, defaults to `true` (note that if `true`, the `file` must exist previously)
+- `fs`: A custom fs to use, defaults to `graceful-fs`
+- `onCompromised`: Called if the lock gets compromised, defaults to a function that simply throws the error which will probably cause the process to die
+- `lockfilePath`: Custom lockfile path. e.g.: If you want to lock a directory and create the lock file inside it, you can pass `file` as `<dir path>` and `options.lockfilePath` as `<dir path>/dir.lock`
+
+
+```js
+const lockfile = require('proper-lockfile');
+
+lockfile.lock('some/file')
+.then((release) => {
+    // Do something while the file is locked
+
+    // Call the provided release function when you're done,
+    // which will also return a promise
+    return release();
+})
+.catch((e) => {
+    // either lock could not be acquired
+    // or releasing it failed
+    console.error(e)
+});
+
+// Alternatively, you may use lockfile('some/file') directly.
+```
+
+
+### .unlock(file, [options])
+
+Releases a previously acquired lock on `file` or rejects the promise on error.
+
+Whenever possible you should use the `release` function instead (as exemplified above). Still there are cases in which it's hard to keep a reference to it around code. In those cases `unlock()` might be handy.
+
+Available options:
+
+- `realpath`: Resolve symlinks using realpath, defaults to `true` (note that if `true`, the `file` must exist previously)
+- `fs`: A custom fs to use, defaults to `graceful-fs`
+- `lockfilePath`: Custom lockfile path. e.g.: If you want to lock a directory and create the lock file inside it, you can pass `file` as `<dir path>` and `options.lockfilePath` as `<dir path>/dir.lock`
+
+
+```js
+const lockfile = require('proper-lockfile');
+
+lockfile.lock('some/file')
+.then(() => {
+    // Do something while the file is locked
+
+    // Later..
+    return lockfile.unlock('some/file');
+});
+```
+
+### .check(file, [options])
+
+Check if the file is locked and its lockfile is not stale, rejects the promise on error.
+
+Available options:
+
+- `stale`: Duration in milliseconds in which the lock is considered stale, defaults to `10000` (minimum value is `5000`)
+- `realpath`: Resolve symlinks using realpath, defaults to `true` (note that if `true`, the `file` must exist previously)
+- `fs`: A custom fs to use, defaults to `graceful-fs`
+- `lockfilePath`: Custom lockfile path. e.g.: If you want to lock a directory and create the lock file inside it, you can pass `file` as `<dir path>` and `options.lockfilePath` as `<dir path>/dir.lock`
+
+
+```js
+const lockfile = require('proper-lockfile');
+
+lockfile.check('some/file')
+.then((isLocked) => {
+    // isLocked will be true if 'some/file' is locked, false otherwise
+});
+```
+
+### .lockSync(file, [options])
+
+Sync version of `.lock()`.   
+Returns the `release` function or throws on error.
+
+### .unlockSync(file, [options])
+
+Sync version of `.unlock()`.   
+Throws on error.
+
+### .checkSync(file, [options])
+
+Sync version of `.check()`.
+Returns a boolean or throws on error.
+
+
+## Graceful exit
+
+`proper-lockfile` automatically removes locks if the process exits, except if the process is killed with SIGKILL or it crashes due to a VM fatal error (e.g.: out of memory).
+
+
+## Tests
+
+`$ npm test`   
+`$ npm test -- --watch` during development
+
+The test suite is very extensive. There's even a stress test to guarantee exclusiveness of locks.
+
+
+## License
+
+Released under the [MIT License](https://www.opensource.org/licenses/mit-license.php).

package/node_modules/proper-lockfile/index.js

@@ -0,0 +1,40 @@
+'use strict';
+
+const lockfile = require('./lib/lockfile');
+const { toPromise, toSync, toSyncOptions } = require('./lib/adapter');
+
+async function lock(file, options) {
+    const release = await toPromise(lockfile.lock)(file, options);
+
+    return toPromise(release);
+}
+
+function lockSync(file, options) {
+    const release = toSync(lockfile.lock)(file, toSyncOptions(options));
+
+    return toSync(release);
+}
+
+function unlock(file, options) {
+    return toPromise(lockfile.unlock)(file, options);
+}
+
+function unlockSync(file, options) {
+    return toSync(lockfile.unlock)(file, toSyncOptions(options));
+}
+
+function check(file, options) {
+    return toPromise(lockfile.check)(file, options);
+}
+
+function checkSync(file, options) {
+    return toSync(lockfile.check)(file, toSyncOptions(options));
+}
+
+module.exports = lock;
+module.exports.lock = lock;
+module.exports.unlock = unlock;
+module.exports.lockSync = lockSync;
+module.exports.unlockSync = unlockSync;
+module.exports.check = check;
+module.exports.checkSync = checkSync;

package/node_modules/proper-lockfile/lib/adapter.js

@@ -0,0 +1,85 @@
+'use strict';
+
+const fs = require('graceful-fs');
+
+function createSyncFs(fs) {
+    const methods = ['mkdir', 'realpath', 'stat', 'rmdir', 'utimes'];
+    const newFs = { ...fs };
+
+    methods.forEach((method) => {
+        newFs[method] = (...args) => {
+            const callback = args.pop();
+            let ret;
+
+            try {
+                ret = fs[`${method}Sync`](...args);
+            } catch (err) {
+                return callback(err);
+            }
+
+            callback(null, ret);
+        };
+    });
+
+    return newFs;
+}
+
+// ----------------------------------------------------------
+
+function toPromise(method) {
+    return (...args) => new Promise((resolve, reject) => {
+        args.push((err, result) => {
+            if (err) {
+                reject(err);
+            } else {
+                resolve(result);
+            }
+        });
+
+        method(...args);
+    });
+}
+
+function toSync(method) {
+    return (...args) => {
+        let err;
+        let result;
+
+        args.push((_err, _result) => {
+            err = _err;
+            result = _result;
+        });
+
+        method(...args);
+
+        if (err) {
+            throw err;
+        }
+
+        return result;
+    };
+}
+
+function toSyncOptions(options) {
+    // Shallow clone options because we are oging to mutate them
+    options = { ...options };
+
+    // Transform fs to use the sync methods instead
+    options.fs = createSyncFs(options.fs || fs);
+
+    // Retries are not allowed because it requires the flow to be sync
+    if (
+        (typeof options.retries === 'number' && options.retries > 0) ||
+        (options.retries && typeof options.retries.retries === 'number' && options.retries.retries > 0)
+    ) {
+        throw Object.assign(new Error('Cannot use retries with the sync api'), { code: 'ESYNC' });
+    }
+
+    return options;
+}
+
+module.exports = {
+    toPromise,
+    toSync,
+    toSyncOptions,
+};

package/node_modules/proper-lockfile/lib/lockfile.js

@@ -0,0 +1,342 @@
+'use strict';
+
+const path = require('path');
+const fs = require('graceful-fs');
+const retry = require('retry');
+const onExit = require('signal-exit');
+const mtimePrecision = require('./mtime-precision');
+
+const locks = {};
+
+function getLockFile(file, options) {
+    return options.lockfilePath || `${file}.lock`;
+}
+
+function resolveCanonicalPath(file, options, callback) {
+    if (!options.realpath) {
+        return callback(null, path.resolve(file));
+    }
+
+    // Use realpath to resolve symlinks
+    // It also resolves relative paths
+    options.fs.realpath(file, callback);
+}
+
+function acquireLock(file, options, callback) {
+    const lockfilePath = getLockFile(file, options);
+
+    // Use mkdir to create the lockfile (atomic operation)
+    options.fs.mkdir(lockfilePath, (err) => {
+        if (!err) {
+            // At this point, we acquired the lock!
+            // Probe the mtime precision
+            return mtimePrecision.probe(lockfilePath, options.fs, (err, mtime, mtimePrecision) => {
+                // If it failed, try to remove the lock..
+                /* istanbul ignore if */
+                if (err) {
+                    options.fs.rmdir(lockfilePath, () => {});
+
+                    return callback(err);
+                }
+
+                callback(null, mtime, mtimePrecision);
+            });
+        }
+
+        // If error is not EEXIST then some other error occurred while locking
+        if (err.code !== 'EEXIST') {
+            return callback(err);
+        }
+
+        // Otherwise, check if lock is stale by analyzing the file mtime
+        if (options.stale <= 0) {
+            return callback(Object.assign(new Error('Lock file is already being held'), { code: 'ELOCKED', file }));
+        }
+
+        options.fs.stat(lockfilePath, (err, stat) => {
+            if (err) {
+                // Retry if the lockfile has been removed (meanwhile)
+                // Skip stale check to avoid recursiveness
+                if (err.code === 'ENOENT') {
+                    return acquireLock(file, { ...options, stale: 0 }, callback);
+                }
+
+                return callback(err);
+            }
+
+            if (!isLockStale(stat, options)) {
+                return callback(Object.assign(new Error('Lock file is already being held'), { code: 'ELOCKED', file }));
+            }
+
+            // If it's stale, remove it and try again!
+            // Skip stale check to avoid recursiveness
+            removeLock(file, options, (err) => {
+                if (err) {
+                    return callback(err);
+                }
+
+                acquireLock(file, { ...options, stale: 0 }, callback);
+            });
+        });
+    });
+}
+
+function isLockStale(stat, options) {
+    return stat.mtime.getTime() < Date.now() - options.stale;
+}
+
+function removeLock(file, options, callback) {
+    // Remove lockfile, ignoring ENOENT errors
+    options.fs.rmdir(getLockFile(file, options), (err) => {
+        if (err && err.code !== 'ENOENT') {
+            return callback(err);
+        }
+
+        callback();
+    });
+}
+
+function updateLock(file, options) {
+    const lock = locks[file];
+
+    // Just for safety, should never happen
+    /* istanbul ignore if */
+    if (lock.updateTimeout) {
+        return;
+    }
+
+    lock.updateDelay = lock.updateDelay || options.update;
+    lock.updateTimeout = setTimeout(() => {
+        lock.updateTimeout = null;
+
+        // Stat the file to check if mtime is still ours
+        // If it is, we can still recover from a system sleep or a busy event loop
+        options.fs.stat(lock.lockfilePath, (err, stat) => {
+            const isOverThreshold = lock.lastUpdate + options.stale < Date.now();
+
+            // If it failed to update the lockfile, keep trying unless
+            // the lockfile was deleted or we are over the threshold
+            if (err) {
+                if (err.code === 'ENOENT' || isOverThreshold) {
+                    return setLockAsCompromised(file, lock, Object.assign(err, { code: 'ECOMPROMISED' }));
+                }
+
+                lock.updateDelay = 1000;
+
+                return updateLock(file, options);
+            }
+
+            const isMtimeOurs = lock.mtime.getTime() === stat.mtime.getTime();
+
+            if (!isMtimeOurs) {
+                return setLockAsCompromised(
+                    file,
+                    lock,
+                    Object.assign(
+                        new Error('Unable to update lock within the stale threshold'),
+                        { code: 'ECOMPROMISED' }
+                    ));
+            }
+
+            const mtime = mtimePrecision.getMtime(lock.mtimePrecision);
+
+            options.fs.utimes(lock.lockfilePath, mtime, mtime, (err) => {
+                const isOverThreshold = lock.lastUpdate + options.stale < Date.now();
+
+                // Ignore if the lock was released
+                if (lock.released) {
+                    return;
+                }
+
+                // If it failed to update the lockfile, keep trying unless
+                // the lockfile was deleted or we are over the threshold
+                if (err) {
+                    if (err.code === 'ENOENT' || isOverThreshold) {
+                        return setLockAsCompromised(file, lock, Object.assign(err, { code: 'ECOMPROMISED' }));
+                    }
+
+                    lock.updateDelay = 1000;
+
+                    return updateLock(file, options);
+                }
+
+                // All ok, keep updating..
+                lock.mtime = mtime;
+                lock.lastUpdate = Date.now();
+                lock.updateDelay = null;
+                updateLock(file, options);
+            });
+        });
+    }, lock.updateDelay);
+
+    // Unref the timer so that the nodejs process can exit freely
+    // This is safe because all acquired locks will be automatically released
+    // on process exit
+
+    // We first check that `lock.updateTimeout.unref` exists because some users
+    // may be using this module outside of NodeJS (e.g., in an electron app),
+    // and in those cases `setTimeout` return an integer.
+    /* istanbul ignore else */
+    if (lock.updateTimeout.unref) {
+        lock.updateTimeout.unref();
+    }
+}
+
+function setLockAsCompromised(file, lock, err) {
+    // Signal the lock has been released
+    lock.released = true;
+
+    // Cancel lock mtime update
+    // Just for safety, at this point updateTimeout should be null
+    /* istanbul ignore if */
+    if (lock.updateTimeout) {
+        clearTimeout(lock.updateTimeout);
+    }
+
+    if (locks[file] === lock) {
+        delete locks[file];
+    }
+
+    lock.options.onCompromised(err);
+}
+
+// ----------------------------------------------------------
+
+function lock(file, options, callback) {
+    /* istanbul ignore next */
+    options = {
+        stale: 10000,
+        update: null,
+        realpath: true,
+        retries: 0,
+        fs,
+        onCompromised: (err) => { throw err; },
+        ...options,
+    };
+
+    options.retries = options.retries || 0;
+    options.retries = typeof options.retries === 'number' ? { retries: options.retries } : options.retries;
+    options.stale = Math.max(options.stale || 0, 2000);
+    options.update = options.update == null ? options.stale / 2 : options.update || 0;
+    options.update = Math.max(Math.min(options.update, options.stale / 2), 1000);
+
+    // Resolve to a canonical file path
+    resolveCanonicalPath(file, options, (err, file) => {
+        if (err) {
+            return callback(err);
+        }
+
+        // Attempt to acquire the lock
+        const operation = retry.operation(options.retries);
+
+        operation.attempt(() => {
+            acquireLock(file, options, (err, mtime, mtimePrecision) => {
+                if (operation.retry(err)) {
+                    return;
+                }
+
+                if (err) {
+                    return callback(operation.mainError());
+                }
+
+                // We now own the lock
+                const lock = locks[file] = {
+                    lockfilePath: getLockFile(file, options),
+                    mtime,
+                    mtimePrecision,
+                    options,
+                    lastUpdate: Date.now(),
+                };
+
+                // We must keep the lock fresh to avoid staleness
+                updateLock(file, options);
+
+                callback(null, (releasedCallback) => {
+                    if (lock.released) {
+                        return releasedCallback &&
+                            releasedCallback(Object.assign(new Error('Lock is already released'), { code: 'ERELEASED' }));
+                    }
+
+                    // Not necessary to use realpath twice when unlocking
+                    unlock(file, { ...options, realpath: false }, releasedCallback);
+                });
+            });
+        });
+    });
+}
+
+function unlock(file, options, callback) {
+    options = {
+        fs,
+        realpath: true,
+        ...options,
+    };
+
+    // Resolve to a canonical file path
+    resolveCanonicalPath(file, options, (err, file) => {
+        if (err) {
+            return callback(err);
+        }
+
+        // Skip if the lock is not acquired
+        const lock = locks[file];
+
+        if (!lock) {
+            return callback(Object.assign(new Error('Lock is not acquired/owned by you'), { code: 'ENOTACQUIRED' }));
+        }
+
+        lock.updateTimeout && clearTimeout(lock.updateTimeout); // Cancel lock mtime update
+        lock.released = true; // Signal the lock has been released
+        delete locks[file]; // Delete from locks
+
+        removeLock(file, options, callback);
+    });
+}
+
+function check(file, options, callback) {
+    options = {
+        stale: 10000,
+        realpath: true,
+        fs,
+        ...options,
+    };
+
+    options.stale = Math.max(options.stale || 0, 2000);
+
+    // Resolve to a canonical file path
+    resolveCanonicalPath(file, options, (err, file) => {
+        if (err) {
+            return callback(err);
+        }
+
+        // Check if lockfile exists
+        options.fs.stat(getLockFile(file, options), (err, stat) => {
+            if (err) {
+                // If does not exist, file is not locked. Otherwise, callback with error
+                return err.code === 'ENOENT' ? callback(null, false) : callback(err);
+            }
+
+            // Otherwise, check if lock is stale by analyzing the file mtime
+            return callback(null, !isLockStale(stat, options));
+        });
+    });
+}
+
+function getLocks() {
+    return locks;
+}
+
+// Remove acquired locks on exit
+/* istanbul ignore next */
+onExit(() => {
+    for (const file in locks) {
+        const options = locks[file].options;
+
+        try { options.fs.rmdirSync(getLockFile(file, options)); } catch (e) { /* Empty */ }
+    }
+});
+
+module.exports.lock = lock;
+module.exports.unlock = unlock;
+module.exports.check = check;
+module.exports.getLocks = getLocks;

package/node_modules/proper-lockfile/lib/mtime-precision.js

@@ -0,0 +1,55 @@
+'use strict';
+
+const cacheSymbol = Symbol();
+
+function probe(file, fs, callback) {
+    const cachedPrecision = fs[cacheSymbol];
+
+    if (cachedPrecision) {
+        return fs.stat(file, (err, stat) => {
+            /* istanbul ignore if */
+            if (err) {
+                return callback(err);
+            }
+
+            callback(null, stat.mtime, cachedPrecision);
+        });
+    }
+
+    // Set mtime by ceiling Date.now() to seconds + 5ms so that it's "not on the second"
+    const mtime = new Date((Math.ceil(Date.now() / 1000) * 1000) + 5);
+
+    fs.utimes(file, mtime, mtime, (err) => {
+        /* istanbul ignore if */
+        if (err) {
+            return callback(err);
+        }
+
+        fs.stat(file, (err, stat) => {
+            /* istanbul ignore if */
+            if (err) {
+                return callback(err);
+            }
+
+            const precision = stat.mtime.getTime() % 1000 === 0 ? 's' : 'ms';
+
+            // Cache the precision in a non-enumerable way
+            Object.defineProperty(fs, cacheSymbol, { value: precision });
+
+            callback(null, stat.mtime, precision);
+        });
+    });
+}
+
+function getMtime(precision) {
+    let now = Date.now();
+
+    if (precision === 's') {
+        now = Math.ceil(now / 1000) * 1000;
+    }
+
+    return new Date(now);
+}
+
+module.exports.probe = probe;
+module.exports.getMtime = getMtime;