@thi.ng/csp 2.1.115 → 3.0.0

package/CHANGELOG.md

@@ -1,6 +1,6 @@
 # Change Log
 
-- **Last updated**: 2024-04-20T14:42:45Z
+- **Last updated**: 2024-04-25T19:44:55Z
 - **Generator**: [thi.ng/monopub](https://thi.ng/monopub)
 
 All notable changes to this project will be documented in this file.
@@ -9,12 +9,46 @@ See [Conventional Commits](https://conventionalcommits.org/) for commit guidelin
 **Note:** Unlisted _patch_ versions only involve non-code or otherwise excluded changes
 and/or version bumps of transitive dependencies.
 
+# [3.0.0](https://github.com/thi-ng/umbrella/tree/@thi.ng/csp@3.0.0) (2024-04-25)
+
+#### 🛑 Breaking changes
+
+- complete rewrite of Channel, Mult, PubSub, remove deps, add new operators ([2cf7431](https://github.com/thi-ng/umbrella/commit/2cf7431))
+- BREAKING CHANGES: complete rewrite of Channel, Mult, PubSub, remove deps, add new operators
+  - remove/replace types
+  - remove buffer impls (now using [@thi.ng/buffers](https://github.com/thi-ng/umbrella/tree/main/packages/buffers))
+  - remove transducer handling (now done externally, e.g. via [@thi.ng/transducers-async](https://github.com/thi-ng/umbrella/tree/main/packages/transducers-async))
+  - replace Channel, Mult, PubSub
+  - remove obsolete dependencies
+  - add/update channel operators:
+    - broadcast()
+    - concat()
+    - consume() / consumeWith()
+    - drain()
+    - fromAsyncIterable()
+    - merge()
+    - pipe()
+    - select()
+    - timeout()
+  - add/update tests, remove old/obsolete ones
+
+#### 🚀 Features
+
+- update consume() to accept opt. limit ([164d1dd](https://github.com/thi-ng/umbrella/commit/164d1dd))
+- add write queue, update tests ([f201401](https://github.com/thi-ng/umbrella/commit/f201401))
+
 ### [2.1.115](https://github.com/thi-ng/umbrella/tree/@thi.ng/csp@2.1.115) (2024-04-20)
 
 #### ♻️ Refactoring
 
 - update type usage ([749b24c](https://github.com/thi-ng/umbrella/commit/749b24c))
 
+### [2.1.114](https://github.com/thi-ng/umbrella/tree/@thi.ng/csp@2.1.114) (2024-04-11)
+
+#### 🚀 Features
+
+- add initial new Channel impl & related ops/tests ([edddf61](https://github.com/thi-ng/umbrella/commit/edddf61))
+
 ### [2.1.113](https://github.com/thi-ng/umbrella/tree/@thi.ng/csp@2.1.113) (2024-04-08)
 
 #### ♻️ Refactoring

package/README.md

@@ -20,47 +20,46 @@
 - [Installation](#installation)
 - [Dependencies](#dependencies)
 - [API](#api)
-  - [File loading & word frequency analysis](#file-loading--word-frequency-analysis)
-  - [Channel merging](#channel-merging)
   - [PubSub](#pubsub)
 - [Authors](#authors)
 - [License](#license)
 
 ## About
 
-ES6 promise based CSP primitives & operations.
-
-- `Channel` with/without buffering and/or
-  [transducers](https://github.com/thi-ng/umbrella/blob/develop/packages/transducers)
-  - optional channel IDs
-  - choice of buffer behaviors (fixed, sliding, dropping)
-  - channel selection
-  - channel merging (many-to-one, serial or parallel)
-  - channel piping (w/ transducers)
-  - timeouts / sleeping / throttling / delaying
-  - prepopulated channel ctors (iterators, ranges, promise, constants etc.)
-- `Mult` for channel multiplexing (one-to-many splitting)
-  - individual transducers per tap
-  - dynamic add/removal of taps
-- `PubSub` for topic subscriptions
-  - each topic implemented as `Mult`
-  - wildcard topic for processing fallthrough messages
+Primitives & operators for Communicating Sequential Processes based on async/await and async iterables.
 
-## Status
+This package was temporarily deprecated (throughout most of 2023), but meanwhile
+has been reanimated in the form of a **complete rewrite**, using a new, more
+simple and more modern approach afforded by contemporary ES language features
+(and widespread support for them).
 
-**DEPRECATED** - superseded by other package(s)
+**This new/current implementation is in most cases NOT compatible with earlier
+versions**.
 
-[Search or submit any issues for this package](https://github.com/thi-ng/umbrella/issues?q=%5Bcsp%5D+in%3Atitle)
+Provided are:
 
-This package is deprecated. Please see the following actively maintained
-alternatives providing similar functionality:
+- [CSP `Channel`
+  primitive](https://docs.thi.ng/umbrella/csp/classes/Channel.html) supporting a
+  choice of buffer behaviors (fifo, sliding, dropping, see
+  [thi.ng/buffers](https://github.com/thi-ng/umbrella/blob/develop/packages/buffers)
+  for options)
+- Composable channel operators (see list below)
+- [`Mult`](https://docs.thi.ng/umbrella/csp/classes/Mult.html) for channel
+  multiplexing (one-to-many splitting) and dynamic add/removal of subscribers
+- [`PubSub`](https://docs.thi.ng/umbrella/csp/classes/PubSub.html) for
+  topic-based subscriptions, each topic implemented as `Mult`
 
-- [@thi.ng/fibers](https://github.com/thi-ng/umbrella/tree/develop/packages/fibers)
-- [@thi.ng/rstream](https://github.com/thi-ng/umbrella/tree/develop/packages/rstream)
+## Status
+
+**BETA** - possibly breaking changes forthcoming
+
+[Search or submit any issues for this package](https://github.com/thi-ng/umbrella/issues?q=%5Bcsp%5D+in%3Atitle)
 
 ## Related packages
 
+- [@thi.ng/fibers](https://github.com/thi-ng/umbrella/tree/develop/packages/fibers) - Process hierarchies & operators for cooperative multitasking
 - [@thi.ng/rstream](https://github.com/thi-ng/umbrella/tree/develop/packages/rstream) - Reactive streams & subscription primitives for constructing dataflow graphs / pipelines
+- [@thi.ng/transducers-async](https://github.com/thi-ng/umbrella/tree/develop/packages/transducers-async) - Async versions of various highly composable transducers, reducers and iterators
 
 ## Installation
 
@@ -88,183 +87,51 @@ For Node.js REPL:
 const csp = await import("@thi.ng/csp");
 ```
 
-Package sizes (brotli'd, pre-treeshake): ESM: 2.58 KB
+Package sizes (brotli'd, pre-treeshake): ESM: 1.77 KB
 
 ## Dependencies
 
 - [@thi.ng/api](https://github.com/thi-ng/umbrella/tree/develop/packages/api)
-- [@thi.ng/arrays](https://github.com/thi-ng/umbrella/tree/develop/packages/arrays)
+- [@thi.ng/buffers](https://github.com/thi-ng/umbrella/tree/develop/packages/buffers)
 - [@thi.ng/checks](https://github.com/thi-ng/umbrella/tree/develop/packages/checks)
-- [@thi.ng/dcons](https://github.com/thi-ng/umbrella/tree/develop/packages/dcons)
 - [@thi.ng/errors](https://github.com/thi-ng/umbrella/tree/develop/packages/errors)
-- [@thi.ng/transducers](https://github.com/thi-ng/umbrella/tree/develop/packages/transducers)
 
 ## API
 
 [Generated API docs](https://docs.thi.ng/umbrella/csp/)
 
-### File loading & word frequency analysis
-
-```ts
-import { Channel } from "@thi.ng/csp";
-import * as tx from "@thi.ng/transducers";
-
-// compose transducer to split source file into words
-// and filter out short strings
-const proc: tx.Transducer<string, string> = tx.comp(
-    tx.mapcat((src: string) => src.toLowerCase().split(/[^\w]+/g)),
-    tx.filter((w: string) => w.length > 1)
-);
-
-// define a channel which receives file paths
-// and resolves them with their contents
-const paths = new Channel<any>(
-    tx.map((path: string) =>
-        new Promise<string>(
-            resolve => fs.readFile(path, (_, data) => resolve(data.toString()))
-        )
-    )
-);
-
-// define multiplexed output channel
-// items in this channel will have this form: `[word, count]`
-const results = new Mult("results");
-
-// tap result channel and sum word counts
-const counter = results
-    .tap(tx.map(x => x[1]))
-    .reduce(tx.add());
-
-// 2nd output channel with streaming sort transducer
-// (using a sliding window size of 500 items) and dropping
-// words with < 20 occurrences
-const sorted = results.tap(
-    tx.comp(
-        tx.streamSort(500, x => x[1]),
-        tx.dropWhile(x => x[1] < 20)
-    )
-);
-
-// define workflow:
-// pipe source files into a new channel and
-// reduce this channel using `frequencies` reducer
-// finally stream the result map (word frequencies)
-// into the `sorted` channel
-// (`freqs` is a JS Map and is iterable)
-paths.pipe(proc)
-    .reduce(tx.frequencies())
-    .then(freqs => results.channel().into(freqs));
-
-// start tracing sorted outputs and
-// wait for all to finish
-Promise
-    .all([sorted.consume(), counter])
-    .then(([_, num]) => console.log("total words:", num));
-
-// no real work has been executed thus far (only scheduled via promises)
-// now kick off entire process by writing file paths into the 1st channel
-paths.into(["src/channel.ts", "src/mult.ts", "src/pubsub.ts"]);
-
-// results-tap1 : [ 'let', 20 ]
-// results-tap1 : [ 'topic', 20 ]
-// results-tap1 : [ 'chan', 20 ]
-// results-tap1 : [ 'number', 22 ]
-// results-tap1 : [ 'buf', 23 ]
-// results-tap1 : [ 'length', 23 ]
-// results-tap1 : [ 'tx', 25 ]
-// results-tap1 : [ 'state', 25 ]
-// results-tap1 : [ 'from', 27 ]
-// results-tap1 : [ 'close', 28 ]
-// results-tap1 : [ 'new', 33 ]
-// results-tap1 : [ 'args', 34 ]
-// results-tap1 : [ 'id', 36 ]
-// results-tap1 : [ 'any', 36 ]
-// results-tap1 : [ 'src', 38 ]
-// results-tap1 : [ 'if', 40 ]
-// results-tap1 : [ 'return', 47 ]
-// results-tap1 : [ 'channel', 57 ]
-// results-tap1 : [ 'this', 120 ]
-// results-tap1 done
-// total words: 1607
-```
-
-### Channel merging
-
-```ts
-import { Channel } from "@thi.ng/csp";
-import { push } from "@thi.ng/transducers";
-
-Channel.merge([
-    Channel.range(0, 3),
-    Channel.range(10, 15),
-    Channel.range(100, 110)
-]).reduce(push()).then(console.log);
-
-// [ 0, 100, 101, 102, 103, 1, 2, 104, 105, 10, 11, 12, 13, 106, 14, 107, 108, 109 ]
-
-// emit tuples of values read from all inputs
-// preserves ordering of all inputs, but
-// throughput controlled by slowest input
-// by default stops & closes when any of the inputs closes
-Channel.mergeTuples([
-    Channel.from([1, 2, 3]),
-    Channel.from([10, 20, 30, 40]),
-    Channel.from([100, 200, 300, 400, 500])
-], null, false).consume();
-
-// chan-3 : [ 1, 10, 100 ]
-// chan-3 : [ 2, 20, 200 ]
-// chan-3 : [ 3, 30, 300 ]
-// chan-3 done
-
-// same as above, however continues until all inputs are closed
-Channel.mergeTuples([
-    Channel.from([1, 2, 3]),
-    Channel.from([10, 20, 30, 40]),
-    Channel.from([100, 200, 300, 400, 500])
-], null, false).consume();
-
-// chan-3 : [ 1, 10, 100 ]
-// chan-3 : [ 2, 20, 200 ]
-// chan-3 : [ 3, 30, 300 ]
-// chan-3 : [ undefined, 40, 400 ]
-// chan-3 : [ undefined, undefined, 500 ]
-// chan-3 done
-```
-
 ### PubSub
 
-```ts
-import { Channel, PubSub } from "@thi.ng/csp";
-import { map } from "@thi.ng/transducers";
+```ts tangle:export/readme-pubsub.ts
+import { channel, consumeWith, pubsub } from "@thi.ng/csp";
 
-// define a channel publisher with transducer and topic function applied to each item
-// the input channel receives names and transforms them into indexable objects
-const pub = new PubSub(
-    new Channel<any>("users", map((x: string) => ({ type: x.charAt(0), val: x }))),
-    (x) => x.type
-);
+// input channel (optional)
+const src = channel<string>({ id: "users" });
 
-// create subscriptions (channel + debug consumer)
+// publisher with a topic function
+// (topic here is the first character of each received string)
+const pub = pubsub<string>(src, (x) => x[0]);
+
+// create topic subscriptions (channel & debug consumer)
 // under the hood each topic is a Mult (multiplexed channel)
-// sub channels are automatically named:
+// subscription channels are automatically named:
 // `<src-id>-<topic>-tap<tapid>` (see below)
 for (let i of "abc") {
-    pub.sub(i).consume();
+    consumeWith(pub.subscribeTopic(i), (x, ch) => console.log(ch.id, x));
 }
 
-// start processing, then close everything down
-// (pubsubs & mults are closed recursively once the input channel is closed)
-pub.channel().into(["alice", "bert", "bella", "charlie", "arthur"]);
-
-// users-a-tap0 : { type: 'a', val: 'alice' }
-// users-b-tap0 : { type: 'b', val: 'bert' }
-// users-b-tap0 : { type: 'b', val: 'bella' }
-// users-c-tap0 : { type: 'c', val: 'charlie' }
-// users-a-tap0 : { type: 'a', val: 'arthur' }
-// users-b-tap0 done
-// users-c-tap0 done
-// users-a-tap0 done
+// start processing
+for (let x of ["alice", "bert", "bella", "charlie", "arthur"]) {
+    await src.write(x);
+}
+// users-a-tap0 alice
+// users-b-tap1 bert
+// users-b-tap1 bella
+// users-c-tap2 charlie
+// users-a-tap0 arthur
+
+// pubsubs & mults are closed recursively once we close the input channel
+src.close();
 ```
 
 ## Authors

package/api.d.ts

@@ -1,27 +1,19 @@
-import type { Fn, IID, ILength, IRelease, Maybe } from "@thi.ng/api";
-import type { Channel } from "./channel.js";
-export interface ChannelItem<T> {
-    value(): Promise<T>;
-    resolve(success: boolean): void;
+import type { Fn, Maybe } from "@thi.ng/api";
+import type { IReadWriteBuffer } from "@thi.ng/buffers";
+export type ChannelValue<T> = [T, Fn<boolean, void>];
+export type ChannelBuffer<T> = IReadWriteBuffer<ChannelValue<T>>;
+export interface IClosable {
+    close(): void;
+    closed(): boolean;
 }
-export interface IBuffer<T> extends ILength, IRelease {
-    isEmpty(): boolean;
-    isFull(): boolean;
-    drop(): Maybe<ChannelItem<T>>;
-    push(x: ChannelItem<T>): boolean;
-}
-export interface IChannel<T> extends IID<string> {
-    channel(): Channel<T>;
-    close(flush?: boolean): Maybe<Promise<void>>;
-}
-export interface IReadableChannel<T> extends IChannel<T> {
+export interface IReadable<T> {
     read(): Promise<Maybe<T>>;
 }
-export interface IWriteableChannel<T> extends IChannel<T> {
-    write(val: any): Promise<boolean>;
+export interface IWriteable<T> {
+    write(val: T): Promise<boolean>;
+    writable(): boolean;
 }
-export interface IReadWriteableChannel<T> extends IReadableChannel<T>, IWriteableChannel<T> {
+export interface IChannel<T> extends IReadable<T>, IWriteable<T>, IClosable {
 }
 export type TopicFn<T> = Fn<T, string>;
-export type ErrorHandler = (e: Error, chan: Channel<any>, val?: any) => void;
 //# sourceMappingURL=api.d.ts.map

package/channel.d.ts

@@ -1,154 +1,46 @@
-import type { Fn, Fn0, Fn2, FnAny, Maybe, Predicate } from "@thi.ng/api";
-import { DCons } from "@thi.ng/dcons/dcons";
-import type { Reducer, Transducer } from "@thi.ng/transducers";
-import type { ChannelItem, ErrorHandler, IBuffer, IReadWriteableChannel } from "./api.js";
-declare const enum State {
-    OPEN = 0,
-    CLOSED = 1,
-    DONE = 2
+import type { Fn, Maybe } from "@thi.ng/api";
+import type { ChannelBuffer, ChannelValue, IChannel } from "./api.js";
+export declare const MAX_READS = 1024;
+export declare const MAX_QUEUE = 1024;
+export interface ChannelOpts {
+    id: string;
 }
-export declare class Channel<T> implements IReadWriteableChannel<T> {
-    static constantly<T>(x: T, delay?: number): Channel<T>;
-    static repeatedly<T>(fn: Fn0<T>, delay?: number): Channel<T>;
-    static cycle<T>(src: Iterable<T>, delay?: number): Channel<unknown>;
-    static range(): Channel<number>;
-    static range(to: number): Channel<number>;
-    static range(from: number, to: number): Channel<number>;
-    static range(from: number, to: number, step: number): Channel<number>;
-    static range(from: number, to: number, step: number, delay: number): Channel<number>;
-    /**
-     * Constructs new channel which closes automatically after given period.
-     *
-     * @param delay - time in ms
-     */
-    static timeout(delay: number): Channel<any>;
-    /**
-     * Shorthand for: `Channel.timeout(delay).take()`
-     *
-     * @param delay - time in ms
-     */
-    static sleep(delay: number): Promise<any>;
-    /**
-     * Creates new channel with single value from given promise, then closes
-     * automatically iff promise has been resolved.
-     *
-     * @param p - promise
-     */
-    static fromPromise<T>(p: Promise<T>): Channel<T>;
-    static from<T>(src: Iterable<any>): Channel<T>;
-    static from<T>(src: Iterable<any>, close: boolean): Channel<T>;
-    static from<T>(src: Iterable<any>, tx: Transducer<any, T>): Channel<T>;
-    static from<T>(src: Iterable<any>, tx: Transducer<any, T>, close: boolean): Channel<T>;
+export declare function channel<T>(opts?: Partial<ChannelOpts>): Channel<T>;
+export declare function channel<T>(buf: ChannelBuffer<T> | number, opts?: Partial<ChannelOpts>): Channel<T>;
+export declare class Channel<T> implements IChannel<T> {
+    id: string;
+    writes: ChannelBuffer<T>;
+    queue: ChannelValue<T>[];
+    reads: Fn<Maybe<T>, void>[];
+    races: Fn<Channel<T>, void>[];
+    protected state: number;
+    constructor(opts?: Partial<ChannelOpts>);
+    constructor(buf: ChannelBuffer<T> | number, opts?: Partial<ChannelOpts>);
+    [Symbol.asyncIterator](): AsyncIterableIterator<T>;
+    read(): Promise<Maybe<T>>;
+    poll(): Maybe<T>;
+    write(msg: T): Promise<boolean>;
+    offer(msg: T): boolean;
+    race(): Promise<Channel<T>>;
+    close(): void;
     /**
-     * Takes an array of channels and blocks until any of them becomes
-     * readable (or has been closed). The returned promised resolves into
-     * an array of `[value, channel]`. Channel order is repeatedly
-     * shuffled for each read attempt.
-     *
-     * @param chans - source channels
+     * Returns true if the channel is principally readable (i.e. not yet
+     * closed), however there might not be any values available yet and reads
+     * might block.
      */
-    static select(chans: Channel<any>[]): Promise<any>;
+    readable(): boolean;
     /**
-     * Takes an array of channels to merge into new channel. Any closed
-     * channels will be automatically removed from the input selection.
-     * Once all inputs are closed, the target channel will close too (by
-     * default).
-     *
-     * @remarks
-     * If `named` is true, the merged channel will have tuples of:
-     * `[src-id, val]` If false (default), only received values will be
-     * forwarded.
-     *
-     * @param chans - source channels
-     * @param out - result channel
-     * @param close - true, if result closes
-     * @param named - true, to emit labeled tuples
+     * Returns true if the channel is principally writable (i.e. not closing or
+     * closed), however depending on buffer behavior the channel might not yet
+     * accept new values and writes might block.
      */
-    static merge(chans: Channel<any>[], out?: Channel<any>, close?: boolean, named?: boolean): Channel<any>;
+    writable(): boolean;
     /**
-     * Takes an array of channels to merge into new channel of tuples.
-     * Whereas `Channel.merge()` realizes a sequential merging with no
-     * guarantees about ordering of the output.
-     *
-     * @remarks
-     * The output channel of this function will collect values from all
-     * channels and a new tuple is emitted only once a new value has
-     * been read from ALL channels. Therefore the overall throughput is
-     * dictated by the slowest of the inputs.
-     *
-     * Once any of the inputs closes, the process is terminated and the
-     * output channel is closed too (by default).
-     *
-     * @example
-     * ```ts tangle:../export/merge-tuples.ts
-     * import { Channel } from "@thi.ng/csp";
-     *
-     * Channel.mergeTuples([
-     *   Channel.from([1, 2, 3]),
-     *   Channel.from([10, 20, 30]),
-     *   Channel.from([100, 200, 300])
-     * ]).consume();
-     *
-     * // chan-0 : [ 1, 10, 100 ]
-     * // chan-0 : [ 2, 20, 200 ]
-     * // chan-0 : [ 3, 30, 300 ]
-     * // chan-0 done
-     *
-     * Channel.mergeTuples([
-     *   Channel.from([1, 2, 3]),
-     *   Channel.from([10, 20, 30]),
-     *   Channel.from([100, 200, 300])
-     * ], null, false).consume();
-     * ```
-     *
-     * @param chans - source channels
-     * @param out - result channel
-     * @param closeOnFirst - true, if result closes when first input is done
-     * @param closeOutput - true, if result closes when all inputs are done
+     * Returns true if the channel is fully closed and no further reads or
+     * writes are possible.
      */
-    static mergeTuples(chans: Channel<any>[], out?: Channel<any>, closeOnFirst?: boolean, closeOutput?: boolean): Channel<any>;
-    static MAX_WRITES: number;
-    static NEXT_ID: number;
-    static SCHEDULE: Fn2<FnAny<void>, number, void>;
-    private static RFN;
-    id: string;
-    onerror: ErrorHandler;
-    protected state: State;
-    protected buf: IBuffer<T>;
-    protected tx: Reducer<T, DCons<T>>;
-    protected writes: DCons<ChannelItem<T>>;
-    protected reads: DCons<Fn<T, void>>;
-    protected txbuf: DCons<T>;
-    protected isBusy: boolean;
-    constructor();
-    constructor(id: string);
-    constructor(buf: number | IBuffer<T>);
-    constructor(tx: Transducer<any, T>);
-    constructor(tx: Transducer<any, T>, err: ErrorHandler);
-    constructor(id: string, buf: number | IBuffer<T>);
-    constructor(id: string, tx: Transducer<any, T>);
-    constructor(id: string, tx: Transducer<any, T>, err: ErrorHandler);
-    constructor(id: string, buf: number | IBuffer<T>, tx: Transducer<any, T>);
-    constructor(id: string, buf: number | IBuffer<T>, tx: Transducer<any, T>, err: ErrorHandler);
-    channel(): this;
-    write(value: any): Promise<boolean>;
-    read(): Promise<Maybe<T>>;
-    tryRead(timeout?: number): Promise<unknown>;
-    close(flush?: boolean): Promise<void> | undefined;
-    isClosed(): boolean;
-    isReadable(): boolean;
-    consume(fn?: Fn<T, any>): Promise<void>;
-    produce(fn: Fn0<T>, close?: boolean): Promise<void>;
-    consumeWhileReadable(fn?: Fn<T, any>): Promise<void>;
-    reduce<R>(rfn: Reducer<T, R>, acc?: R): Promise<R>;
-    transduce<A, B>(tx: Transducer<T, A>, rfn: Reducer<A, B>, acc?: B): Promise<B>;
-    into(src: Iterable<T>, close?: boolean): Promise<void>;
-    pipe<R>(dest: Channel<R> | Transducer<T, R>, close?: boolean): Channel<R>;
-    split<A, B>(pred: Predicate<T>, truthy?: Channel<A>, falsey?: Channel<B>, close?: boolean): (Channel<A> | Channel<B>)[];
-    concat(chans: Iterable<Channel<T>>, close?: boolean): Promise<void>;
-    release(): void;
-    protected process(): Promise<void>;
-    protected flush(): void;
+    closed(): boolean;
+    protected deliver(): void;
+    protected updateQueue(): void;
 }
-export {};
 //# sourceMappingURL=channel.d.ts.map