@qe-libs/rena-wasm 0.1.7 → 0.1.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -91,6 +91,87 @@ const model = ena.fit(rows, { codes, units, conversations, window: 4, weightMode
91
91
 
92
92
  ---
93
93
 
94
+ ## Modeling a Subset of Units
95
+
96
+ `unitsUsed` (= R's `units.used`) lists the units to model, as unit keys (the
97
+ unit column values joined with `__`, as in `model.unitLabels`). Every row
98
+ stays in the data: rows of other units still count as context in the modeled
99
+ units' windows, but those units get no network and never reach sphere
100
+ normalization, centering or rotation. Omit it to model every unit.
101
+
102
+ ```js
103
+ // Students' connections to what teachers said are kept; teachers aren't units.
104
+ const model = ena.fit(rows, { codes, units: ['Role', 'Name'], conversations,
105
+ window: 4, unitsUsed: studentKeys });
106
+ ```
107
+
108
+ `unitsUsed` works in `fit()`, `accumulate()`, `tuneWindowSize()` and `pria()`.
109
+ This differs from dropping the other units' rows, which also removes them from
110
+ everyone's windows.
111
+
112
+ ---
113
+
114
+ ## Infinite Windows
115
+
116
+ `window: Infinity` (= R's `window.size.back = Inf`) connects each line to
117
+ every earlier line in its conversation. It also works as a window in a
118
+ context tensor.
119
+
120
+ ```js
121
+ const model = ena.fit(rows, { codes, units, conversations, window: Infinity });
122
+ ```
123
+
124
+ ---
125
+
126
+ ## Normalization and Centring
127
+
128
+ `sphereNorm` (default `true`, = R's `norm.by = fun_sphere_norm`) scales each
129
+ unit's network to length 1. With `false` (= `fun_skip_sphere_norm`) every
130
+ network is scaled by the longest network's length instead, keeping relative
131
+ magnitudes; for ordered (ONA) models this is
132
+ `ona::model(normalize = skip_sphere_norm)`. `centerAlignToOrigin` (default
133
+ `true`, = `ena.make.set`'s `center.align.to.origin`) leaves zero-network units
134
+ out of the centring mean and at the origin; with `false` the mean is taken
135
+ over, and subtracted from, every unit. It applies to unordered models only —
136
+ ordered models always centre as `ona::model()` does. Both work in `fit()` and
137
+ `pria()`.
138
+
139
+ ```js
140
+ const model = ena.fit(rows, { codes, units, conversations, window: 4,
141
+ sphereNorm: false, centerAlignToOrigin: false });
142
+ ```
143
+
144
+ ---
145
+
146
+ ## Projecting Into Another Model (Custom Rotation)
147
+
148
+ `rotationSet` (= R's `rotation.set` in `ena.make.set()`) projects a model into
149
+ another model's space instead of rotating it: the units are centred on the
150
+ rotation set's `centerVec` and projected with its `rotationMatrix`, and the
151
+ nodes are the rotation set's. Pass `codes` (the rotation set's code order) to
152
+ have it reordered to the model's codes; a rotation set built from other codes
153
+ is rejected. Centroids use the pinned nodes, so goodness of fit is measured
154
+ against the nodes shown. Ordered models keep their own centring, as
155
+ rENA.api's ordered pipeline does.
156
+
157
+ ```js
158
+ const source = ena.fit(rowsA, { codes, units, conversations, window: 4 });
159
+ const model = ena.fit(rowsB, { codes, units, conversations, window: 4,
160
+ rotationSet: {
161
+ rotationMatrix: source.rotationMatrix, rotationCols: source.dims,
162
+ nodes: source.rotation.nodes,
163
+ centerVec: source.rotation.centerVec,
164
+ codes,
165
+ },
166
+ });
167
+ ```
168
+
169
+ `rotationMatrix` and `nodes` may also be arrays of rows (one per connection
170
+ and per code), as R stores them. `pria()` ignores `rotationSet`: it cannot
171
+ rotate the reduced models.
172
+
173
+ ---
174
+
94
175
  ## Reduced-Code Search (PRIA)
95
176
 
96
177
  `pria()` finds the largest set of codes (up to `removeNum`, never leaving
@@ -175,6 +256,40 @@ data[a + nA*b + nA*nB*1] → window for factor combination (a, b)
175
256
  | `dimsReceiver` | These axes use the *response* row's factor values (overrides ground) |
176
257
  | `dimsMode` | These axes use a shared mode value |
177
258
 
259
+ ### Time-based windows
260
+
261
+ Set `timesCol` to measure windows in time instead of lines, and `timeUnit`
262
+ ('secs' (default), 'mins', 'hours', 'days', 'weeks') to give the tensor's
263
+ windows in that unit. Numeric time columns are used as-is; otherwise each
264
+ value is parsed as elapsed time (`MM:SS`, `HH:MM:SS`) or a date-time, in
265
+ seconds, as rENA.api's `parse_date` does. `timesEndCol` uses each row's end
266
+ time instead: a numeric column is a duration added to the start, anything
267
+ else is the end timestamp.
268
+
269
+ ```js
270
+ const tensor = { ...modalityTensor, timesCol: 'Timestamp', timeUnit: 'mins' };
271
+ ```
272
+
273
+ ### Flexible horizons
274
+
275
+ `horizons` gives every unit its own horizon of observation (= tma
276
+ `contexts()` with HOO rules, as rENA.api builds them for flexible horizons).
277
+ `by` is the discriminator column and `rules` maps each of its values to the
278
+ columns that define that value's horizon: a row is in unit U's horizon when
279
+ each of those columns holds a value U has. Each horizon is split on the rules'
280
+ columns — not on the discriminator, which only picks the rule — and windows
281
+ run within each piece. Rows whose discriminator value has no rule are left
282
+ out.
283
+
284
+ ```js
285
+ const model = ena.fit(rows, {
286
+ codes, units, conversations: ['Modality'], tensor,
287
+ horizons: { by: 'Modality', rules: { chat: ['GroupName'], log: ['UserName'] } },
288
+ });
289
+ ```
290
+
291
+ `horizons` works in `fit()`, `accumulate()`, `tuneWindowSize()` and `pria()`.
292
+
178
293
  ### `defaultTensor` helper
179
294
 
180
295
  Express simple windowed accumulation as a tensor (IS_DEFAULT path):
@@ -213,4 +328,9 @@ npm test
213
328
  ```
214
329
 
215
330
  Tests cover the simple windowed pipeline (`test/ena.test.js`) and the
216
- context-tensor path (`test/tensor.test.js`).
331
+ context-tensor path (`test/tensor.test.js`), and check against R: infinite and
332
+ time-based windows, flexible horizons and custom rotations
333
+ (`test/windows-horizons-rotation.test.js`), the normalization and centring
334
+ options against `ena.make.set()` (`test/norm-center.test.js`), and ordered
335
+ models with and without sphere normalization against `ona::model()`
336
+ (`test/ona-model.test.js`, `test/ona-sphere-norm.test.js`).
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@qe-libs/rena-wasm",
3
- "version": "0.1.7",
3
+ "version": "0.1.9",
4
4
  "description": "JavaScript/WebAssembly ENA pipeline — thin orchestration layer over @qe-libs/libqe-wasm",
5
5
  "main": "src/index.js",
6
6
  "type": "module",
package/src/data.js CHANGED
@@ -24,6 +24,10 @@ export function rowKey(row, cols) {
24
24
  * @param {string[]} codes - Code column names
25
25
  * @param {string[]} unitCols - Column(s) that identify a unit (e.g. ['UserName', 'Condition'])
26
26
  * @param {string[]} convoCols - Column(s) that identify a conversation (e.g. ['Condition', 'GroupName'])
27
+ * @param {string[]|null} [unitsUsed=null] - Unit keys (rowKey format) to model
28
+ * (= R's `units.used`). Rows of other units stay in their conversations, so
29
+ * they still count as context in the included units' windows, but they get
30
+ * no unit (unitOf = -1) and so produce no network. null = every unit.
27
31
  *
28
32
  * @returns {{
29
33
  * codeMatrix: Float64Array, // n_rows × n_codes, row-major
@@ -31,14 +35,14 @@ export function rowKey(row, cols) {
31
35
  * nCodes: number,
32
36
  * nUnits: number,
33
37
  * unitLabels: string[], // unit key for each unit index
34
- * unitOf: Int32Array, // unit index for each row
38
+ * unitOf: Int32Array, // unit index for each row (-1 = not a modeled unit)
35
39
  * convoOf: Int32Array, // conversation index for each row
36
40
  * convoGroups: Map<number, number[]>, // convoIdx → [rowIdx, ...]
37
41
  * metaData: Object[], // one metadata object per unit (first-row representative)
38
42
  * metaCols: string[], // names of metadata columns
39
43
  * }}
40
44
  */
41
- export function parseData(rows, codes, unitCols, convoCols) {
45
+ export function parseData(rows, codes, unitCols, convoCols, unitsUsed = null) {
42
46
  const nRows = rows.length;
43
47
  const nCodes = codes.length;
44
48
 
@@ -58,6 +62,8 @@ export function parseData(rows, codes, unitCols, convoCols) {
58
62
  // Per-unit metadata rows — populated on first occurrence of each unit.
59
63
  const metaData = [];
60
64
 
65
+ const used = unitsUsed ? new Set(unitsUsed.map(String)) : null;
66
+
61
67
  for (let i = 0; i < nRows; i++) {
62
68
  const row = rows[i];
63
69
 
@@ -68,7 +74,9 @@ export function parseData(rows, codes, unitCols, convoCols) {
68
74
 
69
75
  // Unit index
70
76
  const uKey = rowKey(row, unitCols);
71
- if (!unitIndex.has(uKey)) {
77
+ if (used && !used.has(uKey)) {
78
+ unitOf[i] = -1;
79
+ } else if (!unitIndex.has(uKey)) {
72
80
  const uIdx = unitIndex.size;
73
81
  unitIndex.set(uKey, uIdx);
74
82
  // First row for this unit → representative metadata
@@ -76,7 +84,7 @@ export function parseData(rows, codes, unitCols, convoCols) {
76
84
  for (const col of metaCols) meta[col] = row[col];
77
85
  metaData[uIdx] = meta;
78
86
  }
79
- unitOf[i] = unitIndex.get(uKey);
87
+ if (unitOf[i] !== -1) unitOf[i] = unitIndex.get(uKey);
80
88
 
81
89
  // Conversation index
82
90
  const cKey = rowKey(row, convoCols);
@@ -90,6 +98,9 @@ export function parseData(rows, codes, unitCols, convoCols) {
90
98
 
91
99
  const unitLabels = Array.from(unitIndex.keys());
92
100
  const nUnits = unitLabels.length;
101
+ if (used && nUnits === 0) {
102
+ throw new Error('opts.unitsUsed matched no units in the data');
103
+ }
93
104
 
94
105
  return { codeMatrix, nRows, nCodes, nUnits, unitLabels, unitOf, convoOf, convoGroups,
95
106
  metaData, metaCols };
package/src/index.js CHANGED
@@ -43,9 +43,9 @@
43
43
  import loadLibQE from '@qe-libs/libqe-wasm';
44
44
  import { parseData } from './data.js';
45
45
  import {
46
- sphereNorm, center,
46
+ sphereNorm, scaleNetworks, center,
47
47
  rotateSVD, rotateMeans, rotateGeneralized,
48
- project, nodePositions, spaceDistCorr,
48
+ project, nodePositions, spaceDistCorr, matmul,
49
49
  } from './pipeline.js';
50
50
  import { accumulateTensor, defaultTensor } from './tensor.js';
51
51
 
@@ -272,22 +272,125 @@ function applyCodeMask(networks, codeMask, nCodes, nUnits, nConnections, ordered
272
272
  }
273
273
  }
274
274
 
275
+ // ── custom rotation (= R's rotation.set) ─────────────────────────────────────
276
+
277
+ // Rows of a matrix given as an array of row arrays, or as flat row-major data
278
+ // with `cols` columns.
279
+ function matrixRows(m, cols) {
280
+ if (m == null) return null;
281
+ if (Array.isArray(m) && (m.length === 0 || Array.isArray(m[0]) || ArrayBuffer.isView(m[0]))) {
282
+ return m.map(r => Array.from(r, Number));
283
+ }
284
+ const flat = Array.from(m, Number);
285
+ if (!cols) throw new Error('a flat rotation matrix needs its column count');
286
+ const rows = [];
287
+ for (let i = 0; i < flat.length; i += cols) rows.push(flat.slice(i, i + cols));
288
+ return rows;
289
+ }
290
+
291
+ /**
292
+ * Validate a supplied rotation set (= R's ena.rotation.set, e.g. another
293
+ * model's rotation) against this model's codes, and reorder it to this
294
+ * model's code order. Throws when it cannot rotate this model.
295
+ *
296
+ * @param {object} rs { rotationMatrix, nodes, centerVec, codes?,
297
+ * rotationCols?, eigenvalues?, columnNames? }
298
+ * rotationMatrix: nConnections rows (array of rows, or
299
+ * flat row-major with rotationCols); nodes: one row per
300
+ * code; centerVec: one entry per connection.
301
+ * @returns {{ rotation: Float64Array, rotRows: number, rotCols: number,
302
+ * nodes: number[][], centerVec: Float64Array,
303
+ * eigenvalues: number[]|null, columnNames: string[] }}
304
+ */
305
+ function resolveRotationSet(rs, codes, nConnections, ordered) {
306
+ const rot = matrixRows(rs.rotationMatrix, rs.rotationCols);
307
+ if (!rot || rot.length !== nConnections) {
308
+ throw new Error(
309
+ `rotationSet has ${rot ? rot.length : 0} connections but this model has ` +
310
+ `${nConnections} -- it was built from different codes` +
311
+ (ordered ? ' or is not an ordered model' : ' or is an ordered model')
312
+ );
313
+ }
314
+ const rotCols = rot[0].length;
315
+ const nodes = matrixRows(rs.nodes, rotCols);
316
+ const center = rs.centerVec ? Array.from(rs.centerVec, Number) : null;
317
+ if (!nodes || nodes.length !== codes.length) {
318
+ throw new Error('rotationSet.nodes must have one row per code');
319
+ }
320
+ if (!ordered && (!center || center.length !== nConnections)) {
321
+ throw new Error('rotationSet.centerVec must have one entry per connection');
322
+ }
323
+
324
+ // Map this model's codes/connections onto the rotation set's order.
325
+ let codeIdx = codes.map((_, i) => i);
326
+ let connIdx = Array.from({ length: nConnections }, (_, k) => k);
327
+ if (rs.codes) {
328
+ const src = rs.codes.map(String);
329
+ codeIdx = codes.map(c => src.indexOf(String(c)));
330
+ if (src.length !== codes.length || codeIdx.some(i => i < 0)) {
331
+ throw new Error('rotationSet.codes do not match this model\'s codes');
332
+ }
333
+ const srcKey = new Map(buildAdjacencyKey(src, ordered).map(([a, b], k) => [a + '\u0000' + b, k]));
334
+ connIdx = buildAdjacencyKey(codes, ordered).map(([a, b]) => {
335
+ const k = srcKey.get(a + '\u0000' + b) ?? (ordered ? undefined : srcKey.get(b + '\u0000' + a));
336
+ if (k === undefined) throw new Error(`rotationSet has no connection '${a} & ${b}'`);
337
+ return k;
338
+ });
339
+ }
340
+
341
+ const rotation = new Float64Array(nConnections * rotCols);
342
+ connIdx.forEach((k, r) => rotation.set(rot[k], r * rotCols));
343
+ return {
344
+ rotation, rotRows: nConnections, rotCols,
345
+ nodes: codeIdx.map(i => nodes[i]),
346
+ centerVec: center ? Float64Array.from(connIdx, k => center[k]) : null,
347
+ eigenvalues: rs.eigenvalues ? Array.from(rs.eigenvalues, Number) : null,
348
+ columnNames: (rs.columnNames && rs.columnNames.length >= rotCols)
349
+ ? rs.columnNames.slice(0, rotCols).map(String)
350
+ : Array.from({ length: rotCols }, (_, d) => `Dim${d + 1}`),
351
+ };
352
+ }
353
+
275
354
  // ── shared pipeline (post-accumulation) ──────────────────────────────────────
276
355
 
356
+ // norm: { sphereNorm = true, centerAlignToOrigin = true } (= R's norm.by /
357
+ // model(normalize =) and ena.make.set's center.align.to.origin). sphereNorm
358
+ // applies to every model; centerAlignToOrigin to unordered models only --
359
+ // ordered (ONA) models always centre as ona::model() does.
277
360
  function runPipeline(qe, rawNetworks, nUnits, nConnections, codes, unitLabels,
278
361
  metaData, rotMethod, groupA, groupB, dims, gParams,
279
- rowConnectionCounts = null, ordered = false) {
362
+ rowConnectionCounts = null, ordered = false, rotationSet = null,
363
+ norm = {}) {
280
364
  const connectionNames = connectionNamesFor(qe, codes, ordered);
365
+ const useSphereNorm = norm.sphereNorm !== false;
366
+ const alignToOrigin = ordered || norm.centerAlignToOrigin !== false;
281
367
 
282
- // Sphere norm → lineWeights (= R's set$line.weights)
283
- const lineWeights = sphereNorm(qe, rawNetworks, nUnits, nConnections);
368
+ // Sphere norm → lineWeights (= R's set$line.weights). Without it, every
369
+ // network is scaled by the largest network's length (fun_skip_sphere_norm).
370
+ const lineWeights = useSphereNorm
371
+ ? sphereNorm(qe, rawNetworks, nUnits, nConnections)
372
+ : scaleNetworks(qe, rawNetworks, nUnits, nConnections);
284
373
 
285
374
  // Center → pointsForProjection (= R's model$points.for.projection)
286
375
  // centerVec (= R's rotation$center.vec)
287
- // Ordered (ONA) models follow ona::model(): zero-network units are left
288
- // out of the mean but still shifted by it.
376
+ // Zero-network units are left out of the mean and stay at the origin,
377
+ // unless centerAlignToOrigin is false (mean over, and shift of, every
378
+ // unit). Ordered (ONA) models follow ona::model(): zero-network units are
379
+ // left out of the mean but still shifted by it.
289
380
  const { centered: pointsForProjection, centerVec } =
290
- center(qe, lineWeights, nUnits, nConnections, ordered);
381
+ center(qe, lineWeights, nUnits, nConnections, ordered, !alignToOrigin);
382
+
383
+ // Custom rotation (= R's rotation.set): project into another model's
384
+ // space -- its rotation matrix and node positions, and, for unordered
385
+ // models, its centering vector (rENA's center.projection). Ordered models
386
+ // keep their own centering, as rENA.api's ordered pipeline does.
387
+ if (rotationSet) {
388
+ return runCustomRotation(qe, rawNetworks, lineWeights, pointsForProjection,
389
+ centerVec, nUnits, nConnections, codes, unitLabels,
390
+ metaData, dims, rowConnectionCounts, ordered,
391
+ resolveRotationSet(rotationSet, codes, nConnections, ordered),
392
+ alignToOrigin);
393
+ }
291
394
 
292
395
  // Rotate
293
396
  let rot;
@@ -378,6 +481,75 @@ function runPipeline(qe, rawNetworks, nUnits, nConnections, codes, unitLabels,
378
481
  });
379
482
  }
380
483
 
484
+ function runCustomRotation(qe, rawNetworks, lineWeights, ownCentered, ownCenterVec,
485
+ nUnits, nConnections, codes, unitLabels, metaData, dims,
486
+ rowConnectionCounts, ordered, rs, alignToOrigin = true) {
487
+ let pointsForProjection = ownCentered;
488
+ let centerVec = ownCenterVec;
489
+ if (!ordered) {
490
+ // rENA center.projection: subtract the rotation set's center vector
491
+ // (zero-network units stay at the origin unless centerAlignToOrigin
492
+ // is false, as center() leaves them).
493
+ centerVec = rs.centerVec;
494
+ pointsForProjection = new Float64Array(lineWeights.length);
495
+ for (let u = 0; u < nUnits; u++) {
496
+ let rowSum = 0;
497
+ for (let c = 0; c < nConnections; c++) rowSum += Math.abs(lineWeights[u * nConnections + c]);
498
+ if (rowSum === 0 && alignToOrigin) continue;
499
+ for (let c = 0; c < nConnections; c++) {
500
+ pointsForProjection[u * nConnections + c] = lineWeights[u * nConnections + c] - centerVec[c];
501
+ }
502
+ }
503
+ }
504
+
505
+ dims = Math.min(dims, rs.rotCols);
506
+ const rotationMatrix = new Float64Array(rs.rotRows * dims);
507
+ for (let r = 0; r < rs.rotRows; r++)
508
+ for (let d = 0; d < dims; d++)
509
+ rotationMatrix[r * dims + d] = rs.rotation[r * rs.rotCols + d];
510
+
511
+ const points = project(pointsForProjection, nUnits, nConnections,
512
+ rs.rotation, rs.rotRows, rs.rotCols, dims);
513
+
514
+ // Nodes are the rotation set's (R keeps rotation.set$nodes). Centroids:
515
+ // ordered models keep optimize()'s centroids, as rENA.api does; unordered
516
+ // ones place each unit at its normalised node-weight average of the
517
+ // pinned nodes -- libqe's node_positions weights × the rotation set's
518
+ // nodes -- so goodness of fit is measured against the nodes shown.
519
+ const nodes = new Float64Array(codes.length * dims);
520
+ rs.nodes.forEach((row, c) => { for (let d = 0; d < dims; d++) nodes[c * dims + d] = Number(row[d]) || 0; });
521
+ let centroids;
522
+ if (ordered) {
523
+ centroids = nodePositions(qe, lineWeights, nUnits, nConnections, points, dims, true).centroids;
524
+ } else {
525
+ const w = qe.node_positions(lineWeights, nUnits, nConnections, points, nUnits, dims, dims).weights;
526
+ centroids = matmul(Float64Array.from(w.data), nUnits, w.cols, nodes, dims);
527
+ }
528
+
529
+ return new ENAModel({
530
+ connectionCounts: rawNetworks,
531
+ rowConnectionCounts,
532
+ lineWeights,
533
+ points,
534
+ rotationMatrix,
535
+ metaData,
536
+ connectionNames: connectionNamesFor(qe, codes, ordered),
537
+ nUnits,
538
+ nConnections,
539
+ dims,
540
+ centroids,
541
+ variance: computeVariance(points, nUnits, dims),
542
+ unitLabels,
543
+ pointsForProjection,
544
+ nodes,
545
+ columnNames: rs.columnNames.slice(0, dims),
546
+ eigenvalues: rs.eigenvalues,
547
+ centerVec,
548
+ codes,
549
+ adjacencyKey: buildAdjacencyKey(codes, ordered),
550
+ });
551
+ }
552
+
381
553
  // ── main factory ─────────────────────────────────────────────────────────────
382
554
 
383
555
  /**
@@ -407,6 +579,25 @@ export default async function loadENA() {
407
579
  * @param {number[]} [opts.groupB] - Unit indices for means rotation group B
408
580
  * @param {object} [opts.gParams] - Pre-built GMR parameters for 'generalized' rotation
409
581
  * @param {number} [opts.dims=2] - Number of dimensions to return
582
+ * @param {string[]} [opts.unitsUsed] - Unit keys to model (= R's units.used);
583
+ * other units' rows remain as context
584
+ * @param {object} [opts.horizons] - Flexible horizons { by, rules: { value: [cols] } }
585
+ * (see tensor.js buildHorizonContexts); replaces
586
+ * the conversations as the windowing contexts
587
+ * @param {object} [opts.rotationSet] - Project into another model's space (= R's
588
+ * rotation.set): { rotationMatrix, nodes,
589
+ * centerVec, codes?, rotationCols?,
590
+ * eigenvalues?, columnNames? }; overrides
591
+ * opts.rotation
592
+ * @param {boolean} [opts.sphereNorm=true] - Sphere-normalize each network (= R's
593
+ * norm.by = fun_sphere_norm); false scales
594
+ * all networks by the longest one
595
+ * (fun_skip_sphere_norm; for ordered models,
596
+ * ona::model(normalize = skip_sphere_norm)).
597
+ * @param {boolean} [opts.centerAlignToOrigin=true] - Keep zero-network units at the
598
+ * origin and out of the centring mean (= R's
599
+ * center.align.to.origin); false centres on
600
+ * every unit. Unordered only.
410
601
  *
411
602
  * @returns {ENAModel}
412
603
  */
@@ -426,6 +617,11 @@ export default async function loadENA() {
426
617
  dims = 2,
427
618
  codeMask,
428
619
  weightModel,
620
+ unitsUsed,
621
+ horizons,
622
+ rotationSet,
623
+ sphereNorm: useSphereNorm = true,
624
+ centerAlignToOrigin = true,
429
625
  } = opts;
430
626
 
431
627
  if (!codes?.length) throw new Error('opts.codes is required');
@@ -442,14 +638,14 @@ export default async function loadENA() {
442
638
 
443
639
  const { codeMatrix, nRows, nCodes, nUnits, unitLabels,
444
640
  unitOf, convoGroups, metaData } =
445
- parseData(rows, codes, units, conversations);
641
+ parseData(rows, codes, units, conversations, unitsUsed);
446
642
 
447
643
  // Every model accumulates through the tensor path; a plain moving
448
644
  // window is expressed as defaultTensor(window).
449
645
  const { networks: rawNetworks, rowConnectionCounts } = accumulateTensor(
450
646
  qe, rows, codeMatrix, nRows, nCodes, nUnits,
451
647
  unitOf, convoGroups, tensorDef ?? defaultTensor(windowSize),
452
- ordered, binary, weight
648
+ ordered, binary, weight, horizons
453
649
  );
454
650
  const nConnections = ordered ? nCodes * nCodes : qe.choose_two(nCodes);
455
651
 
@@ -457,7 +653,8 @@ export default async function loadENA() {
457
653
 
458
654
  return runPipeline(qe, rawNetworks, nUnits, nConnections, codes,
459
655
  unitLabels, metaData, rotMethod, groupA, groupB,
460
- dims, gParams, rowConnectionCounts, ordered);
656
+ dims, gParams, rowConnectionCounts, ordered,
657
+ rotationSet, { sphereNorm: useSphereNorm, centerAlignToOrigin });
461
658
  },
462
659
 
463
660
  /**
@@ -465,7 +662,8 @@ export default async function loadENA() {
465
662
  * Returns raw (un-normalised) network vectors.
466
663
  *
467
664
  * @param {Object[]} rows
468
- * @param {object} opts - codes, units, conversations, window, binary, ordered, tensor, weightModel
665
+ * @param {object} opts - codes, units, conversations, window, binary, ordered, tensor,
666
+ * weightModel, codeMask, unitsUsed, horizons
469
667
  * @returns {{
470
668
  * connectionCounts: Float64Array,
471
669
  * rowConnectionCounts: Float64Array | null,
@@ -486,16 +684,18 @@ export default async function loadENA() {
486
684
  tensor: tensorDef,
487
685
  codeMask,
488
686
  weightModel,
687
+ unitsUsed,
688
+ horizons,
489
689
  } = opts;
490
690
 
491
691
  const { codeMatrix, nRows, nCodes, nUnits, unitLabels,
492
692
  unitOf, convoGroups, metaData } =
493
- parseData(rows, codes, units, conversations);
693
+ parseData(rows, codes, units, conversations, unitsUsed);
494
694
 
495
695
  const { networks, rowConnectionCounts } = accumulateTensor(
496
696
  qe, rows, codeMatrix, nRows, nCodes, nUnits,
497
697
  unitOf, convoGroups, tensorDef ?? defaultTensor(windowSize),
498
- ordered, binary, weightModelName(weightModel)
698
+ ordered, binary, weightModelName(weightModel), horizons
499
699
  );
500
700
  const nConnections = ordered ? nCodes * nCodes : qe.choose_two(nCodes);
501
701
 
@@ -509,7 +709,7 @@ export default async function loadENA() {
509
709
  // (= R's ENAAccumulation$`_function.call`).
510
710
  _call: { rows, codes, units, conversations,
511
711
  window: windowSize, binary, ordered, tensor: tensorDef,
512
- weightModel },
712
+ weightModel, unitsUsed, horizons },
513
713
  };
514
714
  },
515
715
 
@@ -547,12 +747,13 @@ export default async function loadENA() {
547
747
  'maxSize must be greater than minSize to compare windows.'
548
748
  );
549
749
 
550
- const { rows, codes, units, conversations, binary, ordered, weightModel } = call;
750
+ const { rows, codes, units, conversations, binary, ordered, weightModel, unitsUsed,
751
+ horizons } = call;
551
752
 
552
753
  // Parse once; only the window size changes between iterations.
553
754
  const { codeMatrix, nRows, nCodes, nUnits, unitLabels,
554
755
  unitOf, convoGroups, metaData } =
555
- parseData(rows, codes, units, conversations);
756
+ parseData(rows, codes, units, conversations, unitsUsed);
556
757
  const nConnections = ordered ? nCodes * nCodes : qe.choose_two(nCodes);
557
758
 
558
759
  // 1. Rebuild + fit at each window, collecting unit points.
@@ -562,7 +763,7 @@ export default async function loadENA() {
562
763
  const { networks: raw } = accumulateTensor(
563
764
  qe, rows, codeMatrix, nRows, nCodes, nUnits,
564
765
  unitOf, convoGroups, defaultTensor(w), ordered, binary,
565
- weightModelName(weightModel)
766
+ weightModelName(weightModel), horizons
566
767
  );
567
768
  const model = runPipeline(
568
769
  qe, raw, nUnits, nConnections, codes,
@@ -590,7 +791,7 @@ export default async function loadENA() {
590
791
  // 4. Rebuild the accumulation at the selected window size.
591
792
  return api.accumulate(rows, {
592
793
  codes, units, conversations, window: bestWindow, binary, ordered,
593
- weightModel,
794
+ weightModel, unitsUsed, horizons,
594
795
  });
595
796
  },
596
797
 
@@ -722,7 +923,12 @@ export default async function loadENA() {
722
923
  * @param {Object[]} rows
723
924
  * @param {object} opts - fit() options (codes, units, conversations,
724
925
  * window, binary, ordered, tensor, weightModel,
725
- * codeMask, rotation, groupA, groupB, gParams)
926
+ * codeMask, rotation, groupA, groupB, gParams,
927
+ * unitsUsed, horizons, sphereNorm,
928
+ * centerAlignToOrigin). A rotationSet is
929
+ * ignored: it cannot rotate the reduced
930
+ * models, so every candidate is scored in
931
+ * opts.rotation.
726
932
  * plus removeNum (default 3) and threshold
727
933
  * (default 0.95). `dims` is ignored: scoring
728
934
  * uses dims 1:2 and the variance uses all dims.
@@ -735,7 +941,8 @@ export default async function loadENA() {
735
941
  window: windowSize = 4, binary = true, ordered = false,
736
942
  tensor: tensorDef, weightModel, codeMask,
737
943
  rotation = 'svd', gParams, groupA, groupB,
738
- removeNum = 3, threshold = 0.95,
944
+ removeNum = 3, threshold = 0.95, unitsUsed, horizons,
945
+ sphereNorm = true, centerAlignToOrigin = true,
739
946
  } = opts;
740
947
  if (!codes?.length) throw new Error('opts.codes is required');
741
948
  if (!units?.length) throw new Error('opts.units is required');
@@ -749,11 +956,11 @@ export default async function loadENA() {
749
956
  // Accumulate once, exactly as fit() would for these options.
750
957
  const { codeMatrix, nRows, nCodes, nUnits, unitLabels,
751
958
  unitOf, convoGroups, metaData } =
752
- parseData(rows, codes, units, conversations);
959
+ parseData(rows, codes, units, conversations, unitsUsed);
753
960
  const { networks: raw } = accumulateTensor(
754
961
  qe, rows, codeMatrix, nRows, nCodes, nUnits,
755
962
  unitOf, convoGroups, tensorDef ?? defaultTensor(windowSize),
756
- ordered, binary, weightModelName(weightModel)
963
+ ordered, binary, weightModelName(weightModel), horizons
757
964
  );
758
965
  const nConnections = ordered ? nCodes * nCodes : qe.choose_two(nCodes);
759
966
  applyCodeMask(raw, codeMask, nCodes, nUnits, nConnections, ordered);
@@ -789,7 +996,8 @@ export default async function loadENA() {
789
996
  }
790
997
  return runPipeline(qe, networks, nUnits, nConn, keptCodes,
791
998
  unitLabels, metaData, rotation, groupA, groupB,
792
- nConn, gParams, null, ordered);
999
+ nConn, gParams, null, ordered, null,
1000
+ { sphereNorm, centerAlignToOrigin });
793
1001
  };
794
1002
 
795
1003
  const D = 2; // score on the first 2 dims (R's get_pria_scores_2Ds)
package/src/pipeline.js CHANGED
@@ -127,6 +127,16 @@ export function sphereNorm(qe, networks, nUnits, nConnections) {
127
127
  return new Float64Array(result.data);
128
128
  }
129
129
 
130
+ /**
131
+ * Scale every network by the reciprocal of the largest row L2 norm, keeping
132
+ * relative magnitudes across units (= rENA's fun_skip_sphere_norm, used when
133
+ * sphere normalization is off).
134
+ */
135
+ export function scaleNetworks(qe, networks, nUnits, nConnections) {
136
+ const result = qe.scale_networks(networks, nUnits, nConnections);
137
+ return new Float64Array(result.data);
138
+ }
139
+
130
140
  // ── centering ─────────────────────────────────────────────────────────────────
131
141
 
132
142
  /**
@@ -137,17 +147,21 @@ export function sphereNorm(qe, networks, nUnits, nConnections) {
137
147
  * rows, as rENA's center(exclude_zero_networks = TRUE) does -- the ONA
138
148
  * convention (ona::model). By default zero-network rows stay at zero, as in
139
149
  * rENA's ena.make.set().
150
+ * @param {boolean} [includeZeroRows=false] Take the mean over every row,
151
+ * zero networks included, and shift every row by it: rENA's
152
+ * ena.make.set(center.align.to.origin = FALSE). Overrides shiftZeroRows.
140
153
  * @returns {{ centered: Float64Array, centerVec: Float64Array }}
141
154
  * centered — mean-subtracted networks
142
155
  * centerVec — the column means used for centering (= R's rotation$center.vec)
143
156
  */
144
- export function center(qe, networks, nUnits, nConnections, shiftZeroRows = false) {
145
- // Identify non-zero rows
157
+ export function center(qe, networks, nUnits, nConnections, shiftZeroRows = false,
158
+ includeZeroRows = false) {
159
+ // Identify non-zero rows (every row when zero networks are included)
146
160
  const active = [];
147
161
  for (let u = 0; u < nUnits; u++) {
148
162
  let rowSum = 0;
149
163
  for (let c = 0; c < nConnections; c++) rowSum += Math.abs(networks[u * nConnections + c]);
150
- if (rowSum > 0) active.push(u);
164
+ if (rowSum > 0 || includeZeroRows) active.push(u);
151
165
  }
152
166
 
153
167
  // Compute column means over active rows only (= R's rotation$center.vec)
@@ -164,7 +178,7 @@ export function center(qe, networks, nUnits, nConnections, shiftZeroRows = false
164
178
  const activeSet = new Set(active);
165
179
  const centered = new Float64Array(networks.length);
166
180
  for (let u = 0; u < nUnits; u++) {
167
- if (!shiftZeroRows && !activeSet.has(u)) continue; // leave zero-network row as zero
181
+ if (!shiftZeroRows && !includeZeroRows && !activeSet.has(u)) continue; // leave zero-network row as zero
168
182
  for (let c = 0; c < nConnections; c++) {
169
183
  centered[u * nConnections + c] = networks[u * nConnections + c] - means[c];
170
184
  }
package/src/tensor.js CHANGED
@@ -36,9 +36,25 @@
36
36
  *
37
37
  * // Optional column name for per-row timestamps.
38
38
  * // Defaults to row index (0, 1, 2, ...) — equivalent to unit-step time.
39
+ * // Numeric columns are used as-is; otherwise every value is parsed as
40
+ * // elapsed time (MM:SS, HH:MM:SS) or a date-time, in seconds (parseTimes).
39
41
  * timesCol: 'timestamp',
42
+ *
43
+ * // Optional unit the windows in `data` are expressed in when timesCol is
44
+ * // set: 'secs' (default) | 'mins' | 'hours' | 'days' | 'weeks'. Windows
45
+ * // are converted to seconds before accumulation, as rENA.api does with
46
+ * // mins_to_seconds() etc.
47
+ * timeUnit: 'mins',
48
+ *
49
+ * // Optional end-time column (rENA.api's timeend_column). A numeric column
50
+ * // is a duration added to the start time; anything else is parsed as the
51
+ * // row's end timestamp. Either way the row's time becomes its end time.
52
+ * timesEndCol: 'duration',
40
53
  * }
41
54
  *
55
+ * A window of Infinity (e.g. defaultTensor(Infinity), = rENA's
56
+ * window.size.back = Inf) includes every earlier line of the conversation.
57
+ *
42
58
  * Simple windowed accumulation (IS_DEFAULT path):
43
59
  * Pass `{ dims: [2], dimsSender: [], dimsReceiver: [], dimsMode: [],
44
60
  * factors: [], data: Float64Array.of(weight, window) }`
@@ -75,25 +91,123 @@ export function buildContextLookup(rows, rowIndices, factors, factorLevels) {
75
91
  }
76
92
 
77
93
  /**
78
- * Build the context_lookup with explicit timestamp column.
94
+ * Build the context_lookup with explicit per-row times.
95
+ *
96
+ * @param {Float64Array} allTimes time of every row in `rows` (see parseTimes)
79
97
  */
80
- export function buildContextLookupWithTimes(rows, rowIndices, factors, factorLevels, timesCol) {
81
- const nRows = rowIndices.length;
82
- const nFactor = factors.length;
98
+ export function buildContextLookupWithTimes(rows, rowIndices, factors, factorLevels, allTimes) {
99
+ const ctx = buildContextLookup(rows, rowIndices, factors, factorLevels);
100
+ for (let i = 0; i < rowIndices.length; i++) ctx.times[i] = allTimes[rowIndices[i]];
101
+ return ctx;
102
+ }
83
103
 
84
- const contextLookup = new Int32Array(nRows * nFactor);
85
- const times = new Float64Array(nRows);
104
+ // Seconds per window unit (= rENA.api's mins_to_seconds, hours_to_seconds, ...).
105
+ const SECONDS_PER = { secs: 1, mins: 60, hours: 3600, days: 86400, weeks: 604800 };
86
106
 
87
- for (let i = 0; i < nRows; i++) {
88
- const row = rows[rowIndices[i]];
89
- times[i] = Number(row[timesCol]) || i;
90
- for (let f = 0; f < nFactor; f++) {
91
- const col = factors[f];
92
- contextLookup[i * nFactor + f] = factorLevels[col][row[col]] ?? 0;
107
+ /**
108
+ * Seconds per unit of a time-window unit name ('secs', 'mins', 'hours',
109
+ * 'days', 'weeks'; quotes and a trailing 's' are optional). 'lines' and empty
110
+ * values return null (no time scaling).
111
+ */
112
+ export function secondsPerTimeUnit(unit) {
113
+ if (unit == null) return null;
114
+ let u = String(unit).replace(/['"]/g, '').trim().toLowerCase();
115
+ if (u === '' || u === 'lines' || u === 'auto') return null;
116
+ if (u === 'minutes' || u === 'min') u = 'mins';
117
+ if (u === 'seconds' || u === 'sec') u = 'secs';
118
+ if (!u.endsWith('s')) u += 's';
119
+ if (!(u in SECONDS_PER)) throw new Error(`Unknown time unit '${unit}'`);
120
+ return SECONDS_PER[u];
121
+ }
122
+
123
+ const pad2 = (s) => (s.length === 1 ? '0' + s : s);
124
+
125
+ // Date-time formats rENA.api's parse_date tries, in the same order. Each
126
+ // pattern captures the fields named in `order`; a match is accepted only when
127
+ // the fields form a real date (so 9/17/2013 is rejected as d/m/Y and falls
128
+ // through to m/d/Y, exactly as as.POSIXct returns NA for it in R).
129
+ const DATE_FORMATS = [
130
+ { re: /^(\d{4})-(\d{2})-(\d{2})$/, order: 'Ymd' },
131
+ { re: /^(\d{2})\/(\d{2})\/(\d{4})$/, order: 'dmY' },
132
+ { re: /^(\d{2})\/(\d{2})\/(\d{4})$/, order: 'mdY' },
133
+ { re: /^(\d{4})\/(\d{2})\/(\d{2})$/, order: 'Ymd' },
134
+ { re: /^(\d{2})-(\d{2})-(\d{4})$/, order: 'mdY' },
135
+ { re: /^(\d{2})-(\d{2})-(\d{4})$/, order: 'dmY' },
136
+ { re: /^(\d{4})-(\d{2})-(\d{2}) (\d{2}):(\d{2}):(\d{2})$/, order: 'YmdHMS' },
137
+ { re: /^(\d{2})\/(\d{2})\/(\d{4}) (\d{2}):(\d{2}):(\d{2})$/, order: 'dmYHMS' },
138
+ { re: /^(\d{2})\/(\d{2})\/(\d{4}) (\d{2}):(\d{2}):(\d{2})$/, order: 'mdYHMS' },
139
+ { re: /^(\d{2})\/(\d{2})\/(\d{2}) (\d{2}):(\d{2})$/, order: 'mdyHM' },
140
+ { re: /^(\d{2})\/(\d{2})\/(\d{2}) (\d{2}):(\d{2})$/, order: 'dmyHM' },
141
+ { re: /^(\d{1,2})\/(\d{1,2})\/(\d{4}) (\d{2}):(\d{2})$/, order: 'mdYHM' },
142
+ { re: /^(\d{1,2})\/(\d{1,2})\/(\d{2}) (\d{2}):(\d{2})$/, order: 'mdyHM' },
143
+ ];
144
+
145
+ /**
146
+ * Parse one time value to seconds (= rENA.api's parse_date): elapsed
147
+ * "MM:SS" / "HH:MM:SS" → total seconds; a date-time → seconds since the
148
+ * epoch (UTC). Returns NaN when the value matches no supported format.
149
+ */
150
+ export function parseTimeValue(value) {
151
+ let s = String(value).trim();
152
+ s = s.replace(/ (\d):/g, ' 0$1:')
153
+ .replace(/^(\d)(\/|-)/, '0$1$2')
154
+ .replace(/(\/|-)(\d)(\/|-)/g, (_, a, d, b) => a + '0' + d + b);
155
+
156
+ let m;
157
+ if ((m = /^(\d{1,2}):(\d{2})$/.exec(s))) return (+m[1]) * 60 + (+m[2]);
158
+ if ((m = /^(\d{1,3}):(\d{2}):(\d{2})$/.exec(s))) return (+m[1]) * 3600 + (+m[2]) * 60 + (+m[3]);
159
+
160
+ for (const { re, order } of DATE_FORMATS) {
161
+ if (!(m = re.exec(s))) continue;
162
+ const f = { Y: null, m: 1, d: 1, H: 0, M: 0, S: 0 };
163
+ for (let i = 0; i < order.length; i++) {
164
+ const v = +m[i + 1];
165
+ if (order[i] === 'y') f.Y = v <= 68 ? 2000 + v : 1900 + v; // R's %y pivot
166
+ else f[order[i]] = v;
93
167
  }
168
+ if (f.m < 1 || f.m > 12 || f.d < 1 || f.H > 23 || f.M > 59 || f.S > 61) continue;
169
+ const ms = Date.UTC(f.Y, f.m - 1, f.d, f.H, f.M, f.S);
170
+ if (new Date(ms).getUTCDate() !== f.d) continue; // e.g. 31/02
171
+ return ms / 1000;
94
172
  }
173
+ return NaN;
174
+ }
95
175
 
96
- return { contextLookup, clRows: nRows, clCols: nFactor, times };
176
+ /**
177
+ * Per-row times, in seconds, for a time column (= rENA.api's handling of
178
+ * time_column / timeend_column before tma::accumulate). A column whose values
179
+ * are all numeric is used as-is; otherwise every value is parsed with
180
+ * parseTimeValue(). With `endCol`, a numeric end column is a duration added
181
+ * to the start, and any other end column is parsed as the end timestamp.
182
+ *
183
+ * @param {Object[]} rows
184
+ * @param {string} col
185
+ * @param {string|null} [endCol=null]
186
+ * @returns {Float64Array}
187
+ * @throws when a value cannot be parsed as a time
188
+ */
189
+ export function parseTimes(rows, col, endCol = null) {
190
+ const column = (name) => {
191
+ const vals = rows.map(r => r[name]);
192
+ const blank = (v) => v === undefined || v === null || String(v).trim() === '';
193
+ const nums = vals.map(v => (blank(v) ? NaN : Number(v)));
194
+ const numeric = nums.every((n, i) => !Number.isNaN(n) || blank(vals[i]));
195
+ const out = new Float64Array(rows.length);
196
+ for (let i = 0; i < rows.length; i++) {
197
+ const t = numeric ? nums[i] : parseTimeValue(vals[i]);
198
+ if (Number.isNaN(t)) {
199
+ throw new Error(`Unable to read '${vals[i]}' (row ${i + 1}) in time column '${name}' as a time`);
200
+ }
201
+ out[i] = t;
202
+ }
203
+ return { out, numeric };
204
+ };
205
+
206
+ const start = column(col).out;
207
+ if (!endCol) return start;
208
+ const end = column(endCol);
209
+ if (end.numeric) for (let i = 0; i < start.length; i++) end.out[i] += start[i];
210
+ return end.out;
97
211
  }
98
212
 
99
213
  /**
@@ -119,6 +233,79 @@ export function inferFactorLevels(rows, factors) {
119
233
  return levels;
120
234
  }
121
235
 
236
+ // libqe's apply_tensor_unit reads the IS_DEFAULT window as an int, so an
237
+ // infinite window (or one too large for int32) is clamped to INT32_MAX — still
238
+ // longer than any conversation, in lines or in seconds.
239
+ const MAX_WINDOW = 2147483647;
240
+
241
+ /**
242
+ * The tensor data libqe accumulates with: windows (the second half of the
243
+ * column-major data, index 1 along the last axis) converted from `timeUnit`
244
+ * to seconds when a time column is in use, and infinite / oversized windows
245
+ * clamped to MAX_WINDOW. Weights are left unchanged.
246
+ */
247
+ export function resolveTensorData(tensorDef) {
248
+ const data = Float64Array.from(tensorDef.data);
249
+ const nComb = data.length / 2;
250
+ const scale = tensorDef.timesCol ? (secondsPerTimeUnit(tensorDef.timeUnit) ?? 1) : 1;
251
+ for (let c = nComb; c < data.length; c++) {
252
+ const w = data[c] * scale;
253
+ data[c] = (Number.isNaN(w) || w > MAX_WINDOW) ? MAX_WINDOW : w;
254
+ }
255
+ return data;
256
+ }
257
+
258
+ /**
259
+ * Per-unit horizon-of-observation contexts (= tma::contexts with rENA.api's
260
+ * flexible-horizon rules). `horizons.rules` maps each value of the
261
+ * discriminator column `horizons.by` to the columns that define that
262
+ * horizon. A row with discriminator value x is in unit U's context when every
263
+ * rule-x column holds a value U has on at least one of its rows
264
+ * (`col %in% UNIT$col`); rows whose value has no rule are in no context. Each
265
+ * unit's context is then split on the union of every rule's columns (NOT on
266
+ * the discriminator, which only picks the rule), and the moving window runs
267
+ * within each piece.
268
+ *
269
+ * @param {Object[]} rows
270
+ * @param {Int32Array} unitOf unit index per row (-1 = not a modeled unit)
271
+ * @param {number} nUnits
272
+ * @param {{ by: string, rules: Object<string, string[]> }} horizons
273
+ * @returns {number[][][]} per unit, its context pieces as ascending row indices
274
+ */
275
+ export function buildHorizonContexts(rows, unitOf, nUnits, horizons) {
276
+ const { by, rules } = horizons || {};
277
+ if (!by) throw new Error('horizons.by (the discriminator column) is required');
278
+ const ruleCols = new Map();
279
+ for (const [value, cols] of Object.entries(rules || {})) {
280
+ if (!Array.isArray(cols) || cols.length === 0) {
281
+ throw new Error(`horizon '${value}' of '${by}' has no columns assigned`);
282
+ }
283
+ ruleCols.set(String(value), cols);
284
+ }
285
+ if (ruleCols.size === 0) throw new Error('horizons.rules must define at least one horizon');
286
+ const splitCols = [...new Set([].concat(...ruleCols.values()))];
287
+
288
+ // UNIT$col: every value each unit takes on each rule column.
289
+ const unitVals = Array.from({ length: nUnits }, () => new Map(splitCols.map(c => [c, new Set()])));
290
+ for (let i = 0; i < rows.length; i++) {
291
+ const u = unitOf[i];
292
+ if (u < 0) continue;
293
+ for (const c of splitCols) unitVals[u].get(c).add(String(rows[i][c]));
294
+ }
295
+
296
+ return unitVals.map((vals) => {
297
+ const pieces = new Map(); // split key → row indices (first-appearance order)
298
+ for (let i = 0; i < rows.length; i++) {
299
+ const cols = ruleCols.get(String(rows[i][by]));
300
+ if (!cols || !cols.every(c => vals.get(c).has(String(rows[i][c])))) continue;
301
+ const key = splitCols.map(c => String(rows[i][c])).join('\u0000');
302
+ if (!pieces.has(key)) pieces.set(key, []);
303
+ pieces.get(key).push(i);
304
+ }
305
+ return [...pieces.values()];
306
+ });
307
+ }
308
+
122
309
  /**
123
310
  * Accumulate tensor networks for all units across all conversations.
124
311
  *
@@ -141,6 +328,9 @@ export function inferFactorLevels(rows, factors) {
141
328
  * @param {string|null} [weight=null] libqe weight model applied per line before
142
329
  * the unit sum: 'binary' | 'product' |
143
330
  * 'sqrt' | 'log1p'
331
+ * @param {object|null} [horizons=null] flexible horizons (see
332
+ * buildHorizonContexts); replaces
333
+ * convoGroups as the windowing contexts
144
334
  *
145
335
  * @returns {{ networks: Float64Array, rowConnectionCounts: Float64Array }}
146
336
  * networks nUnits × nConnections, row-major
@@ -150,19 +340,21 @@ export function inferFactorLevels(rows, factors) {
150
340
  */
151
341
  export function accumulateTensor(qe, rows, codeMatrix, nRows, nCodes, nUnits,
152
342
  unitOf, convoGroups, tensorDef, ordered = false,
153
- binary = true, weight = null) {
343
+ binary = true, weight = null, horizons = null) {
154
344
  const {
155
345
  dims,
156
346
  dimsSender = [],
157
347
  dimsReceiver = [],
158
348
  dimsMode = [],
159
349
  factors = [],
160
- data: tensorData,
161
350
  timesCol,
351
+ timesEndCol,
162
352
  } = tensorDef;
353
+ const tensorData = resolveTensorData(tensorDef);
163
354
 
164
355
  // Resolve factor levels (infer if not provided)
165
356
  const factorLevels = tensorDef.factorLevels ?? inferFactorLevels(rows, factors);
357
+ const allTimes = timesCol ? parseTimes(rows, timesCol, timesEndCol || null) : null;
166
358
 
167
359
  const nConnections = ordered
168
360
  ? nCodes * nCodes
@@ -179,71 +371,91 @@ export function accumulateTensor(qe, rows, codeMatrix, nRows, nCodes, nUnits,
179
371
  const receiverArr = new Int32Array(dimsReceiver);
180
372
  const modeArr = new Int32Array(dimsMode);
181
373
 
182
- for (const [, rowIndices] of convoGroups) {
183
- const nConvo = rowIndices.length;
374
+ // Accumulate `unit`'s response rows (`localRows`, indices into
375
+ // `rowIndices`) within one context of rows.
376
+ const accumulateContext = (rowIndices, unit, localRows) => {
377
+ const nCtx = rowIndices.length;
184
378
 
185
- // Extract code rows for this conversation
186
- const convoCodes = new Float64Array(nConvo * nCodes);
187
- for (let r = 0; r < nConvo; r++) {
379
+ // Extract code rows for this context
380
+ const ctxCodes = new Float64Array(nCtx * nCodes);
381
+ for (let r = 0; r < nCtx; r++) {
188
382
  const src = rowIndices[r];
189
- convoCodes.set(
383
+ ctxCodes.set(
190
384
  codeMatrix.subarray(src * nCodes, src * nCodes + nCodes),
191
385
  r * nCodes
192
386
  );
193
387
  }
194
388
 
195
- // Build context_lookup and times for this conversation
196
- const { contextLookup, clRows, clCols, times } = timesCol
197
- ? buildContextLookupWithTimes(rows, rowIndices, factors, factorLevels, timesCol)
389
+ // Build context_lookup and times for this context
390
+ const { contextLookup, clRows, clCols, times } = allTimes
391
+ ? buildContextLookupWithTimes(rows, rowIndices, factors, factorLevels, allTimes)
198
392
  : buildContextLookup(rows, rowIndices, factors, factorLevels);
199
393
 
200
- // Group response rows by unit (within this conversation)
394
+ // Always accumulate the DIRECTED per-response-row counts (ordered
395
+ // kernel), exactly as tma does — it calls apply_tensor with the
396
+ // default ordered=TRUE and defers the ordered-vs-unordered decision
397
+ // to aggregation. Passing the unordered flag here would make the
398
+ // kernel emit an already-symmetric matrix that the fold below would
399
+ // then double-count.
400
+ const result = qe.accumulate_tensor_unit(
401
+ tensorData, dimsArr,
402
+ senderArr, receiverArr, modeArr,
403
+ contextLookup, clRows, clCols,
404
+ new Int32Array(localRows),
405
+ ctxCodes, nCtx, nCodes,
406
+ times,
407
+ true
408
+ );
409
+
410
+ // Finalise each raw per-response-row connection vector with libqe's
411
+ // shared kernel, exactly as tma does in R: fold to the upper
412
+ // triangle (unordered) or keep the directed row (ordered), then
413
+ // apply the weight model per line (binary clamp, product, sqrt,
414
+ // log1p) before the per-unit sum. Row i is local row localRows[i].
415
+ const rcc = result.row_connection_counts;
416
+ const fin = qe.finalize_row_connections(
417
+ rcc.data, rcc.rows, rcc.cols, nCodes, ordered, kernelWeight
418
+ );
419
+ const unitOff = unit * nConnections;
420
+ for (let i = 0; i < fin.rows; i++) {
421
+ const rowOff = rowIndices[localRows[i]] * nConnections;
422
+ const finOff = i * fin.cols;
423
+ for (let c = 0; c < nConnections; c++) {
424
+ const v = fin.data[finOff + c];
425
+ rowConnectionCounts[rowOff + c] = v;
426
+ networks[unitOff + c] += v;
427
+ }
428
+ }
429
+ };
430
+
431
+ if (horizons) {
432
+ // Flexible horizons: every unit has its own contexts.
433
+ const unitContexts = buildHorizonContexts(rows, unitOf, nUnits, horizons);
434
+ unitContexts.forEach((pieces, unit) => {
435
+ for (const rowIndices of pieces) {
436
+ const localRows = [];
437
+ for (let r = 0; r < rowIndices.length; r++) {
438
+ if (unitOf[rowIndices[r]] === unit) localRows.push(r);
439
+ }
440
+ if (localRows.length) accumulateContext(rowIndices, unit, localRows);
441
+ }
442
+ });
443
+ return { networks, rowConnectionCounts };
444
+ }
445
+
446
+ for (const [, rowIndices] of convoGroups) {
447
+ // Group response rows by unit (within this conversation). Rows with
448
+ // no unit (unitOf = -1, excluded via unitsUsed) are never responses,
449
+ // but stay in the context for the other units' windows.
201
450
  const unitConvoRows = new Map();
202
- for (let r = 0; r < nConvo; r++) {
451
+ for (let r = 0; r < rowIndices.length; r++) {
203
452
  const unit = unitOf[rowIndices[r]];
453
+ if (unit < 0) continue;
204
454
  if (!unitConvoRows.has(unit)) unitConvoRows.set(unit, []);
205
455
  unitConvoRows.get(unit).push(r); // local (conversation-relative) index
206
456
  }
207
-
208
- // Accumulate per unit
209
457
  for (const [unit, localRows] of unitConvoRows) {
210
- const unitRowsArr = new Int32Array(localRows);
211
-
212
- // Always accumulate the DIRECTED per-response-row counts (ordered
213
- // kernel), exactly as tma does — it calls apply_tensor with the
214
- // default ordered=TRUE and defers the ordered-vs-unordered decision
215
- // to aggregation. Passing the unordered flag here would make the
216
- // kernel emit an already-symmetric matrix that the fold below would
217
- // then double-count.
218
- const result = qe.accumulate_tensor_unit(
219
- tensorData, dimsArr,
220
- senderArr, receiverArr, modeArr,
221
- contextLookup, clRows, clCols,
222
- unitRowsArr,
223
- convoCodes, nConvo, nCodes,
224
- times,
225
- true
226
- );
227
-
228
- // Finalise each raw per-response-row connection vector with libqe's
229
- // shared kernel, exactly as tma does in R: fold to the upper
230
- // triangle (unordered) or keep the directed row (ordered), then
231
- // apply the weight model per line (binary clamp, product, sqrt,
232
- // log1p) before the per-unit sum. Row i is local row localRows[i].
233
- const rcc = result.row_connection_counts;
234
- const fin = qe.finalize_row_connections(
235
- rcc.data, rcc.rows, rcc.cols, nCodes, ordered, kernelWeight
236
- );
237
- const unitOff = unit * nConnections;
238
- for (let i = 0; i < fin.rows; i++) {
239
- const rowOff = rowIndices[localRows[i]] * nConnections;
240
- const finOff = i * fin.cols;
241
- for (let c = 0; c < nConnections; c++) {
242
- const v = fin.data[finOff + c];
243
- rowConnectionCounts[rowOff + c] = v;
244
- networks[unitOff + c] += v;
245
- }
246
- }
458
+ accumulateContext(rowIndices, unit, localRows);
247
459
  }
248
460
  }
249
461