@web-ts-toolkit/json-frame 0.42.2 → 0.47.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +117 -6
- package/index.d.mts +130 -12
- package/index.d.ts +130 -12
- package/index.js +500 -201
- package/index.mjs +500 -201
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -18,7 +18,7 @@ import { fromOrient } from '@web-ts-toolkit/json-frame';
|
|
|
18
18
|
|
|
19
19
|
The package exports named values and types from the package root. There is no default export and no supported deep import path.
|
|
20
20
|
|
|
21
|
-
`JSON_FRAME_MAX_DEPTH` is `1000`. JSON arrays and objects are counted from the
|
|
21
|
+
`JSON_FRAME_MAX_DEPTH` is `1000`. JSON arrays and objects are counted from the validated root at depth `0`; an array or object reached at depth `1000` is accepted, and one reached at depth `1001` fails with `JsonFrameValidationError` before package traversal can exhaust the JavaScript stack. For `toJSONString()` the root is the complete exported payload including orient-specific wrappers: `split` and `table` nest cell values one level deeper than `records`, `values`, `index`, and `columns`. A nested cell accepted at ingestion may therefore be rejected when serialized in a deeper output layout.
|
|
22
22
|
|
|
23
23
|
## Quick Start
|
|
24
24
|
|
|
@@ -46,6 +46,48 @@ The returned `DataFrame` is immutable. `filter`, `sort`, `select`, `rename`, and
|
|
|
46
46
|
|
|
47
47
|
The `TRow` generic is a compile-time row model only. It improves `row()`, `rows()`, `filter()`, and `sort()` types for JSON-compatible domain interfaces, but it does not validate an application schema at runtime; payload validation follows the selected orient contract.
|
|
48
48
|
|
|
49
|
+
### Inferred Row Types
|
|
50
|
+
|
|
51
|
+
Parsed `records` with omitted, `auto`, or explicit `records` orient infer known required fields precisely. Sparse, heterogeneous, and optional fields include `null`, matching missing-cell normalization. Those fields also remain optional: if no input row supplies a column, it is never created and reading it returns `undefined`. Use `!= null` to guard both. Heterogeneous row variants are flattened into one column shape; a discriminant does not prove another cell is present. Nested cell objects retain their own types and are not null-filled internally.
|
|
52
|
+
|
|
53
|
+
Requiredness follows declared keys in every record variant, including names such as `toString`, `constructor`, and `valueOf`; inherited Object members do not supply cells. Numeric literal keys and their equivalent string spellings are combined before inferring cell types and requiredness: `1` and `'1'` describe one column named `'1'`, accessible with either `[1]` or `['1']`. Non-equivalent strings such as `'01'` remain distinct. Dense literals retain precise value unions; sparse or optional aliases require the same null-safe guards as other columns.
|
|
54
|
+
|
|
55
|
+
```ts
|
|
56
|
+
import { fromOrient } from '@web-ts-toolkit/json-frame';
|
|
57
|
+
|
|
58
|
+
const dense = fromOrient([{ n: 1 }, { n: 2 }]);
|
|
59
|
+
const formatted: string = dense.row(0).n.toFixed(); // number, no extra null guard
|
|
60
|
+
|
|
61
|
+
const sparse = fromOrient([{ n: 1 }, {}]);
|
|
62
|
+
const present = sparse.filter((row) => row.n != null && row.n.toFixed() === '1');
|
|
63
|
+
// sparse.row(1).n is null at runtime; an undefined-only guard is insufficient.
|
|
64
|
+
|
|
65
|
+
const members = fromOrient([{ toString: 'ok' }, {}] as const);
|
|
66
|
+
const text = members.row(1).toString?.toUpperCase(); // undefined: the cell is null
|
|
67
|
+
const numeric = fromOrient([{ 1: 1 }, { '1': 'x' }] as const);
|
|
68
|
+
const cell: 1 | 'x' = numeric.row(0)[1]; // same column as numeric.row(0)['1']
|
|
69
|
+
|
|
70
|
+
const values = fromOrient([[1]], { columns: ['n'] });
|
|
71
|
+
// Values rows are objects, including with readonly tuple inputs and auto orient.
|
|
72
|
+
const n = values.row(0).n;
|
|
73
|
+
const valueText = typeof n === 'number' ? n.toFixed() : '';
|
|
74
|
+
|
|
75
|
+
void [formatted, present, text, cell, valueText];
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
Values arrays, JSON strings, other layouts, broad dictionary records, and broadly typed orient options use conservative `JsonRow` inference rather than guessing column types. Supplying `fromOrient<MyRow>(...)` intentionally preserves your asserted domain model, including optional properties; ensure it describes actual normalized rows. It does not perform runtime application-schema validation.
|
|
79
|
+
|
|
80
|
+
Open index domains keep that fallback, including `string`, `number`, and template patterns such as `` `metric_${string}` ``, `` `${number}` ``, or `` `${string}_metric` ``. Unions or intersections with known properties also fall back to `JsonRow` when an open index domain is present. A dictionary describes allowed names, not an infinite set of present columns: missing cells in existing columns become `null`, while columns absent from every row are not created. Use a scalar guard such as `typeof cell === 'number'` before numeric operations; an undefined-only guard is insufficient. The fallback conservatively widens even known fields to JSON values. Finite template-key unions such as `` `metric_${'a' | 'b'}` `` retain precise fields and declared optionality.
|
|
81
|
+
|
|
82
|
+
```ts
|
|
83
|
+
import { fromOrient } from '@web-ts-toolkit/json-frame';
|
|
84
|
+
|
|
85
|
+
const rows: Record<`metric_${string}`, number>[] = [{ metric_n: 1 }, {}];
|
|
86
|
+
const frame = fromOrient(rows); // DataFrame<JsonRow>
|
|
87
|
+
frame.filter((row) => typeof row.metric_n === 'number' && row.metric_n.toFixed() === '1');
|
|
88
|
+
// frame.row(1).metric_n is null; frame.row(0).metric_absent is undefined.
|
|
89
|
+
```
|
|
90
|
+
|
|
49
91
|
## Supported Orients
|
|
50
92
|
|
|
51
93
|
All six pandas DataFrame JSON orients are supported.
|
|
@@ -140,6 +182,7 @@ Other `fromOrient` options:
|
|
|
140
182
|
- `columns`: required whenever the payload is `values`, including non-empty `values` arrays detected by `auto`
|
|
141
183
|
- `columnTypes`: explicit logical types for non-table inputs
|
|
142
184
|
- `packThreshold`: minimum length for internal numeric typed-array packing; `0` disables packing
|
|
185
|
+
- `maxNodes`: optional positive safe integer limiting ingestion traversal occurrences; omitted means no quota (see below)
|
|
143
186
|
|
|
144
187
|
```ts
|
|
145
188
|
import { AmbiguousOrientError, JsonFrameOptionError, fromOrient } from '@web-ts-toolkit/json-frame';
|
|
@@ -246,6 +289,32 @@ Transform behavior:
|
|
|
246
289
|
|
|
247
290
|
Structural immutability is shallow. Frame-owned arrays, row records, exporter containers, table schema records, and internal maps are protected from direct mutation or are freshly allocated. Nested JSON object or array cell values are not deep-frozen or deep-cloned on every read/export; if caller code mutates one of those nested values after obtaining it from `row()`, `rows()`, or an exporter, another read of the same cell may observe that mutation.
|
|
248
291
|
|
|
292
|
+
`toJSONString(orient, options?)` validates the complete selected output with the shared bounded traversal before native serialization. Cycles, over-depth containers, sparse arrays, and newly introduced non-JSON values (`undefined`, `bigint`, `symbol`, `function`, non-finite numbers, non-plain objects) fail with path-bearing `JsonFrameValidationError` carrying the selected orient instead of native `TypeError`/`RangeError` behavior. Valid nested JSON serializes unchanged and preserves shallow cell identity on subsequent reads.
|
|
293
|
+
|
|
294
|
+
Validation is one traversal pass without cloning the payload, followed by native `JSON.stringify()`. Only the ancestor chain is tracked, so repeated references to the same acyclic container are visited once per occurrence and expand as stringification expands them. Optional `maxNodes` bounds validation visits; omission leaves breadth unlimited. Validation reads own enumerable properties and array elements, which invokes caller-installed getters and `Proxy` traps; native serialization may additionally invoke `toJSON` hooks, including non-enumerable ones invisible to validation. Hooks can change values between passes and are caller responsibility; the package does not sandbox arbitrary JavaScript.
|
|
295
|
+
|
|
296
|
+
### Opt-In Traversal Budgets
|
|
297
|
+
|
|
298
|
+
`fromOrient(input, { maxNodes })` and `frame.toJSONString(orient, { maxNodes })` accept a **positive safe integer**. Zero, negatives, fractions, non-finite/unsafe integers, `null`, and other nonnumbers throw `JsonFrameOptionError`. Options are validated for all six serialization orients; `indexField` must be a string when supplied and affects only table output. It can be combined with `maxNodes`.
|
|
299
|
+
|
|
300
|
+
Each traversal counts its root as one node, then every property/array-element **value occurrence**, including scalars, `null`, and empty containers. Property names are not nodes. Repeated aliases count again on every path; ingestion still creates detached clones for each occurrence. Exact-budget traversal succeeds; attempting the next node throws path-bearing `JsonFrameValidationError` with the orient when known (auto ingestion detects orient after cloning). Serialization counts the **complete exported payload**, including orient wrappers, labels and table metadata, so different orients need different budgets. Counters reset per call. An ingestion budget is **not retained by the frame** or inherited by later exports or transforms; omitted options impose no default quota.
|
|
301
|
+
|
|
302
|
+
```ts
|
|
303
|
+
import { fromOrient } from '@web-ts-toolkit/json-frame';
|
|
304
|
+
|
|
305
|
+
// Three nodes: root array, row object, number. JSON strings count after JSON.parse.
|
|
306
|
+
const frame = fromOrient('[{"n":1}]', { orient: 'records', maxNodes: 3 });
|
|
307
|
+
const records = frame.toJSONString('records', { maxNodes: 3 });
|
|
308
|
+
// Split adds wrappers and labels: eight nodes for this one-cell frame.
|
|
309
|
+
const split = frame.toJSONString('split', { maxNodes: 8 });
|
|
310
|
+
const indexed = fromOrient({ r0: { n: 1 } }, { orient: 'index', maxNodes: 3 });
|
|
311
|
+
const table = indexed.toJSONString('table', { indexField: 'row_id', maxNodes: 15 });
|
|
312
|
+
|
|
313
|
+
void [records, split, table];
|
|
314
|
+
```
|
|
315
|
+
|
|
316
|
+
This is a traversal-expansion limit, **not a total resource limit**. It does not bound input/string bytes or the preceding native `JSON.parse()`, object-key enumeration/allocation (including diagnostic key counting), rectangular frame densification, export payload/schema construction before validation, native stringification/output bytes, caller hooks, or total process memory. Scalars may be arbitrarily large. Validation allocates traversal bookkeeping but no redundant full clone. The fixed depth/cycle/JSON-compatibility checks still apply independently. Object-returning exporters such as `toTable()` have no `maxNodes` option. Applications still need their own size and trust boundaries.
|
|
317
|
+
|
|
249
318
|
```ts
|
|
250
319
|
import { fromOrient } from '@web-ts-toolkit/json-frame';
|
|
251
320
|
|
|
@@ -282,14 +351,17 @@ Type rules:
|
|
|
282
351
|
- Table Schema metadata is preserved when `orient: 'table'` is used; pandas-authored field metadata remains caller/pandas responsibility.
|
|
283
352
|
- Other orients infer the narrowest logical type by scanning the full column.
|
|
284
353
|
- `options.columnTypes` supplies explicit logical types for non-table inputs. Declared types are validated against every non-null cell before packing; incompatible cells fail with `JsonFrameValidationError` carrying `path`, `row`, `column`, and `value` diagnostics.
|
|
354
|
+
- Only own enumerable string keys in `columnTypes` are overrides; inherited properties are ignored. Ordinary and null-prototype dictionaries are supported and copied without mutating the caller. Names such as `constructor`, `toString`, `hasOwnProperty`, and `__proto__` remain valid columns with omitted, empty, partial, or explicit overrides. Use a computed key for an explicit `__proto__` override, for example `{ ['__proto__']: 'float' }`. Source Table Schema field types take precedence over overrides.
|
|
285
355
|
- Explicit `integer`, `float`, `string`, and `boolean` require matching JSON scalar cells. `float` accepts integer JSON numbers because Table Schema `number` accepts both integer and fractional JSON numbers.
|
|
286
|
-
- Explicit `datetime` accepts
|
|
356
|
+
- Explicit `datetime` accepts calendar-valid, timezone-naive ISO strings for four-digit years **0000–9999**, using proleptic Gregorian rules: leap years are divisible by 4 except centuries not divisible by 400. Thus `0000-02-29` and `2000-02-29` are valid, while `0100-02-29` and `1900-02-29` are not. The grammar is `YYYY-MM-DD`, optionally followed by `T` or one space and `HH:mm:ss`, optionally followed by a decimal point and 1–9 fractional-second digits. Hours must be 00–23 and minutes/seconds 00–59; timezone suffixes, leap seconds, signed/expanded years, and numeric epochs are rejected. Strings and their fractional precision are preserved exactly. Numeric epochs have no unit metadata in generated Table Schema and pandas reads numeric `datetime` cells as nanoseconds.
|
|
287
357
|
- Explicit `categorical` accepts non-null scalar JSON cells (`string`, `number`, or `boolean`) and generated table output emits `type: 'any'` with `extDtype: 'category'` unless source Table Schema field metadata is being preserved.
|
|
288
358
|
- Explicit `mixed` and `unknown` accept any JSON-compatible cell value.
|
|
289
359
|
- `null` marks a column as nullable without forcing an otherwise numeric column to become `mixed`.
|
|
290
360
|
- ISO-looking strings are preserved as strings unless table metadata or compatible explicit `columnTypes` says otherwise.
|
|
291
361
|
- Values are never coerced to satisfy logical type metadata.
|
|
292
362
|
|
|
363
|
+
Calendar validity does **not** guarantee pandas read-back across the full year range. Pandas dtype and time resolution impose separate limits. The prior JFB-07 investigation (pandas **3.0.3**, CPython **3.14.6**) recorded ISO emission of years `0000`, `0001`, and `0099` at microsecond resolution, but `read_json(..., orient='table')` failed outside nanosecond bounds. This is previously recorded evidence, not a new pandas experiment for the calendar fix; see [JFB-07 evidence](https://github.com/egose/web-ts-toolkit/blob/HEAD/docs/tasks/20260908-180959-json-frame-boundary-health-follow-up.md#task-jfb-07-resolve-datetime-year-range-semantics). Check the target pandas dtype/resolution when exchanging dates outside its read-back range.
|
|
364
|
+
|
|
293
365
|
Packing rules:
|
|
294
366
|
|
|
295
367
|
- typed-array packing is an internal optimization only
|
|
@@ -299,9 +371,9 @@ Packing rules:
|
|
|
299
371
|
|
|
300
372
|
## Round Trips And Index Provenance
|
|
301
373
|
|
|
302
|
-
Round trips are semantic rather than byte-for-byte.
|
|
374
|
+
Round trips are semantic rather than byte-for-byte, with the empty-dimension exceptions below.
|
|
303
375
|
|
|
304
|
-
- column order
|
|
376
|
+
- column labels/order are retained where the layout carries them, subject to JavaScript enumeration of object keys; cell JSON values are preserved apart from native JSON text formatting (for example, `JSON.stringify(-0)` produces `0`)
|
|
305
377
|
- `split` and `table` preserve row order exactly
|
|
306
378
|
- `index` and `columns` row order follows JavaScript property enumeration for object keys; use `split` or `table` when exact order matters for integer-like labels
|
|
307
379
|
- label-bearing orients preserve supported source index labels, subject to object-key stringification for `index`/`columns`
|
|
@@ -311,19 +383,58 @@ Round trips are semantic rather than byte-for-byte.
|
|
|
311
383
|
- `toTable()` emits valid Table Schema, omits a synthetic index from table output, and rejects duplicate source index labels because emitted table primary keys must be unique
|
|
312
384
|
- table primary-key equality uses JavaScript `Map`/SameValueZero semantics for supported string and finite-number labels, so numeric `1` and string `'1'` are distinct table labels even though object-key exporters reject them as stringification collisions
|
|
313
385
|
- Table Schema metadata is cloned under the same `JSON_FRAME_MAX_DEPTH` policy used while parsing. `toTable()` and `toJSONString('table')` report over-depth metadata as `JsonFrameValidationError`.
|
|
386
|
+
- `toJSONString()` depth is measured from the exported payload root including orient wrappers, not from the stored cell alone. A cell accepted at ingestion at the depth limit can be rejected in a deeper `split`/`table` layout while still serializing in a shallower `records`/`values`/`index`/`columns` layout.
|
|
387
|
+
|
|
388
|
+
Source table index and data-field metadata retain their separate roles through `select`, `rename`, `resetIndex`, filtering and sorting, even when the source primary-key field is not first. Renaming a data column to the retained index-field name is supported, but table export then needs a unique `indexField` override (or a subsequent rename/resetIndex) to avoid duplicate emitted field names.
|
|
389
|
+
|
|
390
|
+
### Empty-Dimension Fidelity
|
|
391
|
+
|
|
392
|
+
Choose **`split` or `table`** when an empty report must carry its dimensions in the payload. The matrix gives the shape **after export and re-ingestion with the matching explicit orient**, for both object-returning exporters and `toJSONString()`. `R` is a positive row count and `C` a positive data-column count; the index field is not a data column. `values` results assume the caller separately supplies the original `columns`.
|
|
393
|
+
|
|
394
|
+
| Orient | Zero rows, declared columns (`0 × C`) | Rows, zero columns (`R × 0`) | Fully empty (`0 × 0`) |
|
|
395
|
+
| --------- | ------------------------------------------------------ | ---------------------------------------------------------------- | ---------------------------------------- |
|
|
396
|
+
| `records` | `[]` → `0 × 0`; column labels lost | `R` empty objects → `R × 0` | `[]` → `0 × 0` |
|
|
397
|
+
| `index` | `{}` → `0 × 0`; column labels lost | `R` index keys with empty objects → `R × 0` | `{}` → `0 × 0` |
|
|
398
|
+
| `columns` | `C` column keys with empty objects → `0 × C` | `{}` → `0 × 0`; rows and index labels lost | `{}` → `0 × 0` |
|
|
399
|
+
| `values` | `[]` + supplied columns → `0 × C` | `R` empty arrays + `columns: []` → `R × 0` | `[]` + `columns: []` → `0 × 0` |
|
|
400
|
+
| `split` | `columns` retained, empty `index`/`data` → `0 × C` | Empty `columns`, `R` index labels and empty row arrays → `R × 0` | Empty `columns`/`index`/`data` → `0 × 0` |
|
|
401
|
+
| `table` | Data fields retained in schema, empty `data` → `0 × C` | No data fields, `R` row objects → `R × 0` | No data fields, empty `data` → `0 × 0` |
|
|
402
|
+
|
|
403
|
+
- `values` never carries column labels; omitting `options.columns` throws even for `[]` or `[[], []]`. For empty arrays, also pass the explicit orient. `options.columns` is used only for `values`; it cannot restore lost columns in empty `records`/`index` payloads.
|
|
404
|
+
- `records`/`values` regenerate a synthetic numeric index. `index`/`columns` re-ingest object keys as source string labels with JavaScript enumeration order. `columns` has nowhere to put row labels when there are no columns.
|
|
405
|
+
- `split` always emits the current index array and re-ingests it as a **source** index, even if those labels were synthetic before export. It preserves gapped numeric labels after filtering.
|
|
406
|
+
- `table` emits a source index as a schema field plus `primaryKey`, including on a fully empty frame. With zero data columns, its rows contain only that index field. A synthetic index is omitted: zero-column rows are `{}` and re-ingestion regenerates `0..R-1`, so gaps in a filtered synthetic index are lost. The normal uniqueness and index-field collision checks still apply.
|
|
407
|
+
- Dimension fidelity is separate from metadata fidelity: only `table` carries logical types and retained field/schema metadata. For example, a zero-row numeric column round-tripped through `split`, `columns`, or `values` has no cells from which to infer its original type; supply appropriate non-table `columnTypes` if needed.
|
|
408
|
+
|
|
409
|
+
```ts
|
|
410
|
+
import { fromOrient } from '@web-ts-toolkit/json-frame';
|
|
411
|
+
|
|
412
|
+
const report = fromOrient([{ a: 1 }, { a: 2 }], { orient: 'records' });
|
|
413
|
+
const noRows = report.filter(() => false); // 0 rows, column a retained
|
|
414
|
+
const noColumns = report.select(); // 2 rows, 0 columns
|
|
415
|
+
|
|
416
|
+
const shaped = fromOrient(noRows.toSplit(), { orient: 'split' }); // 0 × 1
|
|
417
|
+
const rowCount = fromOrient(noColumns.toTable(), { orient: 'table' }); // 2 × 0
|
|
418
|
+
const lostRows = fromOrient(noColumns.toColumns(), { orient: 'columns' }); // 0 × 0
|
|
419
|
+
const restoredColumns = fromOrient(noRows.toValues(), { orient: 'values', columns: noRows.columns });
|
|
420
|
+
|
|
421
|
+
void [shaped, rowCount, lostRows, restoredColumns];
|
|
422
|
+
```
|
|
314
423
|
|
|
315
424
|
## Errors
|
|
316
425
|
|
|
317
426
|
Structured runtime errors are exported from the package root.
|
|
318
427
|
|
|
319
428
|
- `JsonFrameParseError`: invalid JSON string input; original `SyntaxError` is available as `cause`
|
|
320
|
-
- `JsonFrameOptionError`: invalid options such as bad `packThreshold` or missing `columns` for `values`
|
|
429
|
+
- `JsonFrameOptionError`: invalid options such as bad `maxNodes`/`packThreshold` or missing `columns` for `values`
|
|
321
430
|
- `JsonFrameValidationError`: payload shape, transform argument, or JSON-value validation failure
|
|
322
431
|
- `AmbiguousOrientError`: `auto` mode cannot distinguish between multiple valid orients
|
|
323
432
|
- `UnsupportedFeatureError`: supported contract deliberately excludes the requested pandas feature
|
|
324
433
|
- `ExportKeyCollisionError`: object-key exporters would collapse distinct index labels to the same JSON key
|
|
325
434
|
|
|
326
|
-
`JsonFrameError` instances expose `orient`, `path`, `row`, `column`, and `value` when relevant. Scalar JSON diagnostic values (`string`, finite or non-finite `number`, `boolean`, `null`) are retained directly. Arrays, objects, functions, symbols, bigints, undefined values, and cyclic containers are replaced with small frozen summaries, so retaining an error does not retain caller-owned payloads and `JSON.stringify(error)` does not invoke user serialization hooks.
|
|
435
|
+
`JsonFrameError` instances expose `orient`, `path`, `row`, `column`, and `value` when relevant. Scalar JSON diagnostic values (`string`, finite or non-finite `number`, `boolean`, `null`) are retained directly and are intentionally not bounded by the container-preview budget below. Arrays, objects, functions, symbols, bigints, undefined values, and cyclic containers are replaced with small frozen summaries, so retaining an error does not retain caller-owned payloads and `JSON.stringify(error)` does not invoke user serialization hooks.
|
|
436
|
+
|
|
437
|
+
Object summaries retain at most 5 keys and at most 200 characters of key-preview text in total. `truncated` is `true` when either the key count or the key-preview text was shortened; `keyCount` always reports the full own enumerable key count. `truncated: false` therefore means the `keys` preview is complete, not merely that the count fit. This bounds the retained/serialized summary preview, not the complete error size: scalar diagnostic strings, `path`, `column`, `option`, `key`, and `labels` fields can still be input-sized. Error construction never calls caller `toJSON` hooks, and counting keys uses `Object.keys()`, which collects all own enumerable string keys — construction is not constant-space even though the retained preview is bounded.
|
|
327
438
|
|
|
328
439
|
Diagnostic summary shapes are:
|
|
329
440
|
|
package/index.d.mts
CHANGED
|
@@ -8,8 +8,10 @@ interface JsonObject {
|
|
|
8
8
|
}
|
|
9
9
|
/**
|
|
10
10
|
* Any JSON-compatible value accepted by `fromOrient` payloads and exporters.
|
|
11
|
-
* Arrays/objects may nest up to `JSON_FRAME_MAX_DEPTH` levels from the
|
|
12
|
-
* root value; deeper containers fail with `JsonFrameValidationError`.
|
|
11
|
+
* Arrays/objects may nest up to `JSON_FRAME_MAX_DEPTH` levels from the
|
|
12
|
+
* validated root value; deeper containers fail with `JsonFrameValidationError`.
|
|
13
|
+
* For serialization the root is the exported payload including orient
|
|
14
|
+
* wrappers, so a deeper output layout may reject a cell accepted at ingestion.
|
|
13
15
|
*/
|
|
14
16
|
type JsonValue = JsonPrimitive | JsonArray | JsonObject;
|
|
15
17
|
/**
|
|
@@ -102,8 +104,15 @@ interface TablePayload extends JsonObject {
|
|
|
102
104
|
* `options.columnTypes`.
|
|
103
105
|
*
|
|
104
106
|
* Explicit non-table `columnTypes` are validated against non-null cells without
|
|
105
|
-
* coercion. `datetime` accepts
|
|
106
|
-
*
|
|
107
|
+
* coercion. `datetime` accepts calendar-valid, timezone-naive ISO strings for
|
|
108
|
+
* four-digit years 0000–9999 under proleptic Gregorian rules (0000 is a leap
|
|
109
|
+
* year). Grammar: `YYYY-MM-DD`, optionally followed by `T` or a space and
|
|
110
|
+
* `HH:mm:ss`, optionally with 1–9 fractional-second digits. Hours are 00–23;
|
|
111
|
+
* minutes/seconds are 00–59. Numeric epochs and timezone suffixes are rejected.
|
|
112
|
+
* Strings are preserved exactly. Calendar validity does not guarantee pandas
|
|
113
|
+
* read-back: dtype/resolution limits, including nanosecond bounds for table
|
|
114
|
+
* read-back, can exclude accepted years. Inferred ISO-looking strings stay
|
|
115
|
+
* `string` unless Table Schema or an explicit type says otherwise.
|
|
107
116
|
* `categorical` accepts non-null scalar JSON values and exports as Table Schema
|
|
108
117
|
* `type: 'any'` with `extDtype: 'category'` when no source field metadata is
|
|
109
118
|
* available. `mixed` and `unknown` accept any JSON-compatible cell value.
|
|
@@ -132,19 +141,44 @@ interface FromOrientOptions {
|
|
|
132
141
|
/**
|
|
133
142
|
* Explicit logical column types applied after parsing when no Table Schema is
|
|
134
143
|
* present. Non-null cells must already be compatible; values are never
|
|
135
|
-
* coerced to satisfy the declared logical type.
|
|
144
|
+
* coerced to satisfy the declared logical type. Only own enumerable string
|
|
145
|
+
* keys are overrides; inherited properties are ignored. Names such as
|
|
146
|
+
* `constructor` and `__proto__` are valid column labels.
|
|
136
147
|
*/
|
|
137
148
|
readonly columnTypes?: Readonly<Partial<Record<ColumnLabel, ColumnType>>>;
|
|
138
149
|
/** Packing threshold for typed-array storage. `0` disables packing. */
|
|
139
150
|
readonly packThreshold?: number;
|
|
151
|
+
/**
|
|
152
|
+
* Optional positive safe integer limiting ingestion traversal occurrences.
|
|
153
|
+
* Counts the parsed root and every scalar/container value, including repeated
|
|
154
|
+
* aliases. Exact-budget inputs succeed; excess fails with a path-bearing
|
|
155
|
+
* JsonFrameValidationError. Omitted means no quota. Not retained by the frame
|
|
156
|
+
* for exports. Does not bound JSON.parse/input bytes, key allocation, frame
|
|
157
|
+
* densification, caller hooks, or total memory.
|
|
158
|
+
* @example fromOrient([{ n: 1 }], { orient: 'records', maxNodes: 3 })
|
|
159
|
+
*/
|
|
160
|
+
readonly maxNodes?: number;
|
|
140
161
|
}
|
|
141
162
|
/** Options for `toTable()` and `toJSONString('table', ...)`. */
|
|
142
163
|
interface ToTableOptions {
|
|
143
164
|
/** Override the emitted table index field name when the default would collide. */
|
|
144
165
|
readonly indexField?: string;
|
|
145
166
|
}
|
|
146
|
-
/**
|
|
147
|
-
|
|
167
|
+
/** Options validated for every `toJSONString()` orient; indexField affects table only. */
|
|
168
|
+
interface ToJSONStringOptions extends ToTableOptions {
|
|
169
|
+
/**
|
|
170
|
+
* Optional positive safe integer limiting the complete exported payload's
|
|
171
|
+
* validation traversal. Counts root, orient wrappers, table metadata and every
|
|
172
|
+
* scalar/container occurrence, revisiting aliases. Exact-budget succeeds;
|
|
173
|
+
* excess throws path-bearing JsonFrameValidationError with the selected orient.
|
|
174
|
+
* No default quota; resets per call, independently of ingestion maxNodes.
|
|
175
|
+
* Does not bound payload construction, key allocation, string bytes, native
|
|
176
|
+
* serialization hooks, or total memory. Validation does not clone cells.
|
|
177
|
+
* @example frame.toJSONString('records', { maxNodes: 3 })
|
|
178
|
+
* @example frame.toJSONString('table', { indexField: 'row_id', maxNodes: 100 })
|
|
179
|
+
*/
|
|
180
|
+
readonly maxNodes?: number;
|
|
181
|
+
}
|
|
148
182
|
/**
|
|
149
183
|
* Immutable column-major view over pandas-compatible tabular JSON.
|
|
150
184
|
*
|
|
@@ -167,7 +201,26 @@ interface DataFrame<TRow extends JsonCompatibleRow<TRow> = JsonRow> {
|
|
|
167
201
|
toSplit(): SplitPayload;
|
|
168
202
|
/** Exports Table Schema JSON; source index labels must be unique primary-key values. */
|
|
169
203
|
toTable(options?: ToTableOptions): TablePayload;
|
|
170
|
-
/**
|
|
204
|
+
/**
|
|
205
|
+
* Serializes an exported orient within `JSON_FRAME_MAX_DEPTH`.
|
|
206
|
+
*
|
|
207
|
+
* Depth is measured from the exported payload root including orient
|
|
208
|
+
* wrappers; `split`/`table` nest cells one level deeper than
|
|
209
|
+
* `records`/`values`/`index`/`columns`, so an input accepted at its own
|
|
210
|
+
* depth limit may be rejected in a deeper output layout. The complete
|
|
211
|
+
* selected output is validated before native serialization: cycles,
|
|
212
|
+
* over-depth containers, sparse arrays, and non-JSON values introduced
|
|
213
|
+
* through caller-mutable nested cells fail with path-bearing
|
|
214
|
+
* `JsonFrameValidationError` carrying the selected orient. Validation is one
|
|
215
|
+
* traversal pass without cloning followed by native stringification; both
|
|
216
|
+
* passes expand repeated references per occurrence. Optional maxNodes limits
|
|
217
|
+
* validation visits, including root/wrappers/metadata, independently per call
|
|
218
|
+
* with no default quota or inherited ingestion budget. It does not bound
|
|
219
|
+
* payload construction, key allocation, bytes, or total memory.
|
|
220
|
+
* Property getters/`Proxy` traps run during traversal and `toJSON`
|
|
221
|
+
* hooks may run during stringification; hooks are caller responsibility and
|
|
222
|
+
* are not sandboxed.
|
|
223
|
+
*/
|
|
171
224
|
toJSONString(orient: ResolvedOrient, options?: ToJSONStringOptions): string;
|
|
172
225
|
row(position: number): Readonly<TRow>;
|
|
173
226
|
rows(): readonly Readonly<TRow>[];
|
|
@@ -178,6 +231,26 @@ interface DataFrame<TRow extends JsonCompatibleRow<TRow> = JsonRow> {
|
|
|
178
231
|
resetIndex(): DataFrame<TRow>;
|
|
179
232
|
}
|
|
180
233
|
|
|
234
|
+
type RecordKeys<T> = T extends unknown ? Extract<keyof T, string | number> : never;
|
|
235
|
+
type IndexRecordKeys<K extends PropertyKey> = K extends unknown ? Record<never, never> extends Record<K, never> ? K : never : never;
|
|
236
|
+
type CanonicalRecord<T> = T extends unknown ? {
|
|
237
|
+
[K in keyof T as K extends string | number ? `${K}` : never]: T[K];
|
|
238
|
+
} : never;
|
|
239
|
+
type VariantRequiresKey<T, K extends PropertyKey> = T extends unknown ? K extends keyof T ? Pick<T, K> extends Required<Pick<T, K>> ? true : false : false : never;
|
|
240
|
+
type RequiredRecordKeys<T> = {
|
|
241
|
+
[K in RecordKeys<T>]: false extends VariantRequiresKey<T, K> ? never : K;
|
|
242
|
+
}[RecordKeys<T>];
|
|
243
|
+
type RecordCell<T, K extends PropertyKey> = T extends unknown ? K extends keyof T ? Exclude<T[K], undefined> : never : never;
|
|
244
|
+
type NormalizedRecord<T> = {
|
|
245
|
+
[K in RequiredRecordKeys<T>]: RecordCell<T, K>;
|
|
246
|
+
} & {
|
|
247
|
+
[K in Exclude<RecordKeys<T>, RequiredRecordKeys<T>>]?: RecordCell<T, K> | null;
|
|
248
|
+
};
|
|
249
|
+
type FlattenedRecord<T> = {
|
|
250
|
+
[K in keyof NormalizedRecord<T>]: NormalizedRecord<T>[K];
|
|
251
|
+
};
|
|
252
|
+
type InferredRecord<T> = Extract<T, readonly unknown[]> extends never ? [RecordKeys<T>] extends [never] ? JsonRow : [IndexRecordKeys<RecordKeys<T>>] extends [never] ? FlattenedRecord<CanonicalRecord<T>> : JsonRow : JsonRow;
|
|
253
|
+
type InferredRecordsFrame<T> = InferredRecord<T> extends infer TRow extends JsonCompatibleRow<TRow> ? DataFrame<TRow> : DataFrame<JsonRow>;
|
|
181
254
|
/**
|
|
182
255
|
* Parses pandas-compatible JSON into an immutable `DataFrame`.
|
|
183
256
|
*
|
|
@@ -193,9 +266,32 @@ interface DataFrame<TRow extends JsonCompatibleRow<TRow> = JsonRow> {
|
|
|
193
266
|
* preserved for those labels. JSON arrays/objects may nest up to
|
|
194
267
|
* `JSON_FRAME_MAX_DEPTH` levels from the parsed root value. Explicit non-table
|
|
195
268
|
* `columnTypes` are validated against non-null cells and never coerce values.
|
|
269
|
+
*
|
|
270
|
+
* Parsed records infer known required fields precisely. Fields missing from
|
|
271
|
+
* any record variant or declared optional include `null` for normalization;
|
|
272
|
+
* they remain optional because a column absent from every row is not created.
|
|
273
|
+
* This includes Object-member names such as `toString` and `constructor`:
|
|
274
|
+
* inherited members do not supply cells. Numeric literal keys and their string
|
|
275
|
+
* spellings (1 and '1') infer one string-named column with the combined cell
|
|
276
|
+
* types, accessible through either spelling. Open index domains (string,
|
|
277
|
+
* number, or template patterns such as `metric_${string}` and `${number}`)
|
|
278
|
+
* fall back to `JsonRow`, including unions/intersections with known fields.
|
|
279
|
+
* They do not promise every matching column is present; use scalar guards
|
|
280
|
+
* such as `typeof cell === 'number'` before numeric operations. Finite template
|
|
281
|
+
* key unions retain precise fields, including their optionality.
|
|
282
|
+
* Union variants are flattened, so discriminants do not imply cell presence.
|
|
283
|
+
* Normalization is shallow; nested cell types are preserved.
|
|
284
|
+
* Arrays (including readonly tuples) infer object-shaped `JsonRow` values,
|
|
285
|
+
* not array rows. Broad records/options and other layouts may also fall back
|
|
286
|
+
* to `JsonRow`. An explicit domain generic preserves the caller's row model;
|
|
287
|
+
* it is an assertion, not runtime schema validation.
|
|
196
288
|
*/
|
|
197
289
|
declare function fromOrient<TRow extends JsonCompatibleRow<TRow> = JsonRow>(input: string, options?: FromOrientOptions): DataFrame<TRow>;
|
|
198
|
-
declare function fromOrient<
|
|
290
|
+
declare function fromOrient<TRecords extends readonly object[]>(input: TRecords & readonly JsonCompatibleRow<TRecords[number]>[], options?: FromOrientOptions & {
|
|
291
|
+
readonly orient?: 'auto' | 'records';
|
|
292
|
+
}): InferredRecordsFrame<TRecords[number]>;
|
|
293
|
+
/** Explicit domain row assertion; does not infer or validate an application schema. */
|
|
294
|
+
declare function fromOrient<TRow extends JsonCompatibleRow<TRow> = JsonRow>(input: readonly NoInfer<TRow>[], options?: FromOrientOptions & {
|
|
199
295
|
readonly orient?: 'auto' | 'records';
|
|
200
296
|
}): DataFrame<TRow>;
|
|
201
297
|
declare function fromOrient<TRow extends JsonCompatibleRow<TRow> = JsonRow>(input: SplitPayload, options: FromOrientOptions & {
|
|
@@ -208,8 +304,15 @@ declare function fromOrient<TRow extends JsonCompatibleRow<TRow> = JsonRow>(inpu
|
|
|
208
304
|
|
|
209
305
|
/**
|
|
210
306
|
* Maximum supported nesting for JSON arrays/objects traversed by the package.
|
|
211
|
-
*
|
|
212
|
-
* an array/object at depth 1001 fails with
|
|
307
|
+
* Depth is measured from the validated root at depth 0; an array/object at
|
|
308
|
+
* depth 1000 is accepted, while an array/object at depth 1001 fails with
|
|
309
|
+
* `JsonFrameValidationError`.
|
|
310
|
+
*
|
|
311
|
+
* The root is the parsed input for ingestion and the complete exported payload
|
|
312
|
+
* for serialization, including orient-specific wrappers. A nested cell accepted
|
|
313
|
+
* at ingestion may therefore be rejected when serialized in a deeper output
|
|
314
|
+
* layout (for example `split`/`table` wrap cells one level deeper than
|
|
315
|
+
* `records`/`values`/`index`/`columns`).
|
|
213
316
|
*/
|
|
214
317
|
declare const JSON_FRAME_MAX_DEPTH = 1000;
|
|
215
318
|
|
|
@@ -220,7 +323,22 @@ type JsonFrameErrorContext = {
|
|
|
220
323
|
readonly column?: string;
|
|
221
324
|
readonly value?: unknown;
|
|
222
325
|
};
|
|
223
|
-
/**
|
|
326
|
+
/**
|
|
327
|
+
* Bounded value stored on structured errors instead of caller-owned containers.
|
|
328
|
+
*
|
|
329
|
+
* Container summaries are bounded: object summaries retain at most 5 keys and
|
|
330
|
+
* at most 200 characters of key-preview text in total. `truncated` is `true`
|
|
331
|
+
* when either the key count or the key-preview text was shortened. `keyCount`
|
|
332
|
+
* always reports the full key count. Array summaries retain only `length`.
|
|
333
|
+
*
|
|
334
|
+
* Scalar diagnostics are intentionally not bounded by that preview budget:
|
|
335
|
+
* scalar `string`/`number`/`boolean`/`null` values, plus `orient`, `path`,
|
|
336
|
+
* `row`, `column`, `option`, `key`, and `labels` fields, can still be
|
|
337
|
+
* input-sized. Summaries are frozen, never retain caller containers, and never
|
|
338
|
+
* invoke user `toJSON` hooks. Counting keys uses `Object.keys()`, which visits
|
|
339
|
+
* every own key, so construction is not constant-space even though the retained
|
|
340
|
+
* preview is bounded.
|
|
341
|
+
*/
|
|
224
342
|
type JsonFrameDiagnosticValue = string | number | boolean | null | Readonly<{
|
|
225
343
|
readonly kind: 'array';
|
|
226
344
|
readonly length: number;
|
package/index.d.ts
CHANGED
|
@@ -8,8 +8,10 @@ interface JsonObject {
|
|
|
8
8
|
}
|
|
9
9
|
/**
|
|
10
10
|
* Any JSON-compatible value accepted by `fromOrient` payloads and exporters.
|
|
11
|
-
* Arrays/objects may nest up to `JSON_FRAME_MAX_DEPTH` levels from the
|
|
12
|
-
* root value; deeper containers fail with `JsonFrameValidationError`.
|
|
11
|
+
* Arrays/objects may nest up to `JSON_FRAME_MAX_DEPTH` levels from the
|
|
12
|
+
* validated root value; deeper containers fail with `JsonFrameValidationError`.
|
|
13
|
+
* For serialization the root is the exported payload including orient
|
|
14
|
+
* wrappers, so a deeper output layout may reject a cell accepted at ingestion.
|
|
13
15
|
*/
|
|
14
16
|
type JsonValue = JsonPrimitive | JsonArray | JsonObject;
|
|
15
17
|
/**
|
|
@@ -102,8 +104,15 @@ interface TablePayload extends JsonObject {
|
|
|
102
104
|
* `options.columnTypes`.
|
|
103
105
|
*
|
|
104
106
|
* Explicit non-table `columnTypes` are validated against non-null cells without
|
|
105
|
-
* coercion. `datetime` accepts
|
|
106
|
-
*
|
|
107
|
+
* coercion. `datetime` accepts calendar-valid, timezone-naive ISO strings for
|
|
108
|
+
* four-digit years 0000–9999 under proleptic Gregorian rules (0000 is a leap
|
|
109
|
+
* year). Grammar: `YYYY-MM-DD`, optionally followed by `T` or a space and
|
|
110
|
+
* `HH:mm:ss`, optionally with 1–9 fractional-second digits. Hours are 00–23;
|
|
111
|
+
* minutes/seconds are 00–59. Numeric epochs and timezone suffixes are rejected.
|
|
112
|
+
* Strings are preserved exactly. Calendar validity does not guarantee pandas
|
|
113
|
+
* read-back: dtype/resolution limits, including nanosecond bounds for table
|
|
114
|
+
* read-back, can exclude accepted years. Inferred ISO-looking strings stay
|
|
115
|
+
* `string` unless Table Schema or an explicit type says otherwise.
|
|
107
116
|
* `categorical` accepts non-null scalar JSON values and exports as Table Schema
|
|
108
117
|
* `type: 'any'` with `extDtype: 'category'` when no source field metadata is
|
|
109
118
|
* available. `mixed` and `unknown` accept any JSON-compatible cell value.
|
|
@@ -132,19 +141,44 @@ interface FromOrientOptions {
|
|
|
132
141
|
/**
|
|
133
142
|
* Explicit logical column types applied after parsing when no Table Schema is
|
|
134
143
|
* present. Non-null cells must already be compatible; values are never
|
|
135
|
-
* coerced to satisfy the declared logical type.
|
|
144
|
+
* coerced to satisfy the declared logical type. Only own enumerable string
|
|
145
|
+
* keys are overrides; inherited properties are ignored. Names such as
|
|
146
|
+
* `constructor` and `__proto__` are valid column labels.
|
|
136
147
|
*/
|
|
137
148
|
readonly columnTypes?: Readonly<Partial<Record<ColumnLabel, ColumnType>>>;
|
|
138
149
|
/** Packing threshold for typed-array storage. `0` disables packing. */
|
|
139
150
|
readonly packThreshold?: number;
|
|
151
|
+
/**
|
|
152
|
+
* Optional positive safe integer limiting ingestion traversal occurrences.
|
|
153
|
+
* Counts the parsed root and every scalar/container value, including repeated
|
|
154
|
+
* aliases. Exact-budget inputs succeed; excess fails with a path-bearing
|
|
155
|
+
* JsonFrameValidationError. Omitted means no quota. Not retained by the frame
|
|
156
|
+
* for exports. Does not bound JSON.parse/input bytes, key allocation, frame
|
|
157
|
+
* densification, caller hooks, or total memory.
|
|
158
|
+
* @example fromOrient([{ n: 1 }], { orient: 'records', maxNodes: 3 })
|
|
159
|
+
*/
|
|
160
|
+
readonly maxNodes?: number;
|
|
140
161
|
}
|
|
141
162
|
/** Options for `toTable()` and `toJSONString('table', ...)`. */
|
|
142
163
|
interface ToTableOptions {
|
|
143
164
|
/** Override the emitted table index field name when the default would collide. */
|
|
144
165
|
readonly indexField?: string;
|
|
145
166
|
}
|
|
146
|
-
/**
|
|
147
|
-
|
|
167
|
+
/** Options validated for every `toJSONString()` orient; indexField affects table only. */
|
|
168
|
+
interface ToJSONStringOptions extends ToTableOptions {
|
|
169
|
+
/**
|
|
170
|
+
* Optional positive safe integer limiting the complete exported payload's
|
|
171
|
+
* validation traversal. Counts root, orient wrappers, table metadata and every
|
|
172
|
+
* scalar/container occurrence, revisiting aliases. Exact-budget succeeds;
|
|
173
|
+
* excess throws path-bearing JsonFrameValidationError with the selected orient.
|
|
174
|
+
* No default quota; resets per call, independently of ingestion maxNodes.
|
|
175
|
+
* Does not bound payload construction, key allocation, string bytes, native
|
|
176
|
+
* serialization hooks, or total memory. Validation does not clone cells.
|
|
177
|
+
* @example frame.toJSONString('records', { maxNodes: 3 })
|
|
178
|
+
* @example frame.toJSONString('table', { indexField: 'row_id', maxNodes: 100 })
|
|
179
|
+
*/
|
|
180
|
+
readonly maxNodes?: number;
|
|
181
|
+
}
|
|
148
182
|
/**
|
|
149
183
|
* Immutable column-major view over pandas-compatible tabular JSON.
|
|
150
184
|
*
|
|
@@ -167,7 +201,26 @@ interface DataFrame<TRow extends JsonCompatibleRow<TRow> = JsonRow> {
|
|
|
167
201
|
toSplit(): SplitPayload;
|
|
168
202
|
/** Exports Table Schema JSON; source index labels must be unique primary-key values. */
|
|
169
203
|
toTable(options?: ToTableOptions): TablePayload;
|
|
170
|
-
/**
|
|
204
|
+
/**
|
|
205
|
+
* Serializes an exported orient within `JSON_FRAME_MAX_DEPTH`.
|
|
206
|
+
*
|
|
207
|
+
* Depth is measured from the exported payload root including orient
|
|
208
|
+
* wrappers; `split`/`table` nest cells one level deeper than
|
|
209
|
+
* `records`/`values`/`index`/`columns`, so an input accepted at its own
|
|
210
|
+
* depth limit may be rejected in a deeper output layout. The complete
|
|
211
|
+
* selected output is validated before native serialization: cycles,
|
|
212
|
+
* over-depth containers, sparse arrays, and non-JSON values introduced
|
|
213
|
+
* through caller-mutable nested cells fail with path-bearing
|
|
214
|
+
* `JsonFrameValidationError` carrying the selected orient. Validation is one
|
|
215
|
+
* traversal pass without cloning followed by native stringification; both
|
|
216
|
+
* passes expand repeated references per occurrence. Optional maxNodes limits
|
|
217
|
+
* validation visits, including root/wrappers/metadata, independently per call
|
|
218
|
+
* with no default quota or inherited ingestion budget. It does not bound
|
|
219
|
+
* payload construction, key allocation, bytes, or total memory.
|
|
220
|
+
* Property getters/`Proxy` traps run during traversal and `toJSON`
|
|
221
|
+
* hooks may run during stringification; hooks are caller responsibility and
|
|
222
|
+
* are not sandboxed.
|
|
223
|
+
*/
|
|
171
224
|
toJSONString(orient: ResolvedOrient, options?: ToJSONStringOptions): string;
|
|
172
225
|
row(position: number): Readonly<TRow>;
|
|
173
226
|
rows(): readonly Readonly<TRow>[];
|
|
@@ -178,6 +231,26 @@ interface DataFrame<TRow extends JsonCompatibleRow<TRow> = JsonRow> {
|
|
|
178
231
|
resetIndex(): DataFrame<TRow>;
|
|
179
232
|
}
|
|
180
233
|
|
|
234
|
+
type RecordKeys<T> = T extends unknown ? Extract<keyof T, string | number> : never;
|
|
235
|
+
type IndexRecordKeys<K extends PropertyKey> = K extends unknown ? Record<never, never> extends Record<K, never> ? K : never : never;
|
|
236
|
+
type CanonicalRecord<T> = T extends unknown ? {
|
|
237
|
+
[K in keyof T as K extends string | number ? `${K}` : never]: T[K];
|
|
238
|
+
} : never;
|
|
239
|
+
type VariantRequiresKey<T, K extends PropertyKey> = T extends unknown ? K extends keyof T ? Pick<T, K> extends Required<Pick<T, K>> ? true : false : false : never;
|
|
240
|
+
type RequiredRecordKeys<T> = {
|
|
241
|
+
[K in RecordKeys<T>]: false extends VariantRequiresKey<T, K> ? never : K;
|
|
242
|
+
}[RecordKeys<T>];
|
|
243
|
+
type RecordCell<T, K extends PropertyKey> = T extends unknown ? K extends keyof T ? Exclude<T[K], undefined> : never : never;
|
|
244
|
+
type NormalizedRecord<T> = {
|
|
245
|
+
[K in RequiredRecordKeys<T>]: RecordCell<T, K>;
|
|
246
|
+
} & {
|
|
247
|
+
[K in Exclude<RecordKeys<T>, RequiredRecordKeys<T>>]?: RecordCell<T, K> | null;
|
|
248
|
+
};
|
|
249
|
+
type FlattenedRecord<T> = {
|
|
250
|
+
[K in keyof NormalizedRecord<T>]: NormalizedRecord<T>[K];
|
|
251
|
+
};
|
|
252
|
+
type InferredRecord<T> = Extract<T, readonly unknown[]> extends never ? [RecordKeys<T>] extends [never] ? JsonRow : [IndexRecordKeys<RecordKeys<T>>] extends [never] ? FlattenedRecord<CanonicalRecord<T>> : JsonRow : JsonRow;
|
|
253
|
+
type InferredRecordsFrame<T> = InferredRecord<T> extends infer TRow extends JsonCompatibleRow<TRow> ? DataFrame<TRow> : DataFrame<JsonRow>;
|
|
181
254
|
/**
|
|
182
255
|
* Parses pandas-compatible JSON into an immutable `DataFrame`.
|
|
183
256
|
*
|
|
@@ -193,9 +266,32 @@ interface DataFrame<TRow extends JsonCompatibleRow<TRow> = JsonRow> {
|
|
|
193
266
|
* preserved for those labels. JSON arrays/objects may nest up to
|
|
194
267
|
* `JSON_FRAME_MAX_DEPTH` levels from the parsed root value. Explicit non-table
|
|
195
268
|
* `columnTypes` are validated against non-null cells and never coerce values.
|
|
269
|
+
*
|
|
270
|
+
* Parsed records infer known required fields precisely. Fields missing from
|
|
271
|
+
* any record variant or declared optional include `null` for normalization;
|
|
272
|
+
* they remain optional because a column absent from every row is not created.
|
|
273
|
+
* This includes Object-member names such as `toString` and `constructor`:
|
|
274
|
+
* inherited members do not supply cells. Numeric literal keys and their string
|
|
275
|
+
* spellings (1 and '1') infer one string-named column with the combined cell
|
|
276
|
+
* types, accessible through either spelling. Open index domains (string,
|
|
277
|
+
* number, or template patterns such as `metric_${string}` and `${number}`)
|
|
278
|
+
* fall back to `JsonRow`, including unions/intersections with known fields.
|
|
279
|
+
* They do not promise every matching column is present; use scalar guards
|
|
280
|
+
* such as `typeof cell === 'number'` before numeric operations. Finite template
|
|
281
|
+
* key unions retain precise fields, including their optionality.
|
|
282
|
+
* Union variants are flattened, so discriminants do not imply cell presence.
|
|
283
|
+
* Normalization is shallow; nested cell types are preserved.
|
|
284
|
+
* Arrays (including readonly tuples) infer object-shaped `JsonRow` values,
|
|
285
|
+
* not array rows. Broad records/options and other layouts may also fall back
|
|
286
|
+
* to `JsonRow`. An explicit domain generic preserves the caller's row model;
|
|
287
|
+
* it is an assertion, not runtime schema validation.
|
|
196
288
|
*/
|
|
197
289
|
declare function fromOrient<TRow extends JsonCompatibleRow<TRow> = JsonRow>(input: string, options?: FromOrientOptions): DataFrame<TRow>;
|
|
198
|
-
declare function fromOrient<
|
|
290
|
+
declare function fromOrient<TRecords extends readonly object[]>(input: TRecords & readonly JsonCompatibleRow<TRecords[number]>[], options?: FromOrientOptions & {
|
|
291
|
+
readonly orient?: 'auto' | 'records';
|
|
292
|
+
}): InferredRecordsFrame<TRecords[number]>;
|
|
293
|
+
/** Explicit domain row assertion; does not infer or validate an application schema. */
|
|
294
|
+
declare function fromOrient<TRow extends JsonCompatibleRow<TRow> = JsonRow>(input: readonly NoInfer<TRow>[], options?: FromOrientOptions & {
|
|
199
295
|
readonly orient?: 'auto' | 'records';
|
|
200
296
|
}): DataFrame<TRow>;
|
|
201
297
|
declare function fromOrient<TRow extends JsonCompatibleRow<TRow> = JsonRow>(input: SplitPayload, options: FromOrientOptions & {
|
|
@@ -208,8 +304,15 @@ declare function fromOrient<TRow extends JsonCompatibleRow<TRow> = JsonRow>(inpu
|
|
|
208
304
|
|
|
209
305
|
/**
|
|
210
306
|
* Maximum supported nesting for JSON arrays/objects traversed by the package.
|
|
211
|
-
*
|
|
212
|
-
* an array/object at depth 1001 fails with
|
|
307
|
+
* Depth is measured from the validated root at depth 0; an array/object at
|
|
308
|
+
* depth 1000 is accepted, while an array/object at depth 1001 fails with
|
|
309
|
+
* `JsonFrameValidationError`.
|
|
310
|
+
*
|
|
311
|
+
* The root is the parsed input for ingestion and the complete exported payload
|
|
312
|
+
* for serialization, including orient-specific wrappers. A nested cell accepted
|
|
313
|
+
* at ingestion may therefore be rejected when serialized in a deeper output
|
|
314
|
+
* layout (for example `split`/`table` wrap cells one level deeper than
|
|
315
|
+
* `records`/`values`/`index`/`columns`).
|
|
213
316
|
*/
|
|
214
317
|
declare const JSON_FRAME_MAX_DEPTH = 1000;
|
|
215
318
|
|
|
@@ -220,7 +323,22 @@ type JsonFrameErrorContext = {
|
|
|
220
323
|
readonly column?: string;
|
|
221
324
|
readonly value?: unknown;
|
|
222
325
|
};
|
|
223
|
-
/**
|
|
326
|
+
/**
|
|
327
|
+
* Bounded value stored on structured errors instead of caller-owned containers.
|
|
328
|
+
*
|
|
329
|
+
* Container summaries are bounded: object summaries retain at most 5 keys and
|
|
330
|
+
* at most 200 characters of key-preview text in total. `truncated` is `true`
|
|
331
|
+
* when either the key count or the key-preview text was shortened. `keyCount`
|
|
332
|
+
* always reports the full key count. Array summaries retain only `length`.
|
|
333
|
+
*
|
|
334
|
+
* Scalar diagnostics are intentionally not bounded by that preview budget:
|
|
335
|
+
* scalar `string`/`number`/`boolean`/`null` values, plus `orient`, `path`,
|
|
336
|
+
* `row`, `column`, `option`, `key`, and `labels` fields, can still be
|
|
337
|
+
* input-sized. Summaries are frozen, never retain caller containers, and never
|
|
338
|
+
* invoke user `toJSON` hooks. Counting keys uses `Object.keys()`, which visits
|
|
339
|
+
* every own key, so construction is not constant-space even though the retained
|
|
340
|
+
* preview is bounded.
|
|
341
|
+
*/
|
|
224
342
|
type JsonFrameDiagnosticValue = string | number | boolean | null | Readonly<{
|
|
225
343
|
readonly kind: 'array';
|
|
226
344
|
readonly length: number;
|