@lotics/xlsx 0.1.5 → 0.1.7

package/AGENTS.md

@@ -0,0 +1,382 @@
+# @lotics/xlsx — the spreadsheet (.xlsx) read / build / write reference
+
+A self-contained spreadsheet engine: it parses `.xlsx` (and legacy `.xls`) into an immutable
+model, builds workbooks from scratch or from rows, evaluates formulas, authors native pivot
+tables, and writes real `.xlsx` bytes — with **no third-party spreadsheet library** in the
+export path. This file is the "reach for what" guide; the **exact API** of any function is its
+source, which ships with the package: read `node_modules/@lotics/xlsx/src/<module>.ts`.
+
+- **The barrel is the stable surface.** `import { parseExcelBuffer, exportWorkbook } from "@lotics/xlsx"`.
+  Everything documented as "barrel" below comes from there.
+- **Subpath imports** (`import { parseBiff8 } from "@lotics/xlsx/biff8_reader"`) reach internals
+  that are NOT in the stability contract — the formula engine, the legacy reader, the template
+  helper. They work and are documented here where they're the right tool, but they can change
+  shape between versions; prefer the barrel when it covers the task.
+- **Never hand-roll xlsx XML or reach for `exceljs`/`xlsx`/SheetJS.** They are not dependencies.
+  If a primitive is missing, add it to this package — every consumer inherits it.
+
+## The two type families — read this first
+
+The whole package is organized around two shapes. Pick the right one and everything else follows.
+
+- **`ParsedSpreadsheet` — the immutable snapshot.** What `parseExcelBuffer` returns. A plain
+  read-only tree: `sheets[]` → `rows[]` → `cells[]`, plus merges, images, charts, tables, pivots,
+  conditional formats. **If you only READ a file, stay here** — never build a model you won't mutate.
+- **`WorkbookModel` — the mutable model.** Sparse cell storage, a deduplicated `StyleRegistry`,
+  undo/redo commands. **Build this when you WRITE** — every builder (`buildDataWorkbook`,
+  `buildPivotTable`, the raw `SheetModel`) produces one, and `exportWorkbook` consumes one.
+  To edit an existing file, parse to a snapshot then `loadWorkbookFromSnapshot(parsed)`.
+
+```
+bytes ─parseExcelBuffer→ ParsedSpreadsheet ─loadWorkbookFromSnapshot→ WorkbookModel ─exportWorkbook→ bytes
+                          (read here)                                  (mutate here)
+```
+
+`exportWorkbook` takes a `WorkbookModel`, never a snapshot. A snapshot is a dead end for writing.
+
+## Reach by task
+
+### Read an `.xlsx` (or `.xls`) — `excel_parser.ts` (barrel)
+
+`parseExcelBuffer(arrayBuffer, options?) → ParsedSpreadsheet`. **Synchronous.** It takes an
+**`ArrayBuffer`**, not a Node `Buffer` — see the slicing gotcha below. It auto-detects an OLE2
+container (legacy `.xls` or an encrypted `.xlsx`) and routes it to the BIFF8 reader; genuinely
+encrypted files throw `PasswordProtectedError` (re-exported from the barrel for your catch site).
+
+```ts
+import { readFileSync } from "node:fs";
+import { parseExcelBuffer } from "@lotics/xlsx";
+
+const buf = readFileSync("report.xlsx");
+// Buffer is a view over a possibly-pooled ArrayBuffer — slice to its own bytes:
+const wb = parseExcelBuffer(buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength));
+
+for (const sheet of wb.sheets) {
+  for (const row of sheet.rows) {          // sparse: only populated rows, in order
+    for (const cell of row.cells) {         // sparse: only populated cols, in order
+      cell.column;        // 1-based column index
+      cell.value;         // the DISPLAY string (number-formatted, e.g. "1.234,00")
+      cell.rawValue;      // the raw number, when the cell is numeric (no reparse of `value`)
+      cell.typedValue;    // lossless typed value string|number|boolean|null (round-trip source)
+      cell.formula;       // formula text WITHOUT a leading "=", e.g. "SUM(A1:A5)"
+      cell.numFmtCode;    // e.g. "#,##0.00", "dd/mm/yyyy"
+      cell.error;         // "#DIV/0!" etc. for error cells
+      cell.content;       // { type: "plain"|"rich_text"|"hyperlink", ... }
+      cell.style;         // CellStyle (fonts, fill, borders, alignment)
+    }
+  }
+}
+```
+
+- **`value` is the formatted display string; `rawValue`/`typedValue` are the machine values.**
+  Do arithmetic on `rawValue`/`typedValue`, never `parseFloat(cell.value)` — the display string
+  is locale/format-mangled.
+- **Rows and cells are sparse and index-bearing.** `row.index` (1-based) and `cell.column`
+  (1-based) are the truth; do not assume `rows[0]` is row 1 or `cells[0]` is column A.
+- **Merges, hidden state, extras:** `sheet.mergedCells` (`{startRow,startCol,endRow,endCol}`,
+  1-based), `sheet.hidden` (hidden/veryHidden sheets are parsed, only tagged — data consumers can
+  still read them), `sheet.freezePane`, `sheet.charts`, `sheet.tables`, `sheet.pivotTables`,
+  `sheet.conditionalFormats`, `sheet.images`, `sheet.dataValidations`. Workbook-scope:
+  `wb.namedRanges`, `wb.pivotCaches`, `wb.fullCalcOnLoad`.
+- **Row cap.** `options.maxRowsPerSheet` defaults to **10 000** (a renderer guard). For full data
+  extraction pass `{ maxRowsPerSheet: Infinity }` and check `sheet.truncated` to see if the cap bit.
+- `parseExcelFromZip(zip, options?)` (barrel) parses from an already-unzipped
+  `Record<string, Uint8Array>` (from `unzipXlsx`, also barrel). Use it when you need BOTH the
+  snapshot AND the raw ZIP entries — pass that same `zip` back to `exportWorkbook` as the
+  `originalZip` to preserve unmodified parts (see Export).
+
+### Legacy `.xls` (BIFF8) — `biff8_reader.ts` (subpath)
+
+The modern parser reads only zip-based OOXML `.xlsx`. A pre-2007 `.xls` is an OLE2 Compound File
+of BIFF records — a completely different container. **`parseExcelBuffer` handles this for you:** it
+detects the OLE2 magic and routes to `parseBiff8`, returning the same `ParsedSpreadsheet` shape, so
+most callers never touch this module. Reach for it directly only when you already hold the bytes as
+a `Uint8Array` and know it's a legacy workbook:
+
+```ts
+import { parseBiff8 } from "@lotics/xlsx/biff8_reader";
+const parsed = parseBiff8(new Uint8Array(bytes), { maxRowsPerSheet: Infinity });
+```
+
+`parseBiff8(bytes: Uint8Array, options?: ParseOptions) → ParsedSpreadsheet`. **Scope is the data
+grid**, so consumers get uniform cell values with no per-caller `.xls` handling:
+
+- Reads: shared strings (`SST`, including `CONTINUE`-split records), string cells (`LABELSST`,
+  `LABEL`), numbers (`NUMBER`, `RK`, `MULRK`), boolean/numeric cached formula results, multi-sheet
+  workbooks (`BOUNDSHEET`), the 1900/1904 date mode (`DATEMODE`), and number-format codes
+  (`FORMAT`/`XF`) — enough to detect dates and currency so serials render correctly downstream.
+- **Does NOT surface:** cell styling (returns `style: {}`), merged cells, charts, images, column
+  widths, row heights, or hidden state. Blank/`MULBLANK`/`BOOLERR` cells are skipped. A formula
+  whose cached result is a **string** or **error** yields nothing (the resolving `STRING` record is
+  out of scope) rather than a wrong value.
+- Encrypted `.xls` (a `FILEPASS` record or an OLE2 `EncryptedPackage` stream) throws
+  `PasswordProtectedError`. A non-BIFF8 OLE2 container throws a plain descriptive error.
+- No third-party dependency and no regex/eval over file bytes — this path replaced SheetJS, which
+  was removed for prototype-pollution + ReDoS CVEs. Do not reintroduce SheetJS.
+
+### Build a sheet from rows — `data_workbook.ts` (barrel)
+
+The common "I have records, give me a styled `.xlsx`" path. Declare each column once with a `type`
+and a `value` accessor; the helper writes a bold frozen header, per-type number formats (dates as
+real Excel serials), right-aligned numerics, and auto-fit widths.
+
+```ts
+import { buildDataWorkbook, exportWorkbook } from "@lotics/xlsx";
+
+const wb = buildDataWorkbook({
+  sheetName: "Phí",                         // sanitized to Excel's rules, ≤31 chars
+  currencyFormat: '#,##0" ₫"',              // default "#,##0"
+  dateFormat: "dd/mm/yyyy",                 // default "dd/mm/yyyy"
+  columns: [
+    { header: "Container", value: (r) => r.container },              // type defaults to "text"
+    { header: "Ngày",     type: "date",     value: (r) => r.date },  // Date | "YYYY-MM-DD"
+    { header: "Số tiền",  type: "currency", value: (r) => r.amount },// number | numeric string
+  ],
+  rows,
+});
+const bytes = exportWorkbook(wb);
+```
+
+- `DataColumnType` = `"text" | "integer" | "number" | "currency" | "date"`. `integer`→`#,##0`,
+  `number`→`#,##0.##`, `currency`→`currencyFormat`, `date`→`dateFormat`; numerics right-align.
+- A `date` accessor returns a `Date` or an ISO `YYYY-MM-DD` string → stored as an Excel serial.
+  An **unparseable** date/number is written as its text so nothing is silently dropped.
+- `value` returning `null`/`undefined`/`""` leaves the cell truly empty. Column `width` (in
+  characters) overrides the auto-fit; auto-fit samples the first 1000 rows.
+- **Multi-sheet:** `buildDataSheet(opts, styles)` builds ONE `SheetModel` into a caller-supplied
+  `StyleRegistry`. Every sheet in one workbook must share the SAME registry (styles are stored by
+  index into it) — build each with `buildDataSheet` against one `new StyleRegistry()`, then
+  `new WorkbookModel({ sheets, activeSheetIndex: 0, styles })`.
+
+### Build cell-by-cell — `workbook_model.ts` (barrel)
+
+When the layout isn't a flat table (forms, invoices, dashboards), drive `SheetModel` directly.
+
+```ts
+import { WorkbookModel, SheetModel, StyleRegistry, exportWorkbook } from "@lotics/xlsx";
+
+const styles = new StyleRegistry();
+const sheet = new SheetModel("Invoice", styles);
+
+sheet.set("A1", "Total", { fontBold: true });          // set(ref, value, style?, numFmt?, formula?)
+sheet.set("B1", 1234.5, undefined, "#,##0.00");        // value + number-format code
+sheet.merge("A1:C1", "Header", { horizontalAlign: "center" });
+sheet.setColumnWidth("A", 24);                          // letter or 0-based index
+sheet.setRowHeight(1, 22);                              // row is 1-based
+sheet.setFreeze("A2");                                  // freeze row 1
+sheet.headerRow(1, ["A", "B", "C"], { background: "#D9E1F2" }); // bold+centered+bordered row
+
+const wb = new WorkbookModel({ sheets: [sheet], activeSheetIndex: 0, styles });
+const bytes = exportWorkbook(wb);
+```
+
+- `set(ref, value, style?, numFmt?, formula?) → CellModel`. `value` is `string | number | boolean
+  | null`. `style` is a `CellStyle` (interned via the registry). `numFmt` is a format-code STRING
+  (`"#,##0.00"`, `"dd/mm/yyyy"`, `"General"`). `formula` is the formula text **WITHOUT `=`** (see
+  Formulas).
+- `SheetModel` also exposes `getCell`, `deleteCell`, `colWidths`/`rowHeights` maps,
+  `hiddenRows`/`hiddenCols` sets, `mergedCells`, `getUsedRange()`, and `view`
+  (`showGridLines`, `zoomScale`, `tabColor`, …).
+- `setWithStyleIndex(ref, value, styleIndex, numFmtCode?, formula?, originalXfIndex?)` is the hot
+  path for bulk fills: it skips the display-value/content materialization. If you later need
+  `cell.displayValue` for such a cell, call `materializeDisplayValue(cell)` (barrel).
+- **Edit a parsed file:** `loadWorkbookFromSnapshot(parsed, date1904?)` turns a `ParsedSpreadsheet`
+  into a `WorkbookModel`, interning every style and marking the registry clean so a value-only edit
+  keeps the export fast path (see Export).
+- **`buildTemplateWorkbook(...)`** (subpath `@lotics/xlsx/template_builder`) is sugar over the above
+  for one or several sheets: `buildTemplateWorkbook((s) => {...})` or
+  `buildTemplateWorkbook({ name, build }, ...)`.
+
+### Styling — `types.ts` (`CellStyle`) + `style_helpers.ts` + `ooxml_color.ts` (barrel)
+
+A `CellStyle` is a flat options bag interned by the `StyleRegistry` (Excel's `cellXfs` model — one
+stored entry per distinct style). Every field is optional:
+
+- **Font:** `fontBold`, `fontItalic`, `fontStrike`, `fontColor` (`#RRGGBB`), `fontSize` (points),
+  `fontName`, `fontUnderline` (`"single"|"double"`), `fontVertAlign` (`"superscript"|"subscript"`).
+- **Fill:** `backgroundColor` (`#RRGGBB`), `patternType`, `gradientData` (`{type, degree, stops}`).
+- **Alignment:** `horizontalAlign` (`"left"|"center"|"right"`), `verticalAlign`
+  (`"top"|"middle"|"bottom"`), `wrapText`, `shrinkToFit`, `indent`, `textRotation`.
+- **Borders:** `borderTop`/`borderRight`/`borderBottom`/`borderLeft`/`borderDiagonal`, each a
+  `BorderStyle` = `{ width, style, color? }` where `style` is an OOXML border style name
+  (`"thin"`, `"medium"`, `"thick"`, `"dashed"`, `"double"`, …).
+
+Reach for the shortcuts instead of hand-writing four border objects:
+
+- `allBorders(style?="thin", color?="#000000")` → all four sides; `topBorder(...)` /
+  `bottomBorder(style?="medium", ...)` → one side. Spread into a `CellStyle`.
+- Colors: `normalizeColor`, `prefixHex`, `hexToArgb`, `hexToOoxmlRgb`, `applyTint` — canonical
+  `#RRGGBB` in the model; the writer converts to OOXML `AARRGGBB` on export.
+
+### Formulas — `set(...)`'s 5th arg + `formula_engine.ts` (subpath)
+
+Two facts govern everything here:
+
+1. **A cell carries formula text AND a cached value.** `sheet.set(ref, cachedValue, style, numFmt,
+   formulaText)`. `exportWorkbook` writes both: `<f>formulaText</f>` and `<v>cachedValue</v>`.
+2. **The writer does NOT recompute.** It also stamps `<calcPr fullCalcOnLoad="1"/>` on the workbook,
+   so **Excel/LibreOffice recompute every formula the moment they open your file** — a formula
+   written with `null` as its cached value still shows the right number in a spreadsheet app.
+
+```ts
+sheet.set("D2", 15, undefined, undefined, "SUM(A2:C2)");  // formula text has NO leading "="
+sheet.set("D3", null, undefined, undefined, "SUM(A3:C3)"); // cached value unknown → Excel fills it
+```
+
+The gap: a **non-Excel** consumer that reads the cached `<v>` (this package's own parser/renderer, a
+PDF/print pipeline, another tool) sees `null`/stale until it recomputes. To fill correct cached
+values yourself before export:
+
+```ts
+import { recomputeAll } from "@lotics/xlsx/formula_engine";
+recomputeAll(workbook);           // evaluates every formula cell, overwriting cached values
+const bytes = exportWorkbook(workbook);
+```
+
+- `recomputeAll(workbook)` and `recomputeIfStale(workbook)` (the latter only recomputes when the
+  workbook declares `fullCalcOnLoad`) are the convenience entry points. For incremental recalc while
+  editing, construct a `FormulaEngine(workbook)` and call `buildGraph()` + `evaluateMissing()` /
+  `evaluateAll()`, or `onCellChanged(sheet, ref, formula)` after each edit.
+- **Unsupported functions:** `LAMBDA`, `LET`, dynamic-array spill, and cube functions are not
+  implemented (no open-source engine covers them) — don't emit workbooks that depend on them and
+  expect this package to evaluate the result. Excel still will, on open.
+- **When you set a cell from a raw user/agent string that may or may not be a formula**, use the
+  command layer instead: `setCellValueCommand` treats a leading `=` as a formula and strips it for
+  you (see Editing). The raw `set()`/`setWithStyleIndex()` path expects the `=` already removed.
+
+### Native pivot tables — `pivot_builder.ts` (barrel)
+
+`buildPivotTable(workbook, spec) → PivotTableModel` authors a **real, refreshable Excel
+PivotTable** (not a pre-rendered cross-tab): `exportWorkbook` serializes it into genuine
+`pivotCacheDefinition` / `pivotCacheRecords` / `pivotTable` OOXML parts, so the user can refresh and
+re-pivot it in Excel.
+
+**v1 shape — the 2D cross-tab:** exactly one row field, one column field, one data (value) field,
+and zero or more page (report-filter) fields.
+
+```ts
+import { buildPivotTable } from "@lotics/xlsx";
+
+buildPivotTable(workbook, {
+  name: "PivotByPort",                          // unique within the workbook
+  source: { sheetName: "Data", ref: "A1:E500" },// first row is the header
+  rowField: 0,                                  // 0-based column index WITHIN the range
+  colField: 2,
+  data: { field: 4, fn: "sum", numFmtId: "3" }, // fn defaults to "sum"
+  pageFields: [1],                              // optional report filters
+  location: { sheetName: "Pivot", anchor: "A3" },
+});
+```
+
+- `data.fn` is a `PivotAggregateFn`: `"sum" | "count" | "countNums" | "average" | "min" | "max" |
+  "product"`. `rowGrandTotals`/`colGrandTotals` default `true`; `styleName` defaults
+  `"PivotStyleLight16"`.
+- **`data.numFmtId` is a BUILT-IN Excel number-format ID** (e.g. `"3"` = `#,##0`), a string, NOT a
+  format code. A custom code like `'#,##0" ₫"'` needs a registered style, which v1 does not wire up.
+- Validation throws (not silently) on an out-of-range field index or a field reused in more than one
+  role (row/column/data/page). The returned model is already attached to the target sheet; just
+  export.
+- The read side of pivots (`ParsedPivotTable`, `ParsedPivotCache`, `PivotTableModel`) is barrel-
+  exported for inspecting pivots parsed out of an existing file.
+
+### Tables, charts, sparklines — authored via the parsed structures (barrel types)
+
+There is **no dedicated `buildTable`/`buildChart` helper.** These features are read-first: the
+parser fills `sheet.tables` (`ParsedTable`), `sheet.charts` (`ParsedChart`), `sheet.sparklineGroups`,
+etc., and they **round-trip through `exportWorkbook` unchanged.** To AUTHOR one, construct the parsed
+structure and push it onto the sheet array on the `SheetModel`; the writer serializes
+`sheet.tables` → `xl/tables/*.xml` and `sheet.charts` → `xl/charts/*.xml` + drawing anchors.
+
+```ts
+sheet.tables.push({
+  name: "tblFees", displayName: "tblFees", ref: "A1:D20",
+  headerRow: true, totalsRow: false, autoFilter: true, styleName: "TableStyleMedium2",
+  columns: [ { id: 1, name: "Container" }, { id: 2, name: "Amount" }, /* … match the range */ ],
+});
+```
+
+A `ParsedChart` needs `chartType` (`"bar"|"col"|"line"|"pie"|"area"|"scatter"|…`), `series[]`
+(`{ name?, values: number[], color? }`), optional `categories`/`axes`/`legendPosition`, and an
+`anchor` (`{ from:{row,col}, to:{row,col} }`, 0-based). Treat this as a lower-level path than
+`buildPivotTable`: you're assembling the read-model by hand, so match the column set to the range and
+verify the output opens in Excel.
+
+### Edit with undo/redo — `spreadsheet_commands.ts` (barrel)
+
+Every mutation that needs to be undoable flows through a command factory that captures inverse state.
+Each returns a `Command` (`{ description, execute, undo }`); run it and push it onto your own history
+(or `command_history.ts`). The full set is barrel-exported: `setCellValueCommand`,
+`setCellStyleCommand`, `setRangeStyleCommand`, `setRangeNumFmtCommand`, `deleteCellCommand`,
+`clearRangeCommand`, `mergeCellsCommand`/`unmergeCellsCommand`, `insertRowsCommand`/
+`deleteRowsCommand`, `insertColsCommand`/`deleteColsCommand`, `setColWidthCommand`/
+`setRowHeightCommand`, `setNamedRangeCommand`, `addSheetCommand`/`deleteSheetCommand`/
+`renameSheetCommand`/`reorderSheetsCommand`, `setHiddenRowsCommand`/`setHiddenColsCommand`,
+`sortColumnCommand`/`multiSortCommand`, `setFreezePanesCommand`, `setZoomCommand`,
+`addConditionalFormatCommand`/`deleteConditionalFormatCommand`, `insertCellsShiftDownCommand`, and
+`batchCommand` (compose several into one undo step).
+
+- `setCellValueCommand(workbook, engine, sheetIndex, row, col, newValue)` — `newValue` is the raw
+  string; a leading `=` is detected and stored as a formula (with `null` cached value for the
+  `engine` to compute). Pass a `FormulaEngine` so dependents recalc; pass `undefined` to skip recalc.
+- Row/col insert/delete re-key populated cells and shift formula references (via
+  `adjustMergedCells*` / `expandFormulaRangesForRowInsert`, also barrel-exported) — this is why they
+  must be commands, not direct `SheetModel` edits.
+
+### Export to `.xlsx` bytes — `xlsx_writer.ts` (barrel)
+
+`exportWorkbook(workbook, originalZip?) → Uint8Array`. **Synchronous.** Returns a fresh
+zip (`fflate` level 6). Write it to disk:
+
+```ts
+import { writeFileSync } from "node:fs";
+import { exportWorkbook } from "@lotics/xlsx";
+
+const bytes = exportWorkbook(wb);                 // Uint8Array
+writeFileSync("out.xlsx", Buffer.from(bytes));    // Buffer.from wraps the bytes for fs
+```
+
+- **`originalZip` preserves unmodified parts.** When you parsed with `parseExcelFromZip(zip)` and
+  edited the resulting model, pass that same `zip` back: `exportWorkbook(wb, zip)`. Parts the writer
+  regenerates (worksheets, shared strings, workbook, styles) are rebuilt; everything else
+  (drawings, media, themes, VBA) passes through byte-for-byte.
+- **Styles-passthrough fast path.** If `originalZip` is supplied AND no style was mutated
+  (`workbook.styles.dirty === false`, which `loadWorkbookFromSnapshot` sets up) AND the original had
+  `xl/styles.xml`, the writer reuses cells' original `s=` indices and skips rebuilding styles
+  entirely — the dominant path for value-only template fills. Touching any style (interning a new
+  `CellStyle` or a new numFmt code) flips `dirty` and takes the full rebuild.
+
+## Gotchas
+
+- **`parseExcelBuffer` takes an `ArrayBuffer`, not a Node `Buffer`.** A `Buffer` is a view over a
+  possibly-shared/pooled `ArrayBuffer`, so `buf.buffer` may contain other files' bytes. Always slice
+  to the view's own window: `buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength)`.
+  Passing `buf.buffer` directly can hand the parser garbage tail bytes.
+- **Formula text has NO leading `=` on the model path.** `sheet.set(ref, v, s, fmt, "SUM(A1:A5)")`,
+  not `"=SUM(...)"`. The barrel `set`/`setWithStyleIndex` write the string verbatim into `<f>`, and
+  OOXML formulas omit the `=`. The ONLY place a `=` is expected is `setCellValueCommand`, which
+  parses raw user input and strips it.
+- **The writer never recomputes; it relies on `fullCalcOnLoad`.** A formula written with a `null` or
+  stale cached value is correct in Excel (which recalcs on open) but wrong to any consumer that
+  trusts the cached `<v>` without recomputing. Call `recomputeAll(workbook)` before export when a
+  non-Excel reader will consume the file.
+- **Multi-sheet workbooks must share ONE `StyleRegistry`.** Cell styles are stored by index into a
+  single registry; sheets built against different registries render with each other's styles when
+  merged. Use `buildDataSheet(opts, styles)` / `new SheetModel(name, styles)` with one shared
+  registry, never `buildDataWorkbook` per sheet.
+- **Dates are Excel serial numbers plus a date number-format code, not `Date` objects.**
+  `buildDataWorkbook`'s `date` columns convert for you (from a `Date` via LOCAL `YYYY-MM-DD`, or from
+  an ISO string). On the raw model path, store the serial and set a date `numFmt`
+  (`parseToExcelSerial(iso)` from `@lotics/xlsx/numfmt_type` computes one; it interprets the ISO date
+  in UTC and anchors at 1899-12-30 for the Lotus-1900-leap-year bug).
+- **`maxRowsPerSheet` defaults to 10 000 and silently truncates.** For a full extract pass
+  `{ maxRowsPerSheet: Infinity }` and check `sheet.truncated`.
+- **`cell.value` is the formatted display string.** Compute on `cell.rawValue`/`cell.typedValue`;
+  never `parseFloat(cell.value)`.
+- **Legacy `.xls` is data-only.** No styles, merges, widths, charts, or images survive a `.xls`
+  read, and string/error formula results are dropped. If a downstream step needs those, it must come
+  from an `.xlsx`.
+- **Pivot `data.numFmtId` is a built-in numeric ID string, not a format code** — a custom currency
+  code won't apply in v1.
+- **`setWithStyleIndex` leaves `displayValue` empty** (it's the export fast path). If you read
+  `cell.displayValue` off such a cell before exporting, it's `""` until you call
+  `materializeDisplayValue(cell)`.

package/package.json

@@ -1,13 +1,14 @@
 {
   "name": "@lotics/xlsx",
-  "version": "0.1.5",
+  "version": "0.1.7",
   "type": "module",
   "exports": {
     ".": "./src/index.ts",
     "./*": "./src/*.ts"
   },
   "files": [
-    "src"
+    "src",
+    "AGENTS.md"
   ],
   "publishConfig": {
     "access": "public"
@@ -19,12 +20,13 @@
   },
   "dependencies": {
     "@formulajs/formulajs": "^4.5.6",
+    "@lotics/office-io": "^0.1.0",
     "fast-formula-parser": "^1.0.19",
     "fast-xml-parser": "^5.5.6",
     "fflate": "^0.8.2"
   },
   "devDependencies": {
     "exceljs": "^4.4.0",
-    "vitest": "^4.1.7"
+    "vitest": "^4.1.9"
   }
 }

package/src/biff8_reader.test.ts

@@ -1,8 +1,9 @@
 import { describe, it, expect } from "vitest";
 import { readFileSync } from "node:fs";
 import { fileURLToPath } from "node:url";
-import { parseBiff8, isOle2 } from "./biff8_reader";
-import { PasswordProtectedError } from "./types";
+import { parseBiff8 } from "./biff8_reader";
+import { loadWorkbookFromSnapshot } from "./workbook_model";
+import { isOle2, PasswordProtectedError } from "@lotics/office-io";
 
 // A synthetic BIFF8 (.xls) workbook: 250 data rows of Unicode strings + integer,
 // float and negative numbers. The Unicode strings push the shared-string table
@@ -28,30 +29,42 @@ describe("parseBiff8", () => {
 
   it("decodes Unicode text cells", () => {
     const s = parseBiff8(fixture).sheets[0];
-    expect(cell(s, 0, 0)).toBe("ID");
-    expect(cell(s, 0, 1)).toBe("Tên khách");
-    expect(cell(s, 0, 2)).toBe("Số tiền");
+    expect(cell(s, 1, 1)).toBe("ID");
+    expect(cell(s, 1, 2)).toBe("Tên khách");
+    expect(cell(s, 1, 3)).toBe("Số tiền");
   });
 
   it("decodes a string that falls past the SST CONTINUE boundary", () => {
     const s = parseBiff8(fixture).sheets[0];
-    expect(cell(s, 200, 1)).toBe("Khách 200 — Công ty TNHH số 200");
-    expect(cell(s, 249, 1)).toBe("Khách 249 — Công ty TNHH số 249");
+    expect(cell(s, 201, 2)).toBe("Khách 200 — Công ty TNHH số 200");
+    expect(cell(s, 250, 2)).toBe("Khách 249 — Công ty TNHH số 249");
   });
 
   it("decodes integer, float and negative numbers to the exact value", () => {
     const s = parseBiff8(fixture).sheets[0];
-    expect(cell(s, 2, 2)).toBe(2000); // integer (RK int)
-    expect(cell(s, 3, 2)).toBe(3000.5); // float
-    expect(cell(s, 5, 2)).toBe(-500); // negative
-    expect(cell(s, 200, 2)).toBe(-20000);
-    expect(cell(s, 249, 2)).toBe(249000.5);
+    expect(cell(s, 3, 3)).toBe(2000); // integer (RK int)
+    expect(cell(s, 4, 3)).toBe(3000.5); // float
+    expect(cell(s, 6, 3)).toBe(-500); // negative
+    expect(cell(s, 201, 3)).toBe(-20000);
+    expect(cell(s, 250, 3)).toBe(249000.5);
   });
 
   it("honors maxRowsPerSheet and flags truncation", () => {
     const s = parseBiff8(fixture, { maxRowsPerSheet: 10 }).sheets[0];
     expect(s.truncated).toBe(true);
-    expect(s.rows.every((r) => r.index < 10)).toBe(true);
+    expect(s.rows.every((r) => r.index <= 10)).toBe(true);
+  });
+
+  it("emits the ParsedSpreadsheet 1-based convention: A1 survives model load and export refs", () => {
+    // The regression that motivated normalization: 0-based BIFF indices fed
+    // loadWorkbookFromSnapshot's 1-based A1-ref construction, mapping the whole
+    // first column to invalid refs (colNumToLetters(0) === ""). The outcome
+    // that matters: the .xls's first cell lands at A1 in the model.
+    const wb = loadWorkbookFromSnapshot(parseBiff8(fixture));
+    const sheet = wb.sheets[0];
+    expect(sheet.cells.get("A1")?.value).toBe("ID");
+    expect(sheet.cells.get("B1")?.value).toBe("Tên khách");
+    expect(sheet.cells.has("0")).toBe(false);
   });
 
   it("rejects an OLE2 buffer with no Workbook stream", () => {
@@ -60,6 +73,29 @@ describe("parseBiff8", () => {
     bogus.set([0xd0, 0xcf, 0x11, 0xe0, 0xa1, 0xb1, 0x1a, 0xe1]);
     expect(() => parseBiff8(bogus)).toThrow();
   });
+
+  it("terminates on a crafted cyclic DIFAT chain instead of hanging (DoS guard)", () => {
+    // A DIFAT sector whose next-pointer references itself. Without a file-size
+    // bound + cycle detection on the DIFAT/FAT assembly, the walk runs its full
+    // guard (~1M iterations) and the FAT loop then balloons to billions of
+    // entries — a small crafted .xls hangs the event loop + exhausts memory.
+    // The reader must bound the walk to the sectors the file actually holds and
+    // fail fast. 2 KB with a valid 512-byte sector size so it passes the
+    // sector-size check and reaches the DIFAT walk (a shorter buffer would be
+    // rejected earlier).
+    const buf = new Uint8Array(2048);
+    buf.set([0xd0, 0xcf, 0x11, 0xe0, 0xa1, 0xb1, 0x1a, 0xe1]);
+    const dv = new DataView(buf.buffer);
+    dv.setUint16(30, 9, true); // sector shift → 512-byte sectors
+    dv.setUint16(32, 6, true); // mini sector shift → 64-byte mini sectors
+    dv.setUint32(48, 0xfffffffe, true); // firstDirSector = ENDOFCHAIN (empty dir)
+    dv.setUint32(60, 0xfffffffe, true); // firstMiniFatSector = ENDOFCHAIN
+    dv.setUint32(68, 0, true); // firstDifatSector = sector 0 …
+    dv.setUint32(1020, 0, true); // …whose last DIFAT entry points back to sector 0
+    // No workbook stream results, so a bounded parse throws quickly; an unbounded
+    // one would never reach this throw (it hangs/OOMs first).
+    expect(() => parseBiff8(buf)).toThrow();
+  });
 });
 
 // A date cell is a NUMBER record (Excel serial) whose XF points at a date
@@ -75,19 +111,19 @@ describe("parseBiff8 number formats", () => {
   };
 
   it("reads a date cell as an Excel serial carrying its date number-format", () => {
-    const c = cell(1, 0);
+    const c = cell(2, 1);
     expect(c?.typedValue).toBe(45672); // 2025-01-15
     expect(c?.numFmtCode).toBe("dd/mm/yyyy");
   });
 
   it("leaves a plain number with no number-format (General)", () => {
-    expect(cell(1, 1)?.typedValue).toBe(42);
-    expect(cell(1, 1)?.numFmtCode).toBeUndefined();
-    expect(cell(1, 2)?.typedValue).toBe(1234.5);
+    expect(cell(2, 2)?.typedValue).toBe(42);
+    expect(cell(2, 2)?.numFmtCode).toBeUndefined();
+    expect(cell(2, 3)?.typedValue).toBe(1234.5);
   });
 
   it("decodes a Unicode string cell", () => {
-    expect(cell(1, 3)?.typedValue).toBe("Ábc");
+    expect(cell(2, 4)?.typedValue).toBe("Ábc");
   });
 });
 

package/src/biff8_reader.ts

@@ -15,110 +15,11 @@
 
 import { emptySheet } from "./ooxml_sheet";
 import type { ParseOptions } from "./excel_parser";
-import { PasswordProtectedError } from "./types";
+import { readCfbStreams, PasswordProtectedError } from "@lotics/office-io";
 import type { ParsedSpreadsheet, ParsedSheet, ParsedRow, ParsedCell } from "./types";
 
-const OLE2_MAGIC = [0xd0, 0xcf, 0x11, 0xe0, 0xa1, 0xb1, 0x1a, 0xe1];
-const ENDOFCHAIN = 0xfffffffe;
-const FREESECT = 0xffffffff;
 const DEFAULT_MAX_ROWS = 10_000;
 
-/** True when the bytes begin with the OLE2 Compound File signature (legacy .xls
- *  or an encrypted OOXML package — both share this container). */
-export function isOle2(bytes: Uint8Array): boolean {
-  return bytes.length >= 8 && OLE2_MAGIC.every((b, i) => bytes[i] === b);
-}
-
-interface CfbStream {
-  name: string;
-  bytes: Uint8Array;
-}
-
-// ─── OLE2 / Compound File Binary container ──────────────────────────────────
-
-function readCfbStreams(bytes: Uint8Array): CfbStream[] {
-  const dv = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
-  const u16 = (o: number) => dv.getUint16(o, true);
-  const u32 = (o: number) => dv.getUint32(o, true);
-
-  const sectorSize = 1 << u16(30);
-  const miniSectorSize = 1 << u16(32);
-  const firstDirSector = u32(48);
-  const miniCutoff = u32(56);
-  const firstMiniFatSector = u32(60);
-  const firstDifatSector = u32(68);
-
-  if (sectorSize < 512 || sectorSize > 1 << 20) {
-    throw new Error("Unsupported .xls: invalid OLE2 sector size");
-  }
-  const sectorOffset = (n: number) => sectorSize * (n + 1);
-
-  // DIFAT: first 109 entries live in the header; the rest chain through sectors.
-  const difat: number[] = [];
-  for (let i = 0; i < 109; i++) {
-    const v = u32(76 + i * 4);
-    if (v !== FREESECT) difat.push(v);
-  }
-  let ds = firstDifatSector;
-  const entriesPerSector = sectorSize / 4;
-  for (let guard = 0; ds !== ENDOFCHAIN && ds !== FREESECT && guard < 1 << 20; guard++) {
-    const base = sectorOffset(ds);
-    for (let i = 0; i < entriesPerSector - 1; i++) {
-      const v = u32(base + i * 4);
-      if (v !== FREESECT) difat.push(v);
-    }
-    ds = u32(base + (entriesPerSector - 1) * 4);
-  }
-
-  // FAT: sector-allocation chains, assembled from the DIFAT-listed FAT sectors.
-  const fat: number[] = [];
-  for (const fatSector of difat) {
-    const base = sectorOffset(fatSector);
-    for (let i = 0; i < entriesPerSector; i++) fat.push(u32(base + i * 4));
-  }
-
-  const readChain = (src: Uint8Array, start: number, size: number | null, ss: number, offOf: (n: number) => number, chain: number[]): Uint8Array => {
-    // A chain can't yield more sectors than the source holds; a longer walk is a
-    // cyclic/corrupt FAT (a crafted .xls DoS) — cap the read at the source size.
-    const maxSectors = Math.floor(src.length / ss) + 2;
-    const parts: Uint8Array[] = [];
-    let s = start;
-    for (let guard = 0; s !== ENDOFCHAIN && s !== FREESECT && s < chain.length && guard < maxSectors; guard++) {
-      const o = offOf(s);
-      parts.push(src.subarray(o, o + ss));
-      s = chain[s];
-    }
-    const merged = concat(parts);
-    return size != null ? merged.subarray(0, size) : merged;
-  };
-  const readBig = (start: number, size: number | null) => readChain(bytes, start, size, sectorSize, sectorOffset, fat);
-
-  // Directory: 128-byte entries in a FAT chain.
-  const dirBytes = readBig(firstDirSector, null);
-  const ddv = new DataView(dirBytes.buffer, dirBytes.byteOffset, dirBytes.byteLength);
-  interface DirEntry { name: string; type: number; start: number; size: number }
-  const dir: DirEntry[] = [];
-  for (let off = 0; off + 128 <= dirBytes.length; off += 128) {
-    const nameLen = ddv.getUint16(off + 64, true);
-    const type = ddv.getUint8(off + 66);
-    if (nameLen < 2 || type === 0) continue;
-    const name = utf16le(dirBytes.subarray(off, off + nameLen - 2));
-    dir.push({ name, type, start: ddv.getUint32(off + 116, true), size: Number(ddv.getBigUint64(off + 120, true)) });
-  }
-
-  // Mini stream (root entry's chain) holds streams below the cutoff, in mini sectors.
-  const root = dir.find((e) => e.type === 5);
-  const miniStream = root ? readBig(root.start, root.size) : new Uint8Array(0);
-  const miniFatBytes = readBig(firstMiniFatSector, null);
-  const miniFat: number[] = [];
-  for (let i = 0; i + 4 <= miniFatBytes.length; i += 4) miniFat.push(miniFatBytes[i] | (miniFatBytes[i + 1] << 8) | (miniFatBytes[i + 2] << 16) | (miniFatBytes[i + 3] << 24));
-  // Mini sectors index into the root mini-stream container, not the raw file.
-  const readMini = (start: number, size: number) => readChain(miniStream, start, size, miniSectorSize, (n) => n * miniSectorSize, miniFat);
-  const readStream = (e: DirEntry) => (e.size >= miniCutoff ? readBig(e.start, e.size) : readMini(e.start, e.size));
-
-  return dir.filter((e) => e.type === 2).map((e) => ({ name: e.name, bytes: readStream(e) }));
-}
-
 // ─── BIFF record stream ─────────────────────────────────────────────────────
 
 const REC = {
@@ -336,7 +237,7 @@ function parseSheet(stream: Uint8Array, start: number, name: string, g: Globals,
     const r = grid.get(ri);
     if (!r) continue;
     const cells = [...r.values()].sort((a, b) => a.column - b.column);
-    rows.push({ index: ri, height: sheet.defaultRowHeight, cells, hidden: false });
+    rows.push({ index: ri + 1, height: sheet.defaultRowHeight, cells, hidden: false });
   }
   sheet.rows = rows;
   sheet.totalRowCount = maxRow + 1;
@@ -346,14 +247,19 @@ function parseSheet(stream: Uint8Array, start: number, name: string, g: Globals,
 
 // ─── Cell builders ──────────────────────────────────────────────────────────
 
+// BIFF stores 0-based row/col on disk; ParsedSpreadsheet is 1-based (the OOXML
+// parser's convention, and what loadWorkbookFromSnapshot's A1-ref construction
+// requires — colNumToLetters(0) is "", so a 0-based column would silently land
+// every first-column cell at an invalid ref). Builders receive the raw BIFF
+// column and normalize at this single escape point.
 function textCell(column: number, text: string): ParsedCell {
-  return { column, value: text, typedValue: text, content: { type: "plain", text }, style: {} };
+  return { column: column + 1, value: text, typedValue: text, content: { type: "plain", text }, style: {} };
 }
 
 function numberCell(column: number, num: number, numFmt: string | undefined): ParsedCell {
   const value = Number.isFinite(num) ? String(num) : "";
   return {
-    column, value, rawValue: num, typedValue: num,
+    column: column + 1, value, rawValue: num, typedValue: num,
     numFmtCode: numFmt && numFmt !== "General" ? numFmt : undefined,
     content: { type: "plain", text: value }, style: {},
   };
@@ -367,7 +273,7 @@ function formulaCell(column: number, body: Uint8Array, numFmt: string | undefine
     const kind = body[6];
     if (kind === 1) {
       const b = body[8] !== 0;
-      return { column, value: b ? "TRUE" : "FALSE", typedValue: b, content: { type: "plain", text: b ? "TRUE" : "FALSE" }, style: {} };
+      return { column: column + 1, value: b ? "TRUE" : "FALSE", typedValue: b, content: { type: "plain", text: b ? "TRUE" : "FALSE" }, style: {} };
     }
     // string (resolved by a following STRING record — none in scope), error, or
     // empty: surface nothing rather than a wrong value.
@@ -412,21 +318,6 @@ function readUnicodeShort(b: Uint8Array, offset: number, lenBytes: 1 | 2): strin
   return s;
 }
 
-function utf16le(b: Uint8Array): string {
-  let s = "";
-  for (let i = 0; i + 1 < b.length; i += 2) s += String.fromCharCode(b[i] | (b[i + 1] << 8));
-  return s;
-}
-
-function concat(parts: Uint8Array[]): Uint8Array {
-  let total = 0;
-  for (const p of parts) total += p.length;
-  const out = new Uint8Array(total);
-  let o = 0;
-  for (const p of parts) { out.set(p, o); o += p.length; }
-  return out;
-}
-
 /** Built-in number-format strings we care about (dates → so serials render as
  *  dates downstream; currency/decimals are otherwise "General"). */
 function builtinFormat(id: number): string {

package/src/excel_parser.test.ts

@@ -655,7 +655,7 @@ EST</v></c></row>
     expect(result.sheets[0].rows[0].cells[0].style.fontStrike).toBe(true);
   });
 
-  test("skips hidden rows", async () => {
+  test("keeps hidden rows, flagged hidden", async () => {
     const workbook = new ExcelJS.Workbook();
     const ws = workbook.addWorksheet("HiddenRows");
 
@@ -669,11 +669,16 @@ EST</v></c></row>
 
     const result = await parseExcelFile("https://example.com/test.xlsx");
 
-
-    const rowIndices = result.sheets[0].rows.map((r) => r.index);
+    const rows = result.sheets[0].rows;
+    const rowIndices = rows.map((r) => r.index);
+    // Hidden rows are real data — kept (so they round-trip and cross-sheet refs
+    // resolve) but flagged hidden so renderers can skip them.
     expect(rowIndices).toContain(1);
-    expect(rowIndices).not.toContain(2);
+    expect(rowIndices).toContain(2);
     expect(rowIndices).toContain(3);
+    expect(rows.find((r) => r.index === 1)?.hidden).toBe(false);
+    expect(rows.find((r) => r.index === 2)?.hidden).toBe(true);
+    expect(rows.find((r) => r.index === 3)?.hidden).toBe(false);
   });
 
   test("marks hidden columns", async () => {

package/src/excel_parser.ts

@@ -16,7 +16,8 @@ import {
   extractWorkbookPivotCaches,
 } from "./ooxml_pivot";
 import { parseVmlDrawing } from "./ooxml_shape";
-import { parseBiff8, isOle2 } from "./biff8_reader";
+import { parseBiff8 } from "./biff8_reader";
+import { isOle2 } from "@lotics/office-io";
 import type { ParsedSpreadsheet, ParsedSheet, PrintTitles } from "./types";
 import type { ParsedChart } from "./ooxml_chart_types";
 import type { ParsedPivotCache, ParsedPivotTable } from "./ooxml_pivot_types";
@@ -38,7 +39,6 @@ export type {
   BorderStyle,
   MergedCellRange,
 } from "./types";
-export { PasswordProtectedError } from "./types";
 
 export type {
   ParsedConditionalFormat,

package/src/index.ts

@@ -35,7 +35,6 @@ export type {
   HeaderFooter,
 } from "./types";
 
-export { PasswordProtectedError } from "./types";
 
 // -----------------------------------------------------------------------------
 // Workbook model & cell storage
@@ -100,6 +99,14 @@ export {
 
 export type { ParseOptions } from "./excel_parser";
 
+// -----------------------------------------------------------------------------
+// Errors
+// -----------------------------------------------------------------------------
+
+// Single definition lives in @lotics/office-io (the .doc reader throws the same
+// class); the barrel keeps it in the public API for external catch sites.
+export { PasswordProtectedError } from "@lotics/office-io";
+
 export { exportWorkbook } from "./xlsx_writer";
 
 // -----------------------------------------------------------------------------

package/src/ooxml_zip.ts

@@ -1,5 +1,5 @@
 import { unzipSync } from "fflate";
-import { PasswordProtectedError } from "./types";
+import { PasswordProtectedError } from "@lotics/office-io";
 
 // OLE2 Compound Document magic bytes — used by legacy .xls and encrypted .xlsx
 const OLE2_MAGIC = [0xd0, 0xcf, 0x11, 0xe0, 0xa1, 0xb1, 0x1a, 0xe1];

package/src/types.ts

@@ -227,13 +227,3 @@ export interface MergedCellRange {
   endCol: number;
 }
 
-// =============================================================================
-// Errors
-// =============================================================================
-
-export class PasswordProtectedError extends Error {
-  constructor() {
-    super("This file is password-protected and cannot be previewed");
-    this.name = "PasswordProtectedError";
-  }
-}