@gmb/bitmark-parser 7.5.0 → 7.7.0

package/README.md

@@ -286,6 +286,48 @@ often than it may be (`|?a|?b|` — the later value wins), and
 `text-tag-value-invalid` when a value does not fit its format but is kept
 (`|symbol:…|width:wide|`).
 
+#### convertWithDetails(input: string, options?: ConvertWithDetailsOptions): ConvertResult
+
+`convert`, returning `{ output, …extras }` — the same conversion, run once,
+with each extra asked for by a flag in the options. `output` is exactly what
+`convert` returns, and it throws exactly as `convert` does. Future extras are
+one more flag and one more field each.
+
+- `bitSpans` — `true` adds `result.bitSpans`, where each top-level bit's text
+  landed in the output, for linking a source pane to an output pane bit by
+  bit:
+
+```ts
+const { output, bitSpans } = convertWithDetails(source, {
+  inputFormat: "bitmark",
+  outputFormat: "html",
+  bitSpans: true,
+});
+// bitSpans = { positionEncoding: "utf-16", spans: [{ index, start, end }, …] }
+for (const { index, start, end } of bitSpans!.spans) {
+  console.log(index, output.slice(start, end));
+}
+```
+
+- `index` is the bit's position in the **input**: its `splitBits` index for
+  bitmark input, its entry index in the top-level array for JSON input. An
+  entry that is not a bit gets no span, so the indexes can have gaps.
+- `start` / `end` are **the bit's text**, tight: never the separators
+  between bits (`\n`, blank lines, `,`) nor the JSON array's `[` / `]`. They
+  are counted in `positionEncoding` — `"utf-16"` by default, so
+  `output.slice(start, end)` is the bit — or `"utf-8"` bytes.
+- Spans are reported for the document outputs: `bitmark`, `json` (compact or
+  pretty), `text`, and the markup formats (`html`, `xml`, …). In `bitmark`
+  output they equal `splitBits(output)`'s `start` / `end`.
+- A bit that writes nothing (an empty or `_error` bit in `text`) gets a
+  zero-width span where its text would start.
+- Only top-level bits get spans. For any other output (`lex`, `ast`,
+  `semantic-tokens`, `diagnostics`, a mapping report) `bitSpans` is present
+  with `spans: []`; without the flag it is absent.
+
+The CLI's `--bit-spans FILE` and the daemon's `convert` option
+`bitSpans: true` (reply field `bitSpans`) return the same object.
+
 #### canonicalize(input: string, options?: CanonicalizeOptions): string
 
 Re-emit input in its own format in canonical form — the single same-format surface. `mode`: `"optimized"` (default, omit natural defaults) or `"full"` (all keys).
@@ -449,7 +491,14 @@ included so the alignment is complete — `status`
 `ops` (a patch turning the A bit into the B bit) and `revert`, the parser
 issue codes gained or lost, and the rendered `hunks`.
 
-#### countBits(input: string): number / splitBits(input: string): BitSlice[]
+#### countBits(input: string): number / splitBits(input: string, options?: SplitBitsOptions): BitSlice[]
+
+`splitBits` returns one `{ index, start, end, byteStart, byteEnd, source }`
+per bit. `start` / `end` locate the bit's text, `source`, counted in
+`options.positionEncoding` — `"utf-16"` by default, so
+`input.slice(start, end) === source` — or `"utf-8"` bytes. `byteStart` /
+`byteEnd` are UTF-8 byte offsets that tile the document (each bit runs to the
+next bit's start), for chunking.
 
 Count or split the bits of a bitmark/JSON input without full parsing. A JSON
 input may be an array or a single value, each entry a `{ bit: {…} }` envelope
@@ -1086,6 +1135,10 @@ bitmark convert input.bitmark --output-format semantic-tokens --tokens-layout ab
 # Dump the lexer token stream as JSON (debug tooling; bitmark input only)
 bitmark convert input.bitmark --input-format bitmark --output-format lex --pretty
 
+# Also write where each bit landed in the output, {positionEncoding, spans},
+# to its own file (offsets into this run's output; --position-encoding applies)
+bitmark convert input.bitmark --output-format html -o out.html --bit-spans spans.json
+
 # Breakscape / unbreakscape text
 bitmark breakscape input.txt
 bitmark breakscape input.txt --format plainText --location tag