rtf-codec 4.0.9 → 4.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (8) hide show
  1. package/README.md +4 -3
  2. package/dist/codepage-dbcs.cjs +523 -0
  3. package/dist/codepage-dbcs.d.cts +5 -0
  4. package/dist/codepage-dbcs.d.ts +5 -0
  5. package/dist/codepage-dbcs.js +521 -0
  6. package/dist/codepage.cjs +28 -1
  7. package/dist/codepage.js +28 -1
  8. package/package.json +1 -1
package/README.md CHANGED
@@ -72,6 +72,8 @@ Everything here is implemented against Microsoft's own **Rich Text Format (RTF)
72
72
 
73
73
  The code-page tables in `src/codepage.ts` were **generated, not transcribed**: each is `bytes([b]).decode(codec)` over `0x80..0xFF` from Python's own codec library, verified byte-for-byte against it, because a hand-typed 128-entry table is exactly where one transposed character hides until a real document decodes wrong.
74
74
 
75
+ The five East Asian double-byte code-page tables in `src/codepage-dbcs.ts` (932 Shift-JIS, 936 GBK/GB2312, 949 UHC/Hangul, 950 Big5, 1361 Johab) were generated the same way, at larger scale: `scripts/generate-dbcs-tables.py` decodes every lead-byte/trail-byte pair through Python's own `cp932`/`cp936`/`cp949`/`cp950`/`cp1361` codecs (<https://docs.python.org/3/library/codecs.html#standard-encodings>) — the exact Windows code pages RTF's own `\ansicpgN`/`\cpgN` name by that number, not a nearby web-oriented substitute. That script's own header comment has the full citation, including which pages were cross-checked against a second, independent decoder (Node's ICU-backed `TextDecoder`, for the four of the five pages the WHATWG Encoding Standard also defines) and where the two genuinely diverge.
76
+
75
77
  ## Architecture
76
78
 
77
79
  Five stages, each its own module, each testable on its own:
@@ -84,7 +86,7 @@ Five stages, each its own module, each testable on its own:
84
86
  | Read | `src/read.ts` | The destination/group state machine that turns the token stream into a `ContentDocument`. |
85
87
  | Write | `src/write.ts` | The inverse: mints the header tables from what the document actually uses, then emits a body that references them by index. |
86
88
 
87
- Supporting modules: `src/codepage.ts` (byte-to-character tables and the `\ansicpgN`/`\fcharsetN`/`\cpgN` precedence), `src/base64.ts` (hex and base64 conversion for picture and object payloads), `src/units.ts` (twips, half-points, pixels), `src/list-id.ts` (the opaque `numId` grammar), `src/constructs.ts` (the fidelity-construct descriptor shapes and the DTTM bit field), `src/cell-format.ts` (the `<celldef>` border, shading, and merge production), `src/embedded-object.ts` (the `\object`/`\objdata` payload -- JSON in, real `[MS-CFB]` compound file out, via `archive-codec`; see [Embedded objects](#embedded-objects)), `src/diagnostics.ts` (the three-tier diagnostic policy).
89
+ Supporting modules: `src/codepage.ts` (byte-to-character tables and the `\ansicpgN`/`\fcharsetN`/`\cpgN` precedence, plus the lead-byte state machine the five DBCS pages in `src/codepage-dbcs.ts` need), `src/base64.ts` (hex and base64 conversion for picture and object payloads), `src/units.ts` (twips, half-points, pixels), `src/list-id.ts` (the opaque `numId` grammar), `src/constructs.ts` (the fidelity-construct descriptor shapes and the DTTM bit field), `src/cell-format.ts` (the `<celldef>` border, shading, and merge production), `src/embedded-object.ts` (the `\object`/`\objdata` payload -- JSON in, real `[MS-CFB]` compound file out, via `archive-codec`; see [Embedded objects](#embedded-objects)), `src/diagnostics.ts` (the three-tier diagnostic policy).
88
90
 
89
91
  ### The reader is the specification's own model, literally
90
92
 
@@ -111,7 +113,7 @@ On the way out, **every non-ASCII character leaves as `\uN`** with a one-charact
111
113
  | Groups, destinations, `{\*` ignorable destinations | Yes — per the spec's own reader conventions |
112
114
  | Control words, control symbols, `\'hh`, `\binN` | Yes |
113
115
  | `\uN` / `\ucN` with ANSI fallback skipping, `\upr`/`\ud` | Yes |
114
- | Code pages | `\ansi`/`\mac`/`\pc`/`\pca`, `\ansicpgN`, per-font `\cpgN`/`\fcharsetN`; the Windows, OEM and Macintosh single-byte pages, plus UTF-8 |
116
+ | Code pages | `\ansi`/`\mac`/`\pc`/`\pca`, `\ansicpgN`, per-font `\cpgN`/`\fcharsetN`; the Windows, OEM and Macintosh single-byte pages, the five East Asian DBCS pages (932 Shift-JIS, 936 GBK/GB2312, 949 UHC/Hangul, 950 Big5, 1361 Johab), plus UTF-8 |
115
117
  | `\fonttbl` | Face name, family keyword, per-font code page |
116
118
  | `\colortbl` | RGB, including a theme colour's own literal RGB; index 0 is the auto colour |
117
119
  | `\stylesheet` | Paragraph style names and heading levels (`\outlinelevelN` or a built-in `heading N` name) |
@@ -152,7 +154,6 @@ Each of these is reported through a diagnostic rather than dropped silently —
152
154
  | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
153
155
  | Headers, footers, footnotes, endnotes, annotations | `ContentDocument`'s flat form has no page-furniture or note position for them. A footnote's real home is `document-schema.js`'s tree-only `definitions` table, which a codec producing the flat form cannot reach. |
154
156
  | Content controls beyond RTF's own form-field vocabulary (richText, comboBox, date, picture, repeatingSection, button, index, group) | **RTF 1.9.1 specifies nothing for these.** It predates OOXML's `w:sdt`: its "Custom XML Tags" (`\xmlopen`/`\xmlclose`) are a bare namespace/name tag with no type, lock, alias or value, and `\*\datastore` is an opaque blob whose "format ... is unknown to RTF" by the spec's own words. `\*\formfield` (see the Scope table above) is the one real analogue RTF has, and covers plainText/checkbox/dropDown only. |
155
- | East Asian DBCS code pages (932, 936, 949, 950, 1361) | Each needs a ~20k-entry table and its own lead-byte state machine. A document declaring one decodes through cp1252 and says so. |
156
157
  | Code page 42 (`SYMBOL_CHARSET`) | Not an encoding: its bytes are glyph indices into whichever symbol font the run names, so there is no correct Unicode for them without that font's own cmap. |
157
158
  | Metafile and bitmap pictures (`\wmetafileN`, `\emfblip`, `\dibitmapN`, `\wbitmapN`, `\macpict`) | `ContentImageBlock` carries PNG and JPEG only. |
158
159
  | A picture with no stated size | `ContentImageBlock` requires a positive width and height, and deriving them from the payload would need an image decoder this package deliberately does not carry. |