pyOpenVBA 3.3.0__tar.gz → 3.5.0__tar.gz

{pyopenvba-3.3.0 → pyopenvba-3.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: pyOpenVBA
-Version: 3.3.0
+Version: 3.5.0
 Summary: Read and write VBA macros inside Excel, Word, and PowerPoint files in pure Python, no dependencies.
 Project-URL: Homepage, https://github.com/WilliamSmithEdward/pyOpenVBA
 Project-URL: Repository, https://github.com/WilliamSmithEdward/pyOpenVBA
@@ -326,6 +326,120 @@ modules, `.cls` for class modules and code-behind.
 
 ---
 
+## Creating and editing a UserForm's design
+
+A form's *code* is a module like any other. Its *design* -- which controls
+exist, how they nest, and what their properties are -- lives in separate
+streams that no module source carries. `forms()` reads them, with no
+Office installed:
+
+```python
+import pyopenvba
+
+with pyopenvba.ExcelFile("book.xlsm") as wb:
+    for form in wb.forms():
+        print(form.name, len(form.walk()), "controls")
+        for control in form.walk():
+            print(f"  {control.name:<16} {control.kind:<22} "
+                  f"{control.properties()}")
+```
+
+And edits them:
+
+```python
+with pyopenvba.ExcelFile("book.xlsm") as wb:
+    form = wb.forms()[0]
+    form.control("OkButton").set_property("Caption", "Save")
+    form.control("NameBox").set_property("MaxLength", 40)
+    form.add_control("Label", "Hint", left=12, top=120, width=200)
+    form.remove_control("OldCheckbox")
+    wb.save()
+```
+
+Containers work too. Each gets a storage of its own, and removing one
+takes its children with it:
+
+```python
+form.add_control("Frame", "Shipping", left=12, top=160, width=200, height=80)
+form.add_control("OptionButton", "Ground", container="Shipping")
+form.remove_control("OldFrame")     # and everything inside it
+```
+
+A `MultiPage` arrives with the two pages Excel gives it, and pages are
+added and removed through it, because a page is also a *tab*:
+
+```python
+form.add_control("MultiPage", "Wizard", left=12, top=40, width=300, height=200)
+form.add_page("Wizard", name="Review", caption="Review && confirm")
+form.add_control("Label", "Summary", container="Review")
+form.remove_page("Page2", multipage="Wizard")
+```
+
+Page names are scoped to their MultiPage rather than to the form, which is
+why `remove_page` takes an optional `multipage` to disambiguate.
+
+And a form can be built from nothing -- `add_form` creates the designer
+storage and the code-behind module together, because a storage without a
+module is not a component the host will show:
+
+```python
+with pyopenvba.ExcelFile("book.xlsm") as wb:
+    form = wb.add_form("Wizard", caption="Setup", width=300, height=200)
+    form.add_control("Label", "Prompt", left=12, top=12, width=200)
+    form.add_control("TextBox", "Answer", left=12, top=40, width=200)
+    form.add_control("CommandButton", "Ok", left=12, top=80)
+    wb.set_module("Wizard", "Private Sub Ok_Click()\r\n    Me.Hide\r\nEnd Sub\r\n")
+    wb.save()
+```
+
+Geometry is in points, the unit the designer shows. `set_property(name,
+None)` clears a property, which is how a control goes back to inheriting
+the default.
+
+Or from the command line:
+
+```bash
+python -m pyopenvba forms book.xlsm
+```
+
+```
+FrmNested  (13 controls)
+  TopLabel             MSForms.Label          id=1    set=0x00000028
+  GroupBox             MSForms.Frame          id=2    set=0x0c0a0c48
+    OptOne               MSForms.OptionButton   id=3    set=0x0000000180c00146
+  Pages                MSForms.MultiPage      id=6    set=0x0c000c48
+    (unnamed)            MSForms.TabStrip       id=7    set=0x00fa8031
+    Page1                MSForms.Form           id=8    set=0x0c000c48
+      PageOneCheck         MSForms.CheckBox       id=10   set=0x0000000080c00146
+```
+
+Containers nest: a `Frame`'s children and a `MultiPage`'s `Page`s live in
+storages of their own, and MSForms sites an unnamed `TabStrip` beside the
+pages. `walk()` flattens the tree depth-first; `form.controls` gives just
+the top level.
+
+### Only what the developer set
+
+MSForms writes a property into a control's record **only when it differs
+from that control's default**, so `properties()` returns the set the
+developer chose -- not every property the control has. That is not
+something a live host can tell you: a sited control reports inherited,
+default, and chosen values indistinguishably.
+
+`properties_set` is the same information as a raw bit mask, if you want to
+diff two files without comparing names. Read a bit index together with
+`property_mask_width`: MorphData controls (`TextBox`, `ListBox`,
+`ComboBox`, `CheckBox`, `OptionButton`, `ToggleButton`) carry an 8-byte
+mask and everything else carries 4.
+
+Writing is lossless. An unedited form saves back byte for byte, because
+alignment padding, string bytes, pictures, and anything the property
+tables do not model are all replayed as they were read. If a form's
+streams do not reconcile, this raises `FormParseError` rather than
+returning a partly-guessed control list.
+
+---
+
 ## Supported formats
 
 ### Excel
@@ -432,12 +546,9 @@ wb.save(allow_invalidate_signature=True)
 
 ## What's out of scope
 
-This library is intentionally focused on **module source code**. The
+A project's code and its UserForm designs are read and written. The
 following are preserved byte-for-byte but not interpreted:
 
-- UserForm **layout** (controls, properties, positions). Editing the
-  **code-behind** of a UserForm works fine; editing the design surface
-  does not.
 - VBA project password decryption / re-encryption.
 - Re-signing digitally signed projects.
 - ActiveX license editing.

{pyopenvba-3.3.0 → pyopenvba-3.5.0}/README.md

@@ -290,6 +290,120 @@ modules, `.cls` for class modules and code-behind.
 
 ---
 
+## Creating and editing a UserForm's design
+
+A form's *code* is a module like any other. Its *design* -- which controls
+exist, how they nest, and what their properties are -- lives in separate
+streams that no module source carries. `forms()` reads them, with no
+Office installed:
+
+```python
+import pyopenvba
+
+with pyopenvba.ExcelFile("book.xlsm") as wb:
+    for form in wb.forms():
+        print(form.name, len(form.walk()), "controls")
+        for control in form.walk():
+            print(f"  {control.name:<16} {control.kind:<22} "
+                  f"{control.properties()}")
+```
+
+And edits them:
+
+```python
+with pyopenvba.ExcelFile("book.xlsm") as wb:
+    form = wb.forms()[0]
+    form.control("OkButton").set_property("Caption", "Save")
+    form.control("NameBox").set_property("MaxLength", 40)
+    form.add_control("Label", "Hint", left=12, top=120, width=200)
+    form.remove_control("OldCheckbox")
+    wb.save()
+```
+
+Containers work too. Each gets a storage of its own, and removing one
+takes its children with it:
+
+```python
+form.add_control("Frame", "Shipping", left=12, top=160, width=200, height=80)
+form.add_control("OptionButton", "Ground", container="Shipping")
+form.remove_control("OldFrame")     # and everything inside it
+```
+
+A `MultiPage` arrives with the two pages Excel gives it, and pages are
+added and removed through it, because a page is also a *tab*:
+
+```python
+form.add_control("MultiPage", "Wizard", left=12, top=40, width=300, height=200)
+form.add_page("Wizard", name="Review", caption="Review && confirm")
+form.add_control("Label", "Summary", container="Review")
+form.remove_page("Page2", multipage="Wizard")
+```
+
+Page names are scoped to their MultiPage rather than to the form, which is
+why `remove_page` takes an optional `multipage` to disambiguate.
+
+And a form can be built from nothing -- `add_form` creates the designer
+storage and the code-behind module together, because a storage without a
+module is not a component the host will show:
+
+```python
+with pyopenvba.ExcelFile("book.xlsm") as wb:
+    form = wb.add_form("Wizard", caption="Setup", width=300, height=200)
+    form.add_control("Label", "Prompt", left=12, top=12, width=200)
+    form.add_control("TextBox", "Answer", left=12, top=40, width=200)
+    form.add_control("CommandButton", "Ok", left=12, top=80)
+    wb.set_module("Wizard", "Private Sub Ok_Click()\r\n    Me.Hide\r\nEnd Sub\r\n")
+    wb.save()
+```
+
+Geometry is in points, the unit the designer shows. `set_property(name,
+None)` clears a property, which is how a control goes back to inheriting
+the default.
+
+Or from the command line:
+
+```bash
+python -m pyopenvba forms book.xlsm
+```
+
+```
+FrmNested  (13 controls)
+  TopLabel             MSForms.Label          id=1    set=0x00000028
+  GroupBox             MSForms.Frame          id=2    set=0x0c0a0c48
+    OptOne               MSForms.OptionButton   id=3    set=0x0000000180c00146
+  Pages                MSForms.MultiPage      id=6    set=0x0c000c48
+    (unnamed)            MSForms.TabStrip       id=7    set=0x00fa8031
+    Page1                MSForms.Form           id=8    set=0x0c000c48
+      PageOneCheck         MSForms.CheckBox       id=10   set=0x0000000080c00146
+```
+
+Containers nest: a `Frame`'s children and a `MultiPage`'s `Page`s live in
+storages of their own, and MSForms sites an unnamed `TabStrip` beside the
+pages. `walk()` flattens the tree depth-first; `form.controls` gives just
+the top level.
+
+### Only what the developer set
+
+MSForms writes a property into a control's record **only when it differs
+from that control's default**, so `properties()` returns the set the
+developer chose -- not every property the control has. That is not
+something a live host can tell you: a sited control reports inherited,
+default, and chosen values indistinguishably.
+
+`properties_set` is the same information as a raw bit mask, if you want to
+diff two files without comparing names. Read a bit index together with
+`property_mask_width`: MorphData controls (`TextBox`, `ListBox`,
+`ComboBox`, `CheckBox`, `OptionButton`, `ToggleButton`) carry an 8-byte
+mask and everything else carries 4.
+
+Writing is lossless. An unedited form saves back byte for byte, because
+alignment padding, string bytes, pictures, and anything the property
+tables do not model are all replayed as they were read. If a form's
+streams do not reconcile, this raises `FormParseError` rather than
+returning a partly-guessed control list.
+
+---
+
 ## Supported formats
 
 ### Excel
@@ -396,12 +510,9 @@ wb.save(allow_invalidate_signature=True)
 
 ## What's out of scope
 
-This library is intentionally focused on **module source code**. The
+A project's code and its UserForm designs are read and written. The
 following are preserved byte-for-byte but not interpreted:
 
-- UserForm **layout** (controls, properties, positions). Editing the
-  **code-behind** of a UserForm works fine; editing the design surface
-  does not.
 - VBA project password decryption / re-encryption.
 - Re-signing digitally signed projects.
 - ActiveX license editing.

{pyopenvba-3.3.0 → pyopenvba-3.5.0}/docs/architecture.md

@@ -244,14 +244,17 @@ top-level helper `pyopenvba.pull_access(database, dest_dir)` mirrors
 
 ### Why no write path
 
-Access stores compiled VBA p-code (the `rU@` + `CAFE` rows) separately
-from the OVBA source cache. The compiled p-code is authoritative for
-the Access GUI; mutations to the source cache do not survive reload
-because Access never recompiles from the cache. We could not locate a
-recompile trigger. A production-quality writer would require a full
-VBA7 p-code assembler (instruction emission, identifier table
-re-indexing, jump fixup, MSysObjects long-value chunk reflow, ACE
-page allocator parity). That is out of scope. See
+Access stores compiled VBA p-code (the `CAFE` rows) separately from the
+OVBA source cache, and executes an `__SRP_*` compiled cache in
+preference to either, so mutations to the source alone do not change
+behaviour. Research later found the lever -- dropping those cache rows
+makes a rewritten module take effect -- and got as far as rewriting
+procedure bodies, including declarations, with output byte-identical to
+Microsoft's compiler. It did **not** get to creating, renaming or
+deleting a module, which is what a useful writer would need. That work
+is parked and unsupported; a production-quality writer would still
+require the `FuncDefn` declaration tables and ACE page allocator parity.
+See
 [docs/msaccess_lessons_learned.md](msaccess_lessons_learned.md) for
 the empirical results matrix and the reasoning in full.
 
@@ -296,7 +299,7 @@ removes signatures.
 | Extension        | Container              | VBA entry path         |
 |------------------|------------------------|------------------------|
 | `.pptm`, `.potm` | ZIP (OOXML)            | `ppt/vbaProject.bin`   |
-| `.ppt`           | raw CFB (PPT 97)       | (whole file is CFB)    |
+| `.ppt`           | CFB (PPT 97)           | embedded in `PowerPoint Document` |
 | anything else    | n/a                    | `UnsupportedFormatError` |
 
 For the ZIP case, the VBA project is at the fixed path
@@ -304,8 +307,17 @@ For the ZIP case, the VBA project is at the fixed path
 every other ZIP entry is preserved byte-for-byte including its
 compression method, external attributes, create system, and timestamp.
 
-For `.xls`, the entire file *is* the CFB; `cfb.to_bytes()` is written
-straight to the output path.
+For `.xls` and `.doc`, the entire file *is* the CFB; `cfb.to_bytes()`
+is written straight to the output path.
+
+`.ppt` is the exception among the legacy containers. Its root holds no
+VBA storage at all: the project is a whole CFB, zlib-deflated, inside an
+`ExOleObjStg` record of the `PowerPoint Document` stream, found through
+the persist chain (`Current User` -> `UserEditAtom` -> `PersistDirectoryAtom`).
+`_ppt_container.py` extracts it on open and splices it back on save,
+shifting every absolute offset past the resized record. The two hooks
+`VBAHostFile._vba_cfb_bytes` / `._container_bytes` are the seam; they are
+identities for every other format.
 
 If a `.xlsm` exists but does not contain `xl/vbaProject.bin` (no VBA
 project has ever been created), `ExcelFile` raises a structured
@@ -313,6 +325,102 @@ project has ever been created), `ExcelFile` raises a structured
 
 ---
 
+## 5a. UserForm designer streams
+
+A form's design lives beside the VBA storage, not inside it: a root
+storage named for the form, holding `f` (the sites: which controls, in
+what order), `o` (each control's own property record), and the
+`VBFrame` text. Containers nest into storages of their own, named
+for the site id -- a `Frame`'s children in `i02`, a `MultiPage`'s Pages
+in `i08` / `i09` under its own `i06`.
+
+`forms.py` reads that tree and writes it back; `_oforms_records.py`
+carries one property table per control class. It never guesses: every
+structure is counted or length-prefixed, so a misread collapses rather
+than yielding a plausible control list, and it raises `FormParseError`
+instead. Three checks have to agree -- `CountOfBytes` runs exactly to the
+end of `f`, the per-site `ObjectStreamSize` values sum to exactly
+`len(o)`, and every child storage is claimed by a site.
+
+Writing is lossless first: alignment padding is captured and replayed
+(the spec leaves those bytes undefined), string bytes are kept raw beside
+their decoded text, pictures stay opaque runs, and any tail the tables do
+not model is preserved. Bytes inside a record's `cb` that the tables
+cannot explain are refused rather than dropped. The gate is that an
+unedited form serializes to the bytes it was read from.
+
+Four details cost the most and none are obvious from a first reading of
+[MS-OFORMS]:
+
+- a site's `cbSite` counts from the **mask**, so the next site begins at
+  `start + 4 + cbSite`;
+- mask **bit 8 carries no fixed field**; reading two bytes for it puts
+  every name two characters late;
+- a `MultiPage`'s `f` carries a trailing MultiPage record after the
+  FormControl, so the sites do not close the stream exactly there. It is
+  version-stamped and length-prefixed, so the reader checks for it rather
+  than merely tolerating a remainder;
+- bit 8 selects no DataBlock field but does select an `fmPosition` in the
+  ExtraDataBlock, between the Name/Tag strings and the rest.
+
+Three things a *written* form needs that reading never reveals, each
+found by Excel refusing the result:
+
+- **`NextAvailableID` is the highest id already handed out**, not the next
+  free one. A new control takes `NextAvailableID + 1`; using the field
+  as-is repeats the last control's id and MSForms refuses the form.
+- **MorphData's mask bit 31 is reserved and MUST be 1**
+  ([MS-OFORMS] 2.2.5.2). Setting it is the single change that makes a new
+  `TextBox` or `OptionButton` load.
+- **A designer edit must invalidate the `_VBA_PROJECT` performance cache.**
+  Adding or removing a control changes the form class's members, and with
+  a stale cache Office loads a member list the form no longer matches.
+- **A container's storage is bound by its CLSID and its `\x01CompObj`**,
+  which names the kind fm20 should treat it as. Get either wrong and the
+  container loads without erroring and simply does not appear, so both are
+  reproduced verbatim from an Excel-authored fixture.
+- **A container's site is not a leaf's.** It carries `BitFlags` and no
+  `ObjectStreamSize`, because its record is the `f` of its own storage
+  rather than a slice of the parent's `o`.
+- **`NextAvailableID` is per container, and it is the highest id anywhere
+  beneath it** -- the fixture's MultiPage carries 11, which is a control
+  two levels down on one of its pages. A new id is recorded on every
+  ancestor up to the form.
+
+Composing a form from nothing needs one more thing the format does not
+announce: the **empty class-table count word**. With `BooleanProperties`
+defaulted, `DONTSAVECLASSTABLE` is 0 and fm20 reads a count before
+`CountOfSites`; omit the word and the low bytes of `CountOfSites` are read
+*as* the count. An empty form survives that by luck -- both are zero -- and
+the form's first control makes the misread count 1, so fm20 parses garbage
+as class info and refuses the whole form.
+
+A form is also two things at once: a designer storage and a code-behind
+module, declared `BaseClass=` (not `Class=`) in the PROJECT stream. A
+storage without a module is not a component the host shows; a module
+without a storage is an ordinary class.
+
+A page is four structures at once, and `_oforms_pages.py` moves them
+together: a site and a storage of its own; an entry in each of the
+MultiPage's five parallel TabStrip arrays (`Items`, `TipStrings`,
+`TabNames`, `Tags`, `Accelerators`) with one flag word per tab after
+them; `TabData` and `TabsAllocated`; and its position in the `x` stream,
+which holds one more `PageProperties` record than there are pages (the
+first ignored) followed by the page site ids. Page names are scoped to
+their MultiPage, not to the form -- Excel gives a second MultiPage its own
+`Page1` and `Page2` -- and pages never appear in `Designer.Controls`.
+
+Nesting is resolved by matching a child storage's numeric suffix against
+a site id, not by rebuilding the storage name from the id: the file says
+which storages exist, and the padding of that name is only observed.
+
+Reading `f` and `o` needs path-addressed CFB navigation
+(`list_storages_at` / `get_stream_at`), because names repeat -- every
+container owns an `f` -- and a name-based lookup finds whichever comes
+first in directory order.
+
+---
+
 ## 6. Encoding conventions
 
 - The VBA `PROJECTCODEPAGE` value (typically `1252`) drives every
@@ -384,9 +492,11 @@ tests/
 - **No silent corruption.** Any code path that could leave a workbook
   inconsistent must either succeed completely or raise. Partial state
   is never written to disk.
-- **Preserve what you don't understand.** Bytes that the library does
-  not interpret (designer sub-storages, PROJECTlk in v1, signature
-  payloads on no-op saves) are round-tripped verbatim.
+- **Preserve what you don't understand.** Bytes the library does not
+  interpret (PROJECTlk, signature payloads on no-op saves) are
+  round-tripped verbatim -- and so are the parts of a structure it *does*
+  interpret but cannot explain: a control record's alignment padding, an
+  unmodelled tail, a CompObj blob.
 - **Drop perf caches on mutation.** `_VBA_PROJECT` body is zeroed,
   `__SRP_*` streams are removed. Office regenerates both.
 - **ASCII only in user-facing strings.** Warnings, error messages,