pyOpenVBA 6.1.2__tar.gz → 6.1.3__tar.gz

{pyopenvba-6.1.2 → pyopenvba-6.1.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: pyOpenVBA
-Version: 6.1.2
+Version: 6.1.3
 Summary: Read and write VBA macros and Excel Power Query in pure Python, no dependencies.
 Project-URL: Homepage, https://github.com/WilliamSmithEdward/pyOpenVBA
 Project-URL: Repository, https://github.com/WilliamSmithEdward/pyOpenVBA
@@ -262,6 +262,9 @@ with ExcelFile("workbook.xlsm") as wb:
     wb.save("out.xlsm")
 ```
 
+Deleting a UserForm's module takes the form's design with it, as the
+editor does.
+
 ```python
 from pyopenvba import AccessDatabase, VBAModuleKind
 
@@ -380,7 +383,10 @@ None)` clears a property, so the control goes back to its default.
 MSForms stores a property only when it differs from the control's
 default, so `properties()` returns what the developer set, which no live
 host can tell you. Writing is lossless: an unedited form saves back byte
-for byte.
+for byte. The stored fields are not always named as VBA names them:
+[docs/userforms.md](docs/userforms.md) says where the designers store a
+property that differs, such as `TextAlign`, and what setting one changes
+besides.
 
 The command line shows the tree:
 
@@ -1049,6 +1055,7 @@ For more:
 - [docs/architecture.md](https://github.com/WilliamSmithEdward/pyOpenVBA/blob/main/docs/architecture.md): internal layout and conventions.
 - [docs/access_engine.md](https://github.com/WilliamSmithEdward/pyOpenVBA/blob/main/docs/access_engine.md): the Access file format as measured, and what the engine reproduces.
 - [docs/power_query.md](https://github.com/WilliamSmithEdward/pyOpenVBA/blob/main/docs/power_query.md): how Excel stores Power Query, and how each rule was measured.
+- [docs/userforms.md](https://github.com/WilliamSmithEdward/pyOpenVBA/blob/main/docs/userforms.md): where the form designers store the UserForm properties VBA names otherwise, as measured.
 - [docs/ms-ovba-implementation-guide_v2.md](https://github.com/WilliamSmithEdward/pyOpenVBA/blob/main/docs/ms-ovba-implementation-guide_v2.md): a language-agnostic guide to re-implementing MS-OVBA.
 - [docs/roadmap.md](https://github.com/WilliamSmithEdward/pyOpenVBA/blob/main/docs/roadmap.md): per-feature status.
 - [docs/host_completeness.md](docs/host_completeness.md): whole-host completion criteria and the Excel coverage audit; Excel, then Word, then PowerPoint.

{pyopenvba-6.1.2 → pyopenvba-6.1.3}/README.md

@@ -225,6 +225,9 @@ with ExcelFile("workbook.xlsm") as wb:
     wb.save("out.xlsm")
 ```
 
+Deleting a UserForm's module takes the form's design with it, as the
+editor does.
+
 ```python
 from pyopenvba import AccessDatabase, VBAModuleKind
 
@@ -343,7 +346,10 @@ None)` clears a property, so the control goes back to its default.
 MSForms stores a property only when it differs from the control's
 default, so `properties()` returns what the developer set, which no live
 host can tell you. Writing is lossless: an unedited form saves back byte
-for byte.
+for byte. The stored fields are not always named as VBA names them:
+[docs/userforms.md](docs/userforms.md) says where the designers store a
+property that differs, such as `TextAlign`, and what setting one changes
+besides.
 
 The command line shows the tree:
 
@@ -1012,6 +1018,7 @@ For more:
 - [docs/architecture.md](https://github.com/WilliamSmithEdward/pyOpenVBA/blob/main/docs/architecture.md): internal layout and conventions.
 - [docs/access_engine.md](https://github.com/WilliamSmithEdward/pyOpenVBA/blob/main/docs/access_engine.md): the Access file format as measured, and what the engine reproduces.
 - [docs/power_query.md](https://github.com/WilliamSmithEdward/pyOpenVBA/blob/main/docs/power_query.md): how Excel stores Power Query, and how each rule was measured.
+- [docs/userforms.md](https://github.com/WilliamSmithEdward/pyOpenVBA/blob/main/docs/userforms.md): where the form designers store the UserForm properties VBA names otherwise, as measured.
 - [docs/ms-ovba-implementation-guide_v2.md](https://github.com/WilliamSmithEdward/pyOpenVBA/blob/main/docs/ms-ovba-implementation-guide_v2.md): a language-agnostic guide to re-implementing MS-OVBA.
 - [docs/roadmap.md](https://github.com/WilliamSmithEdward/pyOpenVBA/blob/main/docs/roadmap.md): per-feature status.
 - [docs/host_completeness.md](docs/host_completeness.md): whole-host completion criteria and the Excel coverage audit; Excel, then Word, then PowerPoint.

{pyopenvba-6.1.2 → pyopenvba-6.1.3}/docs/architecture.md

@@ -345,12 +345,16 @@ written one needs (`scripts/measure_first_macro.py`).
 
 ## 5a. UserForm designer streams
 
-A form's design lives beside the VBA storage, not inside it: a root
-storage named for the form, holding `f` (the sites: which controls, in
-what order), `o` (each control's own property record), and the
-`VBFrame` text. Containers nest into storages of their own, named
-for the site id -- a `Frame`'s children in `i02`, a `MultiPage`'s Pages
-in `i08` / `i09` under its own `i06`.
+A form's design lives beside the VBA storage, not inside it: a storage
+named for the form in the project's storage, holding `f` (the sites:
+which controls, in what order), `o` (each control's own property
+record), and the `VBFrame` text. Containers nest into storages of their
+own, named for the site id -- a `Frame`'s children in `i02`, a
+`MultiPage`'s Pages in `i08` / `i09` under its own `i06`. The project's
+storage is the root of `vbaProject.bin` and of a `.ppt`'s embedded
+project, and `_VBA_PROJECT_CUR` or `Macros` in a `.xls` or `.doc`, which
+hold the document too (`VBAHostFile._project_root`,
+tests/fixtures/binary_forms.json).
 
 `forms.py` reads that tree and writes it back; `_oforms_records.py`
 carries one property table per control class. It never guesses: every

{pyopenvba-6.1.2 → pyopenvba-6.1.3}/docs/ms-ovba-implementation-guide_v2.md

@@ -466,6 +466,7 @@ Document=ThisWorkbook/&H00000000
 Document=Sheet1/&H00000000
 Module=Module1
 Class=Class1
+Package={AC9F2F90-E877-11CE-9F68-00AA00574A4F}
 BaseClass=UserForm1
 Name="VBAProject"
 HelpContextID="0"
@@ -497,6 +498,15 @@ entries together. Do not create `Document=` lines (host-owned). Do
 not create `BaseClass=` lines unless you also emit a designer
 sub-storage.
 
+The editor writes `Package={AC9F2F90-E877-11CE-9F68-00AA00574A4F}`,
+the Forms designer's package, once: immediately before the
+`BaseClass=` line of a form added to a stream that has no such line.
+It never moves or removes the line afterwards, even when the last
+form goes, and a declaration added later goes after it as after any
+other. A project that declares a form without the line gets it only
+with its next new form. Keep every line a mutation does not touch
+byte for byte; a `[Workspace]` value can end in a space.
+
 ---
 
 ## 9. The `PROJECTwm` stream
@@ -530,17 +540,20 @@ deprecated control surface.
 
 ## 11. UserForms / designer storages
 
-For each `BaseClass=` module there is a **sub-storage** at the CFB root
-with the same name as the module. It contains designer streams:
+For each `BaseClass=` module there is a **sub-storage** in the project's
+storage, beside `/VBA`, with the same name as the module: at the CFB
+root of `vbaProject.bin`, in `_VBA_PROJECT_CUR` in a `.xls` and in
+`Macros` in a `.doc`. It contains designer streams:
 typically `f`, `o`, `\x01CompObj`, `\x03VBFrame`, and sometimes more.
 
 Editing the **code-behind** of a UserForm is identical to editing any
 other module's source: the form's *code* lives in `/VBA/<FormName>` and
 is round-tripped through the same compressed-source pipeline.
 
-Editing the **layout** (controls, properties, positions) requires the
-Office Forms binary format and is out of scope for v1. Preserve the
-designer sub-storage byte-for-byte.
+Editing the **layout** (controls, properties, positions) means writing
+the Office Forms binary format, [MS-OFORMS]. pyOpenVBA reads and writes
+it in `forms.py` (docs/architecture.md, section 5a). A writer that does
+not must preserve the designer sub-storage byte for byte.
 
 ---
 
@@ -726,10 +739,12 @@ For each operation, every step must succeed or you must abort.
 | Drop declaration line | `PROJECT` |
 | Drop `[Workspace]` entry | `PROJECT` |
 | Drop NAMEMAP pair | `PROJECTwm` |
+| Remove a form's designer sub-storage | beside `/VBA` |
 | Invalidate + drop SRP + signatures | as above |
 
 Forbidden: deleting `Document=` modules (host-owned), deleting
-designer modules without also removing their sub-storage.
+designer modules without also removing their sub-storage. The
+`Package=` line stays when the last form goes.
 
 ---
 

{pyopenvba-6.1.2 → pyopenvba-6.1.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "pyOpenVBA"
-version = "6.1.2"
+version = "6.1.3"
 description = "Read and write VBA macros and Excel Power Query in pure Python, no dependencies."
 readme = "README.md"
 license = { text = "MIT" }

{pyopenvba-6.1.2 → pyopenvba-6.1.3}/src/pyopenvba/__init__.py

@@ -293,4 +293,4 @@ def push_power_query(
     """
     return _push_queries(src_dir, workbook, out=out, encoding=encoding, remove_missing=remove_missing)
 
-__version__ = "6.1.2"
+__version__ = "6.1.3"

{pyopenvba-6.1.2 → pyopenvba-6.1.3}/src/pyopenvba/_host.py

@@ -24,7 +24,7 @@ from pyopenvba._new_project import with_project
 from pyopenvba._package_signature import signature_parts, without_signature
 from pyopenvba._references import ReferenceManager, module_offset, reference_spans
 from pyopenvba.exceptions import NoVBAProjectError, UnsupportedFormatError, VBAProjectError
-from pyopenvba.forms import VBAForm, create_form, form_names, read_forms
+from pyopenvba.forms import VBAForm, create_form, form_names, read_form, read_forms
 from pyopenvba.vba import (
     SignatureInfo,
     VBAModuleKind,
@@ -226,7 +226,7 @@ class VBAHostFile(ReferenceManager):
         return {"workbook": "excel", "document": "word", "presentation": "powerpoint"}[self._host_noun]
 
     def _reference_forms(self) -> list[str]:
-        return form_names(self._get_cfb())
+        return form_names(self._get_cfb(), root=self._project_root())
 
     def references(self) -> list[VBAReference]:
         """Declared libraries in priority order; none in a file with no project."""
@@ -265,13 +265,14 @@ class VBAHostFile(ReferenceManager):
         streams do not reconcile.
 
         The result is cached, so property edits made on it are the ones
-        :meth:`save` writes back.
+        :meth:`save` writes back.  Adding a form and saving keep the same
+        objects, and with them the edits made through each.
         """
         if not self._has_project:
             return []
         if self._forms is None:
             self._forms = read_forms(
-                self._get_cfb(), code_page=self.vba_project().code_page
+                self._get_cfb(), code_page=self.vba_project().code_page, root=self._project_root()
             )
         return self._forms
 
@@ -294,7 +295,8 @@ class VBAHostFile(ReferenceManager):
         ``MSForms`` types compiles.
 
         The form is returned ready to edit -- ``add_control`` and friends
-        work on it straight away -- and lands on disk at :meth:`save`.
+        work on it straight away -- and lands on disk at :meth:`save`.  The
+        forms read or added before it are left as they are, edits included.
         """
         project = self.vba_project()
         cfb = self._get_cfb()
@@ -305,13 +307,21 @@ class VBAHostFile(ReferenceManager):
             width=width,
             height=height,
             code_page=project.code_page,
+            root=self._project_root(),
         )
         project.add_module(name, header, kind=VBAModuleKind.other)
         # The editor declares Microsoft Forms with a project's first form, and code naming its types needs it.
         self._ensure_forms_reference()
-        # The cache was read before this form existed.
-        self._forms = None
-        return next(f for f in self.forms() if f.name == name)
+        if self._forms is None:
+            # Nothing read yet, so nothing edited: reading every form reads this one too.
+            return next(f for f in self.forms() if f.name == name)
+        # The forms read before keep their objects, and with them any edit made through one and not saved
+        # yet: only the new form is read, and the cache keeps the directory order read_forms gives.
+        root = self._project_root()
+        cached = {form.name: form for form in self._forms}
+        cached[name] = read_form(cfb, name, code_page=project.code_page, root=root)
+        self._forms = [cached[each] for each in form_names(cfb, root=root)]
+        return cached[name]
 
     def module_names(self) -> list[str]:
         """Return the list of VBA module names."""
@@ -630,7 +640,7 @@ class VBAHostFile(ReferenceManager):
             module.dirty = False
             if module.kind == VBAModuleKind.standard:
                 decl_key = "Module"
-            elif module.name in form_names(cfb):
+            elif module.name in form_names(cfb, root=self._project_root()):
                 # A designer is declared BaseClass, not Class.  The
                 # test is structural -- its name is also a storage
                 # beside VBA/ -- which is the same one forms.py uses
@@ -642,12 +652,22 @@ class VBAHostFile(ReferenceManager):
                 decl_key = "Class"
             add_modules_for_project.append((module.name, decl_key))
 
-        # 3. Delete streams the user removed in-memory.
+        # 3. Delete streams the user removed in-memory.  A form goes with its
+        #    designer storage, as the editor removes one; left behind, the
+        #    storage still reads as a form that holds its name.
+        root = self._project_root()
+        forms = {name.casefold(): name for name in form_names(cfb, root=root)}
         for name in sorted(delete_names):
             try:
                 cfb.remove_stream_in_storage("VBA", name)
             except KeyError:
                 pass
+            if name.casefold() in forms:
+                cfb.remove_storage_at(root, forms[name.casefold()])
+                if self._forms is not None:
+                    # Only that form leaves the cache: the others keep their objects, so an edit made
+                    # through one after this save still saves.
+                    self._forms = [form for form in self._forms if form.name != forms[name.casefold()]]
 
         project.pending_renames.clear()
         project.pending_adds.clear()
@@ -901,3 +921,15 @@ class VBAHostFile(ReferenceManager):
         vba_bin = self._zip.read(self._vba_entry)
         self._cfb = CFB.from_bytes(vba_bin)
         return self._cfb
+
+    def _project_root(self) -> tuple[str, ...]:
+        """The path of the project's storage in the CFB ``_get_cfb`` returns.
+
+        Empty where that CFB is the project, as ``vbaProject.bin`` and a
+        .ppt's embedded project are.  A .xls or .doc holds the document
+        too, and Excel and Word keep a form's storage in the project's
+        storage beside ``VBA/`` (tests/fixtures/binary_forms.json).
+        """
+        if self._suffix in self._cfb_formats and self._project_storage is not None:
+            return (self._project_storage,)
+        return ()

{pyopenvba-6.1.2 → pyopenvba-6.1.3}/src/pyopenvba/_oforms_pages.py

@@ -106,8 +106,13 @@ def serialize_string_array(entries: list[TabString], encoding: str) -> bytes:
 
 
 def new_tab_string(text: str) -> TabString:
-    """A fresh entry, compressed when every character fits one byte."""
-    return TabString(text, all(ord(c) <= 0xFF for c in text))
+    """A fresh entry, compressed when every character fits one byte.
+
+    An empty one is stored with no compression flag, a count of zero and
+    nothing else, as the designers store a tab's empty tip, tag and
+    accelerator (tests/fixtures/form_designer.json).
+    """
+    return TabString(text, bool(text) and all(ord(c) <= 0xFF for c in text))
 
 
 @dataclass

{pyopenvba-6.1.2 → pyopenvba-6.1.3}/src/pyopenvba/_oforms_records.py

@@ -69,6 +69,9 @@ class RecordSpec:
     text_props: bool = False
     raw_tail: bool = False
     """Bytes after TextProps (rgColumnInfo, TabStripTabFlags)."""
+    flags: tuple[tuple[int, str, bool], ...] = ()
+    """Mask bits that store a property with no data: (bit, name, the value
+    the bit stands for), the property's default when the bit is clear."""
 
 
 # ---------------------------------------------------------------------------
@@ -153,6 +156,9 @@ COMMAND_BUTTON_SPEC = RecordSpec(
     extra=(ExtraField(3, "Caption", "str"), ExtraField(5, "Size", "size8")),
     stream=((7, "Picture"), (10, "MouseIcon")),
     text_props=True,
+    # fTakeFocusOnClick: set when TakeFocusOnClick is False, with no data
+    # (tests/fixtures/form_properties.json).
+    flags=((9, "TakeFocusOnClick", False),),
 )
 
 LABEL_SPEC = RecordSpec(
@@ -390,6 +396,9 @@ class ParsedRecord:
         for extra in self.spec.extra:
             if extra.name == name:
                 return extra.bit
+        for bit, flag, _ in self.spec.flags:
+            if flag == name:
+                return bit
         return None
 
     def properties(self) -> dict[str, object]:
@@ -406,6 +415,13 @@ class ParsedRecord:
                 out[name] = f"<{len(self.pictures[name])} bytes>"
             else:
                 out[name] = value
+        # A string set since the record was read has no length field until
+        # it is serialized, but it is stored all the same.
+        for name, stored in self.strings.items():
+            out.setdefault(name, stored.text)
+        for bit, name, value in self.spec.flags:
+            if self.mask & (1 << bit):
+                out[name] = value
         out.update(self.sizes)
         for name, blob in self.arrays.items():
             out[name] = f"<{len(blob)} bytes>"
@@ -417,7 +433,15 @@ class ParsedRecord:
     # -- mutation ---------------------------------------------------------
 
     def set_value(self, name: str, value: int | None) -> None:
-        """Set or clear a numeric DataBlock field, adjusting the mask."""
+        """Set or clear a numeric DataBlock field, adjusting the mask.
+
+        A flag, a property the mask stores with no data, is set when
+        ``value`` is the one its bit stands for and cleared otherwise.
+        """
+        flag = next(((bit, when) for bit, flag, when in self.spec.flags if flag == name), None)
+        if flag is not None:
+            self._set_bit(flag[0], value is not None and bool(value) == flag[1])
+            return
         spec_field = next(
             (f for f in self.spec.data if f.name == name and f.kind != "marker"),
             None,

{pyopenvba-6.1.2 → pyopenvba-6.1.3}/src/pyopenvba/cfb.py

@@ -38,6 +38,11 @@ _FATSECT: int = 0xFFFF_FFFD    # sector is part of the FAT
 _DIFSECT: int = 0xFFFF_FFFC    # sector is part of the DIFAT
 _NOSTREAM: int = 0xFFFF_FFFF   # no sibling / no child
 
+# FAT sector numbers the header lists, and a 512-byte DIFAT sector's after it
+# (the sector's last four bytes hold the next DIFAT sector's number).
+_HEADER_DIFAT = 109
+_DIFAT_PER_SECTOR = 127
+
 # Directory entry object types
 _OBJTYPE_EMPTY: int = 0
 _OBJTYPE_STORAGE: int = 1
@@ -836,11 +841,12 @@ class CFB:
         topology and per-entry metadata (CLSID, state, color, timestamps).
 
         Layout, in sector order:
-            [mini-stream data] [regular stream data] [directory] [mini-FAT] [FAT]
+            [mini-stream data] [regular stream data] [directory] [mini-FAT] [FAT] [DIFAT]
 
-        Files large enough to require a DIFAT chain (more than 109 FAT sectors,
-        about 27 MB of total payload) are rejected — vbaProject.bin is always
-        well under this limit.
+        The header lists the first 109 FAT sectors, which cover 109 x 128
+        sectors of 512 bytes, about 7.1 MB.  A larger file lists the rest in
+        DIFAT sectors ([MS-CFB] 2.5): 127 FAT sector numbers each, then the
+        number of the next DIFAT sector, the last one ending the chain.
         """
         SECTOR = 512
         MINI = 64
@@ -952,22 +958,22 @@ class CFB:
             minifat_first = _ENDOFCHAIN
             n_minifat_sectors = 0
 
-        # ---- 4. Decide how many FAT sectors we need (fixed-point) ----
+        # ---- 4. Decide how many FAT and DIFAT sectors we need (fixed-point) ----
+        # The FAT maps every sector, its own and the DIFAT's included, and the
+        # DIFAT lists the FAT sectors the header has no room for.
         n_data = len(sector_payloads)
         n_fat = 1
         while True:
-            needed = ((n_data + n_fat) + ENTRIES_PER_SECTOR - 1) // ENTRIES_PER_SECTOR
+            n_difat = -(-max(0, n_fat - _HEADER_DIFAT) // _DIFAT_PER_SECTOR)
+            needed = -(-(n_data + n_fat + n_difat) // ENTRIES_PER_SECTOR)
             if needed <= n_fat:
                 break
             n_fat = needed
-        if n_fat > 109:
-            raise CFBError(
-                "CFB writer requires DIFAT chain support for files this large; "
-                "not implemented."
-            )
 
         fat_first = n_data
         fat_sector_ids = list(range(fat_first, fat_first + n_fat))
+        difat_first = fat_first + n_fat
+        difat_sector_ids = list(range(difat_first, difat_first + n_difat))
 
         # ---- 5. Build the FAT ----
         fat = [_FREESECT] * (n_fat * ENTRIES_PER_SECTOR)
@@ -989,14 +995,26 @@ class CFB:
             chain(minifat_first, n_minifat_sectors)
         for s in fat_sector_ids:
             fat[s] = _FATSECT
+        for s in difat_sector_ids:
+            fat[s] = _DIFSECT
 
         fat_bytes = struct.pack(f"<{len(fat)}I", *fat)
         for k in range(n_fat):
             sector_payloads.append(fat_bytes[k * SECTOR:(k + 1) * SECTOR])
 
+        # The FAT sectors past the header's 109, 127 to a DIFAT sector, each
+        # sector ending with the next one's number.
+        rest = fat_sector_ids[_HEADER_DIFAT:]
+        for k in range(n_difat):
+            listed = rest[k * _DIFAT_PER_SECTOR:(k + 1) * _DIFAT_PER_SECTOR]
+            listed += [_FREESECT] * (_DIFAT_PER_SECTOR - len(listed))
+            following = difat_sector_ids[k + 1] if k + 1 < n_difat else _ENDOFCHAIN
+            sector_payloads.append(struct.pack(f"<{_DIFAT_PER_SECTOR + 1}I", *listed, following))
+
         # ---- 6. Build the header ----
-        difat = list(fat_sector_ids) + [_FREESECT] * (109 - len(fat_sector_ids))
-        difat_bytes = struct.pack("<109I", *difat)
+        difat = fat_sector_ids[:_HEADER_DIFAT]
+        difat += [_FREESECT] * (_HEADER_DIFAT - len(difat))
+        difat_bytes = struct.pack(f"<{_HEADER_DIFAT}I", *difat)
 
         header = struct.pack(
             _HEADER_FMT,
@@ -1015,8 +1033,8 @@ class CFB:
             CUTOFF,               # mini-stream cutoff size
             minifat_first,        # first mini-FAT sector
             n_minifat_sectors,    # num mini-FAT sectors
-            _ENDOFCHAIN,          # first DIFAT sector (none)
-            0,                    # num DIFAT sectors
+            difat_first if n_difat else _ENDOFCHAIN,  # first DIFAT sector
+            n_difat,              # num DIFAT sectors
             difat_bytes,          # 436-byte DIFAT array
         )