npm - @hominis/fireforge - Versions diffs - 0.34.1 → 0.34.2 - Mend

@hominis/fireforge 0.34.1 → 0.34.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (5) hide show

package/CHANGELOG.md +4 -0
package/dist/src/core/mach-resource-shim.d.ts +42 -7
package/dist/src/core/mach-resource-shim.js +227 -25
package/dist/src/core/test-harness-crash.js +12 -0
package/package.json +1 -1

package/CHANGELOG.md CHANGED Viewed

@@ -15,6 +15,10 @@
 - Added `furnace scan --track`, which persists every discovered untracked component into the `stock` section of furnace.json non-interactively (same locked, rollback-journaled write path as the interactive confirm flow). Scan's help and its non-interactive output now state that scan is report-only by default and where the inventory is consumed, ending the report-persist-nothing-explain-nothing behavior.
 - Added `furnace chrome-doc create --browser-window`, a browser.xhtml-like scaffold for the document that ships as the fork's main browser window: `<html id="main-window">` root with the `windowtype="navigator:browser"`/`chromehidden`/geometry-`persist` attributes platform C++ reads before scripts run, while keeping the generic scaffold's bootstrap wiring, sentinel, and (already-correct) jar.mn registrations. When the target document matches a configured `tokenHostDocuments` entry and the flag was not passed, create warns that the browser-window variant is probably intended.
 - Added `furnace create --test-dir <dir>` to redirect the `--with-tests` scaffold (browser-chrome and xpcshell styles) to any engine-relative directory under `browser/base/content/test/` — nested manifests register correctly — and made all test scaffolds collision-safe: existing `browser.toml`/`xpcshell.toml`/`chrome.toml` manifests are appended to (with a shared-manifest notice) instead of scaffolded over, and existing `head.js`/test implementation files are never overwritten.
+- Fixed the degraded-psutil guard wedging `mach build` at exit on flapping hosts (0.34.1 post-upgrade field report: psutil's vm/swap syscalls flap between working and degraded, the fallback swap reading arrived svmem-shaped (8 fields) where mozsystemmonitor's parent reconstructs a 6-field `sswap` via `self._swap_type(*swap_mem)`, so every collector sample was rejected — `resourcemonitor.py:766: UserWarning: failed to read the received data` — upstream's handler breaks its drain loop, the pipe fills, the collector child blocks in `send()` and never reads the terminate sentinel, and mach hangs forever in its atexit `waitpid`). The guard's fallback readings are now **per-function, arity-correct, module-level namedtuples** (svmem 8 / sswap 6 / scputimes 4 / sdiskio 6; psutil's own result classes are still preferred when resolvable) that pickle by reference across the collector pipe and survive `type(reading)(*values)` reconstruction in **either** flapping direction (real type + degraded reading, or degraded type + real reading); `cpu_percent`/`cpu_times` fallbacks honour `percpu=True` (return `[]`) for the collector child's per-CPU sampling, and the last-resort `_DegradedReading` now tolerates positional-field construction. The report's "factory resolution bypassed in the collector child" observation is consistent with the two-copy load (`sitecustomize` via PYTHONPATH plus the in-venv `.pth`) or degraded-import-time `psutil._psplatform` resolution — the fix no longer depends on psutil internals resolving, so it is robust to both.
+- Added **collector suppression on degradation** to the guard: any observed psutil degradation sets a process-wide flag, after which monitors are kept inert — `start` never spawns the mozsystemmonitor collector child — and a degraded transition (any wrapped monitor method raising) or a raising `stop` best-effort terminates a live collector child (`_process.terminate()` + bounded join) and drains the parent end of the pipe, so a malformed sample stream can never fail the command or wedge shutdown even if a future shape mismatch slips through. (The upstream enabler — mozsystemmonitor warning then `break`ing out of its drain loop without draining, leaving the pipe to fill — is worth reporting to Mozilla but is not FireForge-fixable.)
+- Fixed the same degradation **failing `mach build` after a fully successful compile** ("Error running mach" with complete, valid `dist/` artifacts): degraded aggregate methods now return zeroed shapes instead of `None` where callers subscript the result (`aggregate_io` → zeroed io namedtuple, so mozbuild's `log_resource_usage` no longer dies on `usage["io"].read_bytes` at `mozbuild/controller/building.py:526`; `aggregate_cpu_percent`/`aggregate_cpu_times` return `[]`/`0.0`/zeroed shapes per their `per_cpu` mode), and the guard additionally wraps `mozbuild.controller.building.BuildMonitor.log_resource_usage` to warn-and-continue on any exception. Runtime-verified by new python3-executed regression tests covering per-function fallback arity, pickle/`type(*values)` round-trips in both flapping directions, collector suppression (stub monitor with upstream-named `_process`/`_pipe`), and the BuildMonitor wrap.
+- Taught the harness-crash classifier the post-success `AttributeError: 'NoneType' object has no attribute 'read_bytes'` traceback (protected builds retry it as crash-not-failure — with the fixed guard the incremental retry is cheap and green; real test failures still take precedence), and exempted the upstream drain-loop `failed to read the received data` warning from crash evidence so runs that complete despite the rejected-sample stream never classify as crashes.
 ## 0.33.0

package/dist/src/core/mach-resource-shim.d.ts CHANGED Viewed

@@ -30,17 +30,52 @@
  *    reading instead of aborting. Downstream report (0.34.0 cycle):
  *    mozsystemmonitor subscripts (`_build_meta` does
  *    `psutil.virtual_memory()[0]`) and iterates/unpacks the reading, so the
- *    zeroed fallback is a full namedtuple duck type — a zeroed instance of
- *    psutil's own result namedtuple where resolvable without touching the
- *    failing syscall, else `_DegradedReading` (subscriptable, iterable, with
- *    `len()`, `_fields`, `_asdict()`); `cpu_percent` degrades to `0.0`
- *    because callers do arithmetic on it;
+ *    zeroed fallback is a full namedtuple duck type; `cpu_percent` degrades
+ *    to `0.0` because callers do arithmetic on it. Field report (0.34.1,
+ *    flapping host): the fallback must ALSO be per-function arity-correct
+ *    and reconstructible — mozsystemmonitor's parent captures reading types
+ *    at `__init__` (`self._swap_type = type(psutil.swap_memory())`) and
+ *    rebuilds each collector sample via `self._swap_type(*swap_mem)`, and an
+ *    svmem-shaped (8-field) fallback in the swap (6-field `sswap`) position
+ *    made the parent reject every sample ("failed to read the received
+ *    data") and BREAK its drain loop, so the pipe filled, the collector
+ *    child blocked in `send()` and never read the terminate sentinel, and
+ *    mach hung forever in its atexit `waitpid`. Each wrapped function now
+ *    degrades to a zeroed instance of psutil's own result namedtuple where
+ *    resolvable, else a module-level guard-owned namedtuple with the exact
+ *    macOS field order/arity for THAT function (module-level so readings
+ *    pickle by reference across the collector pipe); `_DegradedReading`
+ *    remains only as a documented last resort and now tolerates
+ *    `type(reading)(*values)` reconstruction;
  *  - guards `SystemResourceMonitor` CONSTRUCTION via an import hook:
  *    `poll_interval` is pre-populated before the real `__init__` runs, a
  *    failing `__init__` marks the instance degraded instead of raising,
  *    and every monitor method no-ops on a degraded instance — so a
  *    partially-constructed monitor can never surface the
- *    `AttributeError: poll_interval` variant.
+ *    `AttributeError: poll_interval` variant. Degraded aggregate methods
+ *    return zeroed shapes (not `None`) where callers subscript the result
+ *    (`aggregate_io` → zeroed io reading, so mozbuild's
+ *    `log_resource_usage` no longer dies on `usage["io"].read_bytes` after
+ *    a fully successful compile);
+ *  - suppresses the mozsystemmonitor collector child on degradation: once
+ *    any psutil degradation is observed in the process, monitors are kept
+ *    inert (`start` never spawns the collector), and a degraded transition
+ *    or raising `stop` best-effort terminates a live collector child and
+ *    drains the pipe — so a malformed sample stream can never wedge mach's
+ *    shutdown even if a future shape mismatch slips through;
+ *  - additionally wraps `mozbuild.controller.building.BuildMonitor
+ *    .log_resource_usage` to warn-and-continue on any exception, so
+ *    end-of-build resource reporting can never fail a build whose
+ *    artifacts are complete.
+ *
+ * Why the 0.34.1 report saw `_DegradedReading` in the collector child even
+ * though the psutil result classes resolve when probed directly: two guard
+ * copies can load (PYTHONPATH `sitecustomize` in the bootstrap phase plus
+ * the in-venv `.pth`), each with its own factory/classes, and flapping at
+ * guard-import time can perturb `psutil._psplatform` attribute resolution
+ * in the spawn child specifically. The fix is robust to either mechanism:
+ * the guard-owned fallback classes never depend on psutil internals
+ * resolving and are arity-correct in every position.
  */
 /**
  * Python guard body, importable both as the in-venv `fireforge_mach_guard`
@@ -51,7 +86,7 @@
  * regression test (mach-resource-shim.python.test.ts), which asserts the
  * degraded readings' runtime behavior rather than string-matching the source.
  */
-export declare const GUARD_PYTHON_SOURCE = "# Generated by FireForge - do not edit.\n# Degrades the broken host resource monitor family (psutil vs Darwin) from\n# fatal startup errors into non-fatal warnings, so mach builds and tests\n# proceed. Installed into the mach virtualenv site-packages via a .pth file\n# (survives mach's venv re-exec, which drops PYTHONPATH).\nimport builtins\nimport sys\nimport warnings\n\n\ndef _fireforge_degraded_notice(exc):\n    warnings.warn(\n        \"FireForge: host resource monitor degraded (%s); continuing without resource monitoring.\" % exc\n    )\n\n\nclass _DegradedReading(object):\n    # Duck type of psutil's namedtuple results (svmem field order):\n    # mozsystemmonitor subscripts, iterates, and unpacks readings, so the\n    # degraded fallback must survive r[0], list(r), len(r), r._fields and\n    # r._asdict(), not just attribute access.\n    _fields = (\"total\", \"available\", \"percent\", \"used\", \"free\", \"active\", \"inactive\", \"wired\")\n    total = available = used = free = active = inactive = wired = 0\n    percent = 0.0\n\n    def __getattr__(self, _name):\n        return 0\n\n    def _values(self):\n        return tuple(getattr(self, _field) for _field in self._fields)\n\n    def __getitem__(self, index):\n        return self._values()[index]\n\n    def __iter__(self):\n        return iter(self._values())\n\n    def __len__(self):\n        return len(self._fields)\n\n    def _asdict(self):\n        return dict(zip(self._fields, self._values()))\n\n\n# psutil result namedtuple per wrapped function, resolvable via getattr from\n# the platform module / _common without invoking the failing syscall.\n_PSUTIL_RESULT_CLASSES = {\n    \"virtual_memory\": \"svmem\",\n    \"swap_memory\": \"sswap\",\n    \"cpu_times\": \"scputimes\",\n    \"disk_io_counters\": \"sdiskio\",\n}\n\n\ndef _degraded_result_factory(psutil, name):\n    # cpu_percent returns a plain float callers do arithmetic on.\n    if name == \"cpu_percent\":\n        return lambda: 0.0\n    _cls_name = _PSUTIL_RESULT_CLASSES.get(name)\n    if _cls_name is not None:\n        for _mod in (getattr(psutil, \"_psplatform\", None), getattr(psutil, \"_common\", None)):\n            _cls = getattr(_mod, _cls_name, None) if _mod is not None else None\n            _fields = getattr(_cls, \"_fields\", None)\n            if _cls is not None and _fields:\n                return lambda _cls=_cls, _n=len(_fields): _cls(*([0] * _n))\n    return _DegradedReading\n\n\ndef _guard_psutil():\n    try:\n        import psutil\n    except Exception:\n        return\n\n    def _wrap(orig, fallback):\n        def wrapper(*args, **kwargs):\n            try:\n                return orig(*args, **kwargs)\n            except Exception as exc:  # noqa: BLE001\n                _fireforge_degraded_notice(exc)\n                try:\n                    return fallback()\n                except Exception:  # noqa: BLE001\n                    return _DegradedReading()\n\n        return wrapper\n\n    for _name in (\"virtual_memory\", \"swap_memory\", \"cpu_percent\", \"cpu_times\", \"disk_io_counters\"):\n        _orig = getattr(psutil, _name, None)\n        if _orig is not None and not getattr(_orig, \"_fireforge_guarded\", False):\n            try:\n                _fallback = _degraded_result_factory(psutil, _name)\n            except Exception:  # noqa: BLE001\n                _fallback = _DegradedReading\n            _wrapped = _wrap(_orig, _fallback)\n            _wrapped._fireforge_guarded = True\n            setattr(psutil, _name, _wrapped)\n\n\n_MONITOR_METHOD_NAMES = (\n    \"start\",\n    \"stop\",\n    \"record_event\",\n    \"record_marker\",\n    \"begin_phase\",\n    \"finish_phase\",\n    \"aggregate_cpu_percent\",\n    \"aggregate_cpu_times\",\n    \"aggregate_io\",\n    \"min_memory_available\",\n    \"as_dict\",\n)\n\n\ndef _patch_monitor_class(cls):\n    if cls is None or getattr(cls, \"_fireforge_guarded\", False):\n        return\n    orig_init = cls.__init__\n\n    def guarded_init(self, *args, **kwargs):\n        # Pre-populate the attributes a partially-constructed instance is\n        # later polled for, so a failing __init__ can never surface\n        # AttributeError: poll_interval.\n        self.poll_interval = kwargs.get(\"poll_interval\") or 1.0\n        self._fireforge_degraded = False\n        try:\n            orig_init(self, *args, **kwargs)\n        except Exception as exc:  # noqa: BLE001\n            self._fireforge_degraded = True\n            _fireforge_degraded_notice(exc)\n\n    cls.__init__ = guarded_init\n\n    def _wrap_method(orig):\n        def wrapper(self, *args, **kwargs):\n            if getattr(self, \"_fireforge_degraded\", False):\n                return None\n            try:\n                return orig(self, *args, **kwargs)\n            except Exception as exc:  # noqa: BLE001\n                self._fireforge_degraded = True\n                _fireforge_degraded_notice(exc)\n                return None\n\n        return wrapper\n\n    for _name in _MONITOR_METHOD_NAMES:\n        _orig = getattr(cls, _name, None)\n        if _orig is not None:\n            setattr(cls, _name, _wrap_method(_orig))\n    cls._fireforge_guarded = True\n\n\n_MONITOR_MODULES = (\"mozsystemmonitor.resourcemonitor\", \"mozbuild.resources\")\n\n\ndef _patch_loaded_monitor_modules():\n    for _mod_name in _MONITOR_MODULES:\n        _mod = sys.modules.get(_mod_name)\n        if _mod is not None:\n            try:\n                _patch_monitor_class(getattr(_mod, \"SystemResourceMonitor\", None))\n            except Exception:  # noqa: BLE001\n                pass\n\n\ndef _install_import_hook():\n    _orig_import = builtins.__import__\n    if getattr(_orig_import, \"_fireforge_guarded\", False):\n        return\n\n    def _guarding_import(name, *args, **kwargs):\n        _module = _orig_import(name, *args, **kwargs)\n        try:\n            _patch_loaded_monitor_modules()\n        except Exception:  # noqa: BLE001\n            pass\n        return _module\n\n    _guarding_import._fireforge_guarded = True\n    builtins.__import__ = _guarding_import\n\n\ntry:\n    _guard_psutil()\n    _patch_loaded_monitor_modules()\n    _install_import_hook()\nexcept Exception:  # noqa: BLE001\n    pass\n";
+export declare const GUARD_PYTHON_SOURCE = "# Generated by FireForge - do not edit.\n# Degrades the broken host resource monitor family (psutil vs Darwin) from\n# fatal startup errors into non-fatal warnings, so mach builds and tests\n# proceed. Installed into the mach virtualenv site-packages via a .pth file\n# (survives mach's venv re-exec, which drops PYTHONPATH).\nimport builtins\nimport collections\nimport sys\nimport warnings\n\n# Any psutil degradation observed in this process. Once set, monitors are\n# kept inert (start never spawns the collector child) \u2014 on a flapping host a\n# healthy-looking moment must not be trusted to start a sample stream that a\n# later degraded reading could malform.\n_fireforge_host_degraded = [False]\n\n\ndef _fireforge_degraded_notice(exc):\n    _fireforge_host_degraded[0] = True\n    warnings.warn(\n        \"FireForge: host resource monitor degraded (%s); continuing without resource monitoring.\" % exc\n    )\n\n\nclass _DegradedReading(object):\n    # Last-resort duck type of psutil's namedtuple results (svmem field\n    # order): mozsystemmonitor subscripts, iterates, and unpacks readings,\n    # so the degraded fallback must survive r[0], list(r), len(r), r._fields\n    # and r._asdict(), not just attribute access. Field report (0.34.1): the\n    # parent also reconstructs readings via type(reading)(*values), so the\n    # constructor must tolerate the full positional field list. Normal\n    # degraded paths use the per-function namedtuple fallbacks below; this\n    # class only backstops a fallback that itself raised.\n    _fields = (\"total\", \"available\", \"percent\", \"used\", \"free\", \"active\", \"inactive\", \"wired\")\n    total = available = used = free = active = inactive = wired = 0\n    percent = 0.0\n\n    def __init__(self, *_args, **_kwargs):\n        pass\n\n    def __getattr__(self, _name):\n        return 0\n\n    def _values(self):\n        return tuple(getattr(self, _field) for _field in self._fields)\n\n    def __getitem__(self, index):\n        return self._values()[index]\n\n    def __iter__(self):\n        return iter(self._values())\n\n    def __len__(self):\n        return len(self._fields)\n\n    def _asdict(self):\n        return dict(zip(self._fields, self._values()))\n\n\n# Per-function fallback result classes with the exact macOS field\n# orders/arities. Module-level on purpose: readings cross the\n# mozsystemmonitor collector pipe via pickle (by reference), and the parent\n# rebuilds each sample with type(reading)(*values) \u2014 so a swap fallback must\n# be 6-field sswap-shaped in every position (field report 0.34.1: an\n# svmem-shaped fallback in the swap position made the parent reject every\n# sample, fill the pipe, block the collector child in send(), and wedge\n# mach's atexit join forever).\n_FallbackVmem = collections.namedtuple(\n    \"_FallbackVmem\",\n    (\"total\", \"available\", \"percent\", \"used\", \"free\", \"active\", \"inactive\", \"wired\"),\n)\n_FallbackSwap = collections.namedtuple(\n    \"_FallbackSwap\", (\"total\", \"used\", \"free\", \"percent\", \"sin\", \"sout\")\n)\n_FallbackCpuTimes = collections.namedtuple(\"_FallbackCpuTimes\", (\"user\", \"nice\", \"system\", \"idle\"))\n_FallbackDiskIO = collections.namedtuple(\n    \"_FallbackDiskIO\",\n    (\"read_count\", \"write_count\", \"read_bytes\", \"write_bytes\", \"read_time\", \"write_time\"),\n)\n\n_FALLBACK_CLASSES = {\n    \"virtual_memory\": _FallbackVmem,\n    \"swap_memory\": _FallbackSwap,\n    \"cpu_times\": _FallbackCpuTimes,\n    \"disk_io_counters\": _FallbackDiskIO,\n}\n\n# psutil result namedtuple per wrapped function, resolvable via getattr from\n# the platform module / _common without invoking the failing syscall.\n_PSUTIL_RESULT_CLASSES = {\n    \"virtual_memory\": \"svmem\",\n    \"swap_memory\": \"sswap\",\n    \"cpu_times\": \"scputimes\",\n    \"disk_io_counters\": \"sdiskio\",\n}\n\n\ndef _zeroed(cls):\n    return cls(*([0] * len(cls._fields)))\n\n\ndef _percpu_requested(args, kwargs, positional_index):\n    if \"percpu\" in kwargs:\n        return bool(kwargs[\"percpu\"])\n    return len(args) > positional_index and bool(args[positional_index])\n\n\ndef _degraded_result_factory(psutil, name):\n    # cpu_percent returns a plain float (or a per-CPU list) callers do\n    # arithmetic on.\n    if name == \"cpu_percent\":\n        def _cpu_percent_fallback(*args, **kwargs):\n            # cpu_percent(interval=None, percpu=False)\n            if _percpu_requested(args, kwargs, 1):\n                return []\n            return 0.0\n\n        return _cpu_percent_fallback\n    # The guard-owned class is always shape-correct for this function;\n    # psutil's own result namedtuple is still preferred when resolvable so\n    # type() captures stay identical to real readings on a flapping host.\n    _cls = _FALLBACK_CLASSES.get(name, _DegradedReading)\n    _cls_name = _PSUTIL_RESULT_CLASSES.get(name)\n    if _cls_name is not None:\n        for _mod in (getattr(psutil, \"_psplatform\", None), getattr(psutil, \"_common\", None)):\n            _real = getattr(_mod, _cls_name, None) if _mod is not None else None\n            if _real is not None and getattr(_real, \"_fields\", None):\n                _cls = _real\n                break\n    if name == \"cpu_times\":\n        def _cpu_times_fallback(*args, **kwargs):\n            # cpu_times(percpu=False); the collector child samples per-CPU.\n            if _percpu_requested(args, kwargs, 0):\n                return []\n            return _zeroed(_cls)\n\n        return _cpu_times_fallback\n\n    def _reading_fallback(*_args, **_kwargs):\n        return _zeroed(_cls)\n\n    return _reading_fallback\n\n\ndef _guard_psutil():\n    try:\n        import psutil\n    except Exception:\n        return\n\n    def _wrap(orig, fallback):\n        def wrapper(*args, **kwargs):\n            try:\n                return orig(*args, **kwargs)\n            except Exception as exc:  # noqa: BLE001\n                _fireforge_degraded_notice(exc)\n                try:\n                    return fallback(*args, **kwargs)\n                except Exception:  # noqa: BLE001\n                    return _DegradedReading()\n\n        return wrapper\n\n    for _name in (\"virtual_memory\", \"swap_memory\", \"cpu_percent\", \"cpu_times\", \"disk_io_counters\"):\n        _orig = getattr(psutil, _name, None)\n        if _orig is not None and not getattr(_orig, \"_fireforge_guarded\", False):\n            try:\n                _fallback = _degraded_result_factory(psutil, _name)\n            except Exception:  # noqa: BLE001\n                _fallback = _DegradedReading\n            _wrapped = _wrap(_orig, _fallback)\n            _wrapped._fireforge_guarded = True\n            setattr(psutil, _name, _wrapped)\n\n\n_MONITOR_METHOD_NAMES = (\n    \"start\",\n    \"stop\",\n    \"record_event\",\n    \"record_marker\",\n    \"begin_phase\",\n    \"finish_phase\",\n    \"aggregate_cpu_percent\",\n    \"aggregate_cpu_times\",\n    \"aggregate_io\",\n    \"min_memory_available\",\n    \"as_dict\",\n)\n\n\ndef _fireforge_stop_collector(monitor):\n    # Best-effort: terminate a live collector child and drain the parent end\n    # of the pipe, so a child blocked in send() on a full pipe can never\n    # keep mach's atexit join (os.waitpid) waiting forever. Attribute names\n    # follow upstream resourcemonitor.py (self._process / self._pipe); a\n    # miss on a refactored upstream is harmless \u2014 the shape-correct\n    # fallbacks above still keep the sample stream well-formed.\n    try:\n        proc = getattr(monitor, \"_process\", None)\n        if proc is not None and getattr(proc, \"is_alive\", lambda: False)():\n            proc.terminate()\n            proc.join(1)\n    except Exception:  # noqa: BLE001\n        pass\n    try:\n        pipe = getattr(monitor, \"_pipe\", None)\n        if pipe is not None:\n            while pipe.poll(0):\n                pipe.recv()\n    except Exception:  # noqa: BLE001\n        pass\n\n\ndef _monitor_per_cpu_requested(args, kwargs):\n    # aggregate_cpu_percent/aggregate_cpu_times(start=None, end=None,\n    # phase=None, per_cpu=True) \u2014 self already stripped from args.\n    if \"per_cpu\" in kwargs:\n        return bool(kwargs[\"per_cpu\"])\n    if len(args) > 3:\n        return bool(args[3])\n    return True\n\n\ndef _monitor_degraded_result(name, args, kwargs):\n    # Degraded aggregate methods return zeroed shapes, not None, where\n    # callers subscript the result: mozbuild's log_resource_usage does\n    # usage[\"io\"].read_bytes after a successful compile (field report\n    # 0.34.1: 'NoneType' object has no attribute 'read_bytes' failed the\n    # build with complete artifacts).\n    if name == \"aggregate_io\":\n        return _zeroed(_FallbackDiskIO)\n    if name == \"aggregate_cpu_percent\":\n        return [] if _monitor_per_cpu_requested(args, kwargs) else 0.0\n    if name == \"aggregate_cpu_times\":\n        return [] if _monitor_per_cpu_requested(args, kwargs) else _zeroed(_FallbackCpuTimes)\n    return None\n\n\ndef _patch_monitor_class(cls):\n    if cls is None or getattr(cls, \"_fireforge_guarded\", False):\n        return\n    orig_init = cls.__init__\n\n    def guarded_init(self, *args, **kwargs):\n        # Pre-populate the attributes a partially-constructed instance is\n        # later polled for, so a failing __init__ can never surface\n        # AttributeError: poll_interval.\n        self.poll_interval = kwargs.get(\"poll_interval\") or 1.0\n        self._fireforge_degraded = False\n        try:\n            orig_init(self, *args, **kwargs)\n        except Exception as exc:  # noqa: BLE001\n            self._fireforge_degraded = True\n            _fireforge_degraded_notice(exc)\n        if _fireforge_host_degraded[0] and not self._fireforge_degraded:\n            # Host already degraded once: keep the monitor inert so the\n            # collector child never spawns on a flapping host.\n            self._fireforge_degraded = True\n        if self._fireforge_degraded:\n            _fireforge_stop_collector(self)\n\n    cls.__init__ = guarded_init\n\n    def _wrap_method(name, orig):\n        def wrapper(self, *args, **kwargs):\n            if (\n                name == \"start\"\n                and _fireforge_host_degraded[0]\n                and not getattr(self, \"_fireforge_degraded\", False)\n            ):\n                self._fireforge_degraded = True\n                _fireforge_stop_collector(self)\n            if getattr(self, \"_fireforge_degraded\", False):\n                if name == \"stop\":\n                    _fireforge_stop_collector(self)\n                return _monitor_degraded_result(name, args, kwargs)\n            try:\n                return orig(self, *args, **kwargs)\n            except Exception as exc:  # noqa: BLE001\n                self._fireforge_degraded = True\n                _fireforge_degraded_notice(exc)\n                # A raising stop (or any degraded transition) must still\n                # take the collector child down, or the parent's atexit\n                # join hangs on a child blocked writing to a full pipe.\n                _fireforge_stop_collector(self)\n                return _monitor_degraded_result(name, args, kwargs)\n\n        return wrapper\n\n    for _name in _MONITOR_METHOD_NAMES:\n        _orig = getattr(cls, _name, None)\n        if _orig is not None:\n            setattr(cls, _name, _wrap_method(_name, _orig))\n    cls._fireforge_guarded = True\n\n\ndef _patch_build_monitor_class(cls):\n    # mozbuild's BuildMonitor is not a SystemResourceMonitor; only\n    # log_resource_usage needs guarding \u2014 end-of-build resource reporting\n    # must never fail a build whose artifacts are complete. No __init__\n    # replacement, no degraded-instance gating.\n    if cls is None or getattr(cls, \"_fireforge_guarded\", False):\n        return\n    _orig = getattr(cls, \"log_resource_usage\", None)\n    if _orig is not None:\n        def _guarded_log_resource_usage(self, *args, **kwargs):\n            try:\n                return _orig(self, *args, **kwargs)\n            except Exception as exc:  # noqa: BLE001\n                _fireforge_degraded_notice(exc)\n                return None\n\n        cls.log_resource_usage = _guarded_log_resource_usage\n    cls._fireforge_guarded = True\n\n\n_MONITOR_CLASS_PATCHES = (\n    (\"mozsystemmonitor.resourcemonitor\", \"SystemResourceMonitor\", _patch_monitor_class),\n    (\"mozbuild.resources\", \"SystemResourceMonitor\", _patch_monitor_class),\n    (\"mozbuild.controller.building\", \"BuildMonitor\", _patch_build_monitor_class),\n)\n\n\ndef _patch_loaded_monitor_modules():\n    for _mod_name, _cls_name, _patcher in _MONITOR_CLASS_PATCHES:\n        _mod = sys.modules.get(_mod_name)\n        if _mod is not None:\n            try:\n                _patcher(getattr(_mod, _cls_name, None))\n            except Exception:  # noqa: BLE001\n                pass\n\n\ndef _install_import_hook():\n    _orig_import = builtins.__import__\n    if getattr(_orig_import, \"_fireforge_guarded\", False):\n        return\n\n    def _guarding_import(name, *args, **kwargs):\n        _module = _orig_import(name, *args, **kwargs)\n        try:\n            _patch_loaded_monitor_modules()\n        except Exception:  # noqa: BLE001\n            pass\n        return _module\n\n    _guarding_import._fireforge_guarded = True\n    builtins.__import__ = _guarding_import\n\n\ntry:\n    _guard_psutil()\n    _patch_loaded_monitor_modules()\n    _install_import_hook()\nexcept Exception:  # noqa: BLE001\n    pass\n";
 /** Result of installing the mach resource guard before a dispatch. */
 export interface MachResourceGuardInstallation {
     /**

package/dist/src/core/mach-resource-shim.js CHANGED Viewed

@@ -31,17 +31,52 @@
  *    reading instead of aborting. Downstream report (0.34.0 cycle):
  *    mozsystemmonitor subscripts (`_build_meta` does
  *    `psutil.virtual_memory()[0]`) and iterates/unpacks the reading, so the
- *    zeroed fallback is a full namedtuple duck type — a zeroed instance of
- *    psutil's own result namedtuple where resolvable without touching the
- *    failing syscall, else `_DegradedReading` (subscriptable, iterable, with
- *    `len()`, `_fields`, `_asdict()`); `cpu_percent` degrades to `0.0`
- *    because callers do arithmetic on it;
+ *    zeroed fallback is a full namedtuple duck type; `cpu_percent` degrades
+ *    to `0.0` because callers do arithmetic on it. Field report (0.34.1,
+ *    flapping host): the fallback must ALSO be per-function arity-correct
+ *    and reconstructible — mozsystemmonitor's parent captures reading types
+ *    at `__init__` (`self._swap_type = type(psutil.swap_memory())`) and
+ *    rebuilds each collector sample via `self._swap_type(*swap_mem)`, and an
+ *    svmem-shaped (8-field) fallback in the swap (6-field `sswap`) position
+ *    made the parent reject every sample ("failed to read the received
+ *    data") and BREAK its drain loop, so the pipe filled, the collector
+ *    child blocked in `send()` and never read the terminate sentinel, and
+ *    mach hung forever in its atexit `waitpid`. Each wrapped function now
+ *    degrades to a zeroed instance of psutil's own result namedtuple where
+ *    resolvable, else a module-level guard-owned namedtuple with the exact
+ *    macOS field order/arity for THAT function (module-level so readings
+ *    pickle by reference across the collector pipe); `_DegradedReading`
+ *    remains only as a documented last resort and now tolerates
+ *    `type(reading)(*values)` reconstruction;
  *  - guards `SystemResourceMonitor` CONSTRUCTION via an import hook:
  *    `poll_interval` is pre-populated before the real `__init__` runs, a
  *    failing `__init__` marks the instance degraded instead of raising,
  *    and every monitor method no-ops on a degraded instance — so a
  *    partially-constructed monitor can never surface the
- *    `AttributeError: poll_interval` variant.
+ *    `AttributeError: poll_interval` variant. Degraded aggregate methods
+ *    return zeroed shapes (not `None`) where callers subscript the result
+ *    (`aggregate_io` → zeroed io reading, so mozbuild's
+ *    `log_resource_usage` no longer dies on `usage["io"].read_bytes` after
+ *    a fully successful compile);
+ *  - suppresses the mozsystemmonitor collector child on degradation: once
+ *    any psutil degradation is observed in the process, monitors are kept
+ *    inert (`start` never spawns the collector), and a degraded transition
+ *    or raising `stop` best-effort terminates a live collector child and
+ *    drains the pipe — so a malformed sample stream can never wedge mach's
+ *    shutdown even if a future shape mismatch slips through;
+ *  - additionally wraps `mozbuild.controller.building.BuildMonitor
+ *    .log_resource_usage` to warn-and-continue on any exception, so
+ *    end-of-build resource reporting can never fail a build whose
+ *    artifacts are complete.
+ *
+ * Why the 0.34.1 report saw `_DegradedReading` in the collector child even
+ * though the psutil result classes resolve when probed directly: two guard
+ * copies can load (PYTHONPATH `sitecustomize` in the bootstrap phase plus
+ * the in-venv `.pth`), each with its own factory/classes, and flapping at
+ * guard-import time can perturb `psutil._psplatform` attribute resolution
+ * in the spawn child specifically. The fix is robust to either mechanism:
+ * the guard-owned fallback classes never depend on psutil internals
+ * resolving and are arity-correct in every position.
  */
 import { readdir } from 'node:fs/promises';
 import { homedir, tmpdir } from 'node:os';
@@ -68,25 +103,40 @@ export const GUARD_PYTHON_SOURCE = `# Generated by FireForge - do not edit.
 # proceed. Installed into the mach virtualenv site-packages via a .pth file
 # (survives mach's venv re-exec, which drops PYTHONPATH).
 import builtins
+import collections
 import sys
 import warnings
+# Any psutil degradation observed in this process. Once set, monitors are
+# kept inert (start never spawns the collector child) — on a flapping host a
+# healthy-looking moment must not be trusted to start a sample stream that a
+# later degraded reading could malform.
+_fireforge_host_degraded = [False]
 def _fireforge_degraded_notice(exc):
+    _fireforge_host_degraded[0] = True
     warnings.warn(
         "FireForge: host resource monitor degraded (%s); continuing without resource monitoring." % exc
     )
 class _DegradedReading(object):
-    # Duck type of psutil's namedtuple results (svmem field order):
-    # mozsystemmonitor subscripts, iterates, and unpacks readings, so the
-    # degraded fallback must survive r[0], list(r), len(r), r._fields and
-    # r._asdict(), not just attribute access.
+    # Last-resort duck type of psutil's namedtuple results (svmem field
+    # order): mozsystemmonitor subscripts, iterates, and unpacks readings,
+    # so the degraded fallback must survive r[0], list(r), len(r), r._fields
+    # and r._asdict(), not just attribute access. Field report (0.34.1): the
+    # parent also reconstructs readings via type(reading)(*values), so the
+    # constructor must tolerate the full positional field list. Normal
+    # degraded paths use the per-function namedtuple fallbacks below; this
+    # class only backstops a fallback that itself raised.
     _fields = ("total", "available", "percent", "used", "free", "active", "inactive", "wired")
     total = available = used = free = active = inactive = wired = 0
     percent = 0.0
+    def __init__(self, *_args, **_kwargs):
+        pass
     def __getattr__(self, _name):
         return 0
@@ -106,6 +156,34 @@ class _DegradedReading(object):
         return dict(zip(self._fields, self._values()))
+# Per-function fallback result classes with the exact macOS field
+# orders/arities. Module-level on purpose: readings cross the
+# mozsystemmonitor collector pipe via pickle (by reference), and the parent
+# rebuilds each sample with type(reading)(*values) — so a swap fallback must
+# be 6-field sswap-shaped in every position (field report 0.34.1: an
+# svmem-shaped fallback in the swap position made the parent reject every
+# sample, fill the pipe, block the collector child in send(), and wedge
+# mach's atexit join forever).
+_FallbackVmem = collections.namedtuple(
+    "_FallbackVmem",
+    ("total", "available", "percent", "used", "free", "active", "inactive", "wired"),
+)
+_FallbackSwap = collections.namedtuple(
+    "_FallbackSwap", ("total", "used", "free", "percent", "sin", "sout")
+)
+_FallbackCpuTimes = collections.namedtuple("_FallbackCpuTimes", ("user", "nice", "system", "idle"))
+_FallbackDiskIO = collections.namedtuple(
+    "_FallbackDiskIO",
+    ("read_count", "write_count", "read_bytes", "write_bytes", "read_time", "write_time"),
+)
+_FALLBACK_CLASSES = {
+    "virtual_memory": _FallbackVmem,
+    "swap_memory": _FallbackSwap,
+    "cpu_times": _FallbackCpuTimes,
+    "disk_io_counters": _FallbackDiskIO,
+}
 # psutil result namedtuple per wrapped function, resolvable via getattr from
 # the platform module / _common without invoking the failing syscall.
 _PSUTIL_RESULT_CLASSES = {
@@ -116,18 +194,51 @@ _PSUTIL_RESULT_CLASSES = {
 }
+def _zeroed(cls):
+    return cls(*([0] * len(cls._fields)))
+def _percpu_requested(args, kwargs, positional_index):
+    if "percpu" in kwargs:
+        return bool(kwargs["percpu"])
+    return len(args) > positional_index and bool(args[positional_index])
 def _degraded_result_factory(psutil, name):
-    # cpu_percent returns a plain float callers do arithmetic on.
+    # cpu_percent returns a plain float (or a per-CPU list) callers do
+    # arithmetic on.
     if name == "cpu_percent":
-        return lambda: 0.0
+        def _cpu_percent_fallback(*args, **kwargs):
+            # cpu_percent(interval=None, percpu=False)
+            if _percpu_requested(args, kwargs, 1):
+                return []
+            return 0.0
+        return _cpu_percent_fallback
+    # The guard-owned class is always shape-correct for this function;
+    # psutil's own result namedtuple is still preferred when resolvable so
+    # type() captures stay identical to real readings on a flapping host.
+    _cls = _FALLBACK_CLASSES.get(name, _DegradedReading)
     _cls_name = _PSUTIL_RESULT_CLASSES.get(name)
     if _cls_name is not None:
         for _mod in (getattr(psutil, "_psplatform", None), getattr(psutil, "_common", None)):
-            _cls = getattr(_mod, _cls_name, None) if _mod is not None else None
-            _fields = getattr(_cls, "_fields", None)
-            if _cls is not None and _fields:
-                return lambda _cls=_cls, _n=len(_fields): _cls(*([0] * _n))
-    return _DegradedReading
+            _real = getattr(_mod, _cls_name, None) if _mod is not None else None
+            if _real is not None and getattr(_real, "_fields", None):
+                _cls = _real
+                break
+    if name == "cpu_times":
+        def _cpu_times_fallback(*args, **kwargs):
+            # cpu_times(percpu=False); the collector child samples per-CPU.
+            if _percpu_requested(args, kwargs, 0):
+                return []
+            return _zeroed(_cls)
+        return _cpu_times_fallback
+    def _reading_fallback(*_args, **_kwargs):
+        return _zeroed(_cls)
+    return _reading_fallback
 def _guard_psutil():
@@ -143,7 +254,7 @@ def _guard_psutil():
             except Exception as exc:  # noqa: BLE001
                 _fireforge_degraded_notice(exc)
                 try:
-                    return fallback()
+                    return fallback(*args, **kwargs)
                 except Exception:  # noqa: BLE001
                     return _DegradedReading()
@@ -176,6 +287,54 @@ _MONITOR_METHOD_NAMES = (
 )
+def _fireforge_stop_collector(monitor):
+    # Best-effort: terminate a live collector child and drain the parent end
+    # of the pipe, so a child blocked in send() on a full pipe can never
+    # keep mach's atexit join (os.waitpid) waiting forever. Attribute names
+    # follow upstream resourcemonitor.py (self._process / self._pipe); a
+    # miss on a refactored upstream is harmless — the shape-correct
+    # fallbacks above still keep the sample stream well-formed.
+    try:
+        proc = getattr(monitor, "_process", None)
+        if proc is not None and getattr(proc, "is_alive", lambda: False)():
+            proc.terminate()
+            proc.join(1)
+    except Exception:  # noqa: BLE001
+        pass
+    try:
+        pipe = getattr(monitor, "_pipe", None)
+        if pipe is not None:
+            while pipe.poll(0):
+                pipe.recv()
+    except Exception:  # noqa: BLE001
+        pass
+def _monitor_per_cpu_requested(args, kwargs):
+    # aggregate_cpu_percent/aggregate_cpu_times(start=None, end=None,
+    # phase=None, per_cpu=True) — self already stripped from args.
+    if "per_cpu" in kwargs:
+        return bool(kwargs["per_cpu"])
+    if len(args) > 3:
+        return bool(args[3])
+    return True
+def _monitor_degraded_result(name, args, kwargs):
+    # Degraded aggregate methods return zeroed shapes, not None, where
+    # callers subscript the result: mozbuild's log_resource_usage does
+    # usage["io"].read_bytes after a successful compile (field report
+    # 0.34.1: 'NoneType' object has no attribute 'read_bytes' failed the
+    # build with complete artifacts).
+    if name == "aggregate_io":
+        return _zeroed(_FallbackDiskIO)
+    if name == "aggregate_cpu_percent":
+        return [] if _monitor_per_cpu_requested(args, kwargs) else 0.0
+    if name == "aggregate_cpu_times":
+        return [] if _monitor_per_cpu_requested(args, kwargs) else _zeroed(_FallbackCpuTimes)
+    return None
 def _patch_monitor_class(cls):
     if cls is None or getattr(cls, "_fireforge_guarded", False):
         return
@@ -192,38 +351,81 @@ def _patch_monitor_class(cls):
         except Exception as exc:  # noqa: BLE001
             self._fireforge_degraded = True
             _fireforge_degraded_notice(exc)
+        if _fireforge_host_degraded[0] and not self._fireforge_degraded:
+            # Host already degraded once: keep the monitor inert so the
+            # collector child never spawns on a flapping host.
+            self._fireforge_degraded = True
+        if self._fireforge_degraded:
+            _fireforge_stop_collector(self)
     cls.__init__ = guarded_init
-    def _wrap_method(orig):
+    def _wrap_method(name, orig):
         def wrapper(self, *args, **kwargs):
+            if (
+                name == "start"
+                and _fireforge_host_degraded[0]
+                and not getattr(self, "_fireforge_degraded", False)
+            ):
+                self._fireforge_degraded = True
+                _fireforge_stop_collector(self)
             if getattr(self, "_fireforge_degraded", False):
-                return None
+                if name == "stop":
+                    _fireforge_stop_collector(self)
+                return _monitor_degraded_result(name, args, kwargs)
             try:
                 return orig(self, *args, **kwargs)
             except Exception as exc:  # noqa: BLE001
                 self._fireforge_degraded = True
                 _fireforge_degraded_notice(exc)
-                return None
+                # A raising stop (or any degraded transition) must still
+                # take the collector child down, or the parent's atexit
+                # join hangs on a child blocked writing to a full pipe.
+                _fireforge_stop_collector(self)
+                return _monitor_degraded_result(name, args, kwargs)
         return wrapper
     for _name in _MONITOR_METHOD_NAMES:
         _orig = getattr(cls, _name, None)
         if _orig is not None:
-            setattr(cls, _name, _wrap_method(_orig))
+            setattr(cls, _name, _wrap_method(_name, _orig))
     cls._fireforge_guarded = True
-_MONITOR_MODULES = ("mozsystemmonitor.resourcemonitor", "mozbuild.resources")
+def _patch_build_monitor_class(cls):
+    # mozbuild's BuildMonitor is not a SystemResourceMonitor; only
+    # log_resource_usage needs guarding — end-of-build resource reporting
+    # must never fail a build whose artifacts are complete. No __init__
+    # replacement, no degraded-instance gating.
+    if cls is None or getattr(cls, "_fireforge_guarded", False):
+        return
+    _orig = getattr(cls, "log_resource_usage", None)
+    if _orig is not None:
+        def _guarded_log_resource_usage(self, *args, **kwargs):
+            try:
+                return _orig(self, *args, **kwargs)
+            except Exception as exc:  # noqa: BLE001
+                _fireforge_degraded_notice(exc)
+                return None
+        cls.log_resource_usage = _guarded_log_resource_usage
+    cls._fireforge_guarded = True
+_MONITOR_CLASS_PATCHES = (
+    ("mozsystemmonitor.resourcemonitor", "SystemResourceMonitor", _patch_monitor_class),
+    ("mozbuild.resources", "SystemResourceMonitor", _patch_monitor_class),
+    ("mozbuild.controller.building", "BuildMonitor", _patch_build_monitor_class),
+)
 def _patch_loaded_monitor_modules():
-    for _mod_name in _MONITOR_MODULES:
+    for _mod_name, _cls_name, _patcher in _MONITOR_CLASS_PATCHES:
         _mod = sys.modules.get(_mod_name)
         if _mod is not None:
             try:
-                _patch_monitor_class(getattr(_mod, "SystemResourceMonitor", None))
+                _patcher(getattr(_mod, _cls_name, None))
             except Exception:  # noqa: BLE001
                 pass

package/dist/src/core/test-harness-crash.js CHANGED Viewed

@@ -75,6 +75,13 @@ const STARTUP_TRACEBACK_SIGNALS = [
     /HOST_VM_INFO64/,
     /\(ipc\/mig\) array not large enough/,
     /psutil\.[A-Za-z]*Error/,
+    // Field report (0.34.1): on a degraded host every collector sample is
+    // rejected, aggregation has nothing, and mozbuild's log_resource_usage
+    // dies on usage["io"].read_bytes AFTER a fully successful compile
+    // ("Error running mach" with complete artifacts). Environmental, not a
+    // build regression — the protected build retries it (incremental retry
+    // is cheap and the guard keeps the retry green).
+    /AttributeError: 'NoneType' object has no attribute 'read_bytes'/,
     ...DEGRADED_READING_CRASH_SIGNALS,
 ];
 function findLine(output, patterns) {
@@ -102,6 +109,11 @@ const NOISE_LINE_PATTERNS = [
     /_collect failed(?!.*_DegradedReading)/i,
     /FireForge: host resource monitor degraded/i,
     /warnings\.warn\(/,
+    // mozsystemmonitor's parent-side drain loop rejecting a malformed sample
+    // (field report 0.34.1); mach sometimes reprints the warning text without
+    // the `UserWarning:` token, and a run that completes despite the stream
+    // must never classify as crash on it.
+    /failed to read the received data/i,
 ];
 /** Matches the caught/telemetry context that marks a traceback as benign. */
 const BENIGN_TRACEBACK_CONTEXT = /telemetry|glean/i;

package/package.json CHANGED Viewed

@@ -1,6 +1,6 @@
 {
   "name": "@hominis/fireforge",
-  "version": "0.34.1",
+  "version": "0.34.2",
   "description": "FireForge — a build tool for customizing Firefox",
   "type": "module",
   "main": "./dist/src/index.js",