engineering-process 1.0.1__tar.gz → 1.1.1__tar.gz

{engineering_process-1.0.1/engineering_process.egg-info → engineering_process-1.1.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: engineering-process
-Version: 1.0.1
+Version: 1.1.1
 Summary: A small agent-neutral engineering lifecycle with managed adoption
 License-Expression: MIT
 Project-URL: Homepage, https://github.com/phuongnse/engineering-process
@@ -92,10 +92,72 @@ Commands are argument arrays, never shell strings. Each command has a finite tim
 Output has a hard aggregate budget; evidence stores byte counts and hashes, never raw
 stdout or stderr that could contain secrets.
 
+### Production readiness
+
+A consumer declares `.process/readiness.json` with production as its direction, its
+current stage, immutable pack versions, and the state of every required capability.
+An enforced capability maps to project-required profiles; a planned capability names
+the concrete gap without pretending to have evidence:
+
+    {
+      "target": "production",
+      "stage": "production",
+      "packs": [{"id": "library-cli", "version": 1}],
+      "capabilities": [
+        {"id": "correctness", "state": "enforced", "evidenceProfiles": ["development"]},
+        {"id": "runtime-safety", "state": "enforced", "evidenceProfiles": ["development"]},
+        {"id": "compatibility", "state": "enforced", "evidenceProfiles": ["development"]},
+        {"id": "portability", "state": "enforced", "evidenceProfiles": ["development", "review"]},
+        {"id": "installability", "state": "enforced", "evidenceProfiles": ["review"]},
+        {"id": "distribution-integrity", "state": "enforced", "evidenceProfiles": ["review"]},
+        {"id": "adoption-integrity", "state": "enforced", "evidenceProfiles": ["development", "review"]}
+      ]
+    }
+
+`project validate` and `doctor` resolve that declaration to the exact checks owned by
+the consumer. The declaration does not make a weak command sufficient: normal CI and
+independent review still judge whether those commands prove the named capability. A
+building consumer may keep planned gaps while ordinary development continues. A
+production-stage declaration fails closed if any capability remains planned.
+
+The sidecar is a deliberate self-hosting boundary. Public authority N continues to
+validate the unchanged strict `.process/project.json` while source N+1 validates and
+self-applies the new readiness contract. Adoption leaves the consumer-owned sidecar
+in place, so every later authority can repeat the same forward-compatible sequence.
+Pack versions are also immutable: a process update must keep `operations@1` working
+even after `operations@2` exists. Process adoption and pack upgrades are separate
+consumer-owned changes, preventing a new standard from deadlocking authority adoption.
+
+`library-cli@1` was derived from this repository as a real producer and self-consumer.
+`operations@1` was then derived from renovate-ops and requires auditability, automation
+correctness, bounded execution, least privilege, policy integrity, recovery, and
+target-selection integrity. `desktop-media@1` is derived from LyricRail and keeps its
+existing correctness, input, source-portability, audit, media, package, and recovery-
+mechanism evidence enforced. Stable dependency/recovery claims, signing, key custody,
+runtime/license delivery, Linux advisory resolution, real-host workspace security,
+updater, incident recovery, and independent security review remain planned.
+Consumers without readiness remain compatible during that evidence-backed rollout.
+
+For each ordinary change, `run-change` first surfaces this readiness view. The accepted
+request and consumer rules determine which capabilities are affected. Every change
+retains the project's baseline `requiredProfiles`; start and plan add any conditional
+evidence profiles needed by affected capabilities and include a planned gap only when
+the accepted request explicitly selects it. Implement and review protect the enforced
+floor. A planned-to-enforced promotion is a reviewed consumer source diff with fresh
+evidence. Unrelated planned gaps remain visible but do not block development, and no
+skill chooses product priorities or changes readiness automatically.
+
+When a consumer incident exposes a reusable process gap, `improve-process` first keeps
+the consumer safe, then prepares a sanitized GitHub issue draft from that checkout.
+It deduplicates by consumer/process-version/invariant, requires owner authorization
+before `gh issue create`, and uses an accepted issue as the later process change source
+and `consumerEvidence`. No producer clone, consumer-CI write token, automatic process
+mutation, or wait for a process release is required to continue consumer development.
+
 Pin the process in requirements/process.in:
 
     --only-binary :all:
-    engineering-process==0.9.0
+    engineering-process==1.0.1
 
 Generate requirements/process.txt with hashes, install that lock, then run:
 
@@ -232,7 +294,7 @@ At any point:
 ## Release to consumer PR
 
 Every opted-in consumer uses Renovate's pip-compile manager. Its engineering-process
-package rule is enabled, never automerges, and runs exactly:
+package rule keeps the adoption pull request in draft and runs exactly:
 
     python .process/adopt-process.py --project-root . --requirements-lock requirements/process.txt
 
@@ -247,7 +309,8 @@ The release workflow publishes exact wheel and sdist bytes to PyPI, verifies the
 registry hashes, creates the immutable GitHub release, and sends one authenticated
 engineering-process-published event to renovate-ops. That control plane runs Renovate
 for each repository whose protected config explicitly opts in. Each consumer's normal
-CI and independent review decide whether its PR can merge.
+CI and independent review decide whether its draft PR can merge; the consumer owner
+authorizes that merge.
 
 This repository opts in through .github/renovate.json, so it receives the same
 adoption PR as every other consumer. See SELF_HOSTING.md and RELEASING.md.

{engineering_process-1.0.1 → engineering_process-1.1.1}/README.md

@@ -68,10 +68,72 @@ Commands are argument arrays, never shell strings. Each command has a finite tim
 Output has a hard aggregate budget; evidence stores byte counts and hashes, never raw
 stdout or stderr that could contain secrets.
 
+### Production readiness
+
+A consumer declares `.process/readiness.json` with production as its direction, its
+current stage, immutable pack versions, and the state of every required capability.
+An enforced capability maps to project-required profiles; a planned capability names
+the concrete gap without pretending to have evidence:
+
+    {
+      "target": "production",
+      "stage": "production",
+      "packs": [{"id": "library-cli", "version": 1}],
+      "capabilities": [
+        {"id": "correctness", "state": "enforced", "evidenceProfiles": ["development"]},
+        {"id": "runtime-safety", "state": "enforced", "evidenceProfiles": ["development"]},
+        {"id": "compatibility", "state": "enforced", "evidenceProfiles": ["development"]},
+        {"id": "portability", "state": "enforced", "evidenceProfiles": ["development", "review"]},
+        {"id": "installability", "state": "enforced", "evidenceProfiles": ["review"]},
+        {"id": "distribution-integrity", "state": "enforced", "evidenceProfiles": ["review"]},
+        {"id": "adoption-integrity", "state": "enforced", "evidenceProfiles": ["development", "review"]}
+      ]
+    }
+
+`project validate` and `doctor` resolve that declaration to the exact checks owned by
+the consumer. The declaration does not make a weak command sufficient: normal CI and
+independent review still judge whether those commands prove the named capability. A
+building consumer may keep planned gaps while ordinary development continues. A
+production-stage declaration fails closed if any capability remains planned.
+
+The sidecar is a deliberate self-hosting boundary. Public authority N continues to
+validate the unchanged strict `.process/project.json` while source N+1 validates and
+self-applies the new readiness contract. Adoption leaves the consumer-owned sidecar
+in place, so every later authority can repeat the same forward-compatible sequence.
+Pack versions are also immutable: a process update must keep `operations@1` working
+even after `operations@2` exists. Process adoption and pack upgrades are separate
+consumer-owned changes, preventing a new standard from deadlocking authority adoption.
+
+`library-cli@1` was derived from this repository as a real producer and self-consumer.
+`operations@1` was then derived from renovate-ops and requires auditability, automation
+correctness, bounded execution, least privilege, policy integrity, recovery, and
+target-selection integrity. `desktop-media@1` is derived from LyricRail and keeps its
+existing correctness, input, source-portability, audit, media, package, and recovery-
+mechanism evidence enforced. Stable dependency/recovery claims, signing, key custody,
+runtime/license delivery, Linux advisory resolution, real-host workspace security,
+updater, incident recovery, and independent security review remain planned.
+Consumers without readiness remain compatible during that evidence-backed rollout.
+
+For each ordinary change, `run-change` first surfaces this readiness view. The accepted
+request and consumer rules determine which capabilities are affected. Every change
+retains the project's baseline `requiredProfiles`; start and plan add any conditional
+evidence profiles needed by affected capabilities and include a planned gap only when
+the accepted request explicitly selects it. Implement and review protect the enforced
+floor. A planned-to-enforced promotion is a reviewed consumer source diff with fresh
+evidence. Unrelated planned gaps remain visible but do not block development, and no
+skill chooses product priorities or changes readiness automatically.
+
+When a consumer incident exposes a reusable process gap, `improve-process` first keeps
+the consumer safe, then prepares a sanitized GitHub issue draft from that checkout.
+It deduplicates by consumer/process-version/invariant, requires owner authorization
+before `gh issue create`, and uses an accepted issue as the later process change source
+and `consumerEvidence`. No producer clone, consumer-CI write token, automatic process
+mutation, or wait for a process release is required to continue consumer development.
+
 Pin the process in requirements/process.in:
 
     --only-binary :all:
-    engineering-process==0.9.0
+    engineering-process==1.0.1
 
 Generate requirements/process.txt with hashes, install that lock, then run:
 
@@ -208,7 +270,7 @@ At any point:
 ## Release to consumer PR
 
 Every opted-in consumer uses Renovate's pip-compile manager. Its engineering-process
-package rule is enabled, never automerges, and runs exactly:
+package rule keeps the adoption pull request in draft and runs exactly:
 
     python .process/adopt-process.py --project-root . --requirements-lock requirements/process.txt
 
@@ -223,7 +285,8 @@ The release workflow publishes exact wheel and sdist bytes to PyPI, verifies the
 registry hashes, creates the immutable GitHub release, and sends one authenticated
 engineering-process-published event to renovate-ops. That control plane runs Renovate
 for each repository whose protected config explicitly opts in. Each consumer's normal
-CI and independent review decide whether its PR can merge.
+CI and independent review decide whether its draft PR can merge; the consumer owner
+authorizes that merge.
 
 This repository opts in through .github/renovate.json, so it receives the same
 adoption PR as every other consumer. See SELF_HOSTING.md and RELEASING.md.

{engineering_process-1.0.1 → engineering_process-1.1.1}/engineering_process/__init__.py

@@ -1,3 +1,3 @@
 """Agent-neutral engineering process."""
 
-VERSION = "1.0.1"
+VERSION = "1.1.1"

{engineering_process-1.0.1 → engineering_process-1.1.1}/engineering_process/_supervisor_posix.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+from contextlib import suppress
 import ctypes
 import os
 from pathlib import Path
@@ -17,6 +18,8 @@ from .supervision import CleanupOutcome, NATURAL_DRAIN_GRACE_MILLISECONDS
 
 PR_SET_CHILD_SUBREAPER = 36
 RUN_ID_ENVIRONMENT_VARIABLE = "ENGINEERING_PROCESS_RUN_ID"
+PROCESS_TABLE_TIMEOUT_SECONDS = 10
+PROCESS_TABLE_OBSERVATION_INTERVAL_SECONDS = 0.05
 _SUBREAPER_ENABLED = False
 
 
@@ -31,39 +34,44 @@ def _enable_subreaper() -> None:
     _SUBREAPER_ENABLED = True
 
 
-def _process_table() -> dict[int, int]:
+def _ps_process_table() -> tuple[dict[int, int], str | None]:
+    try:
+        result = subprocess.run(
+            ["ps", "-axo", "pid=,ppid="],
+            stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL,
+            timeout=PROCESS_TABLE_TIMEOUT_SECONDS, check=False, text=True,
+        )
+    except subprocess.TimeoutExpired:
+        return {}, f"process table snapshot timed out after {PROCESS_TABLE_TIMEOUT_SECONDS} seconds"
+    except OSError:
+        return {}, "process table snapshot could not start"
+    if result.returncode != 0:
+        return {}, f"process table snapshot exited with status {result.returncode}"
+    try:
+        rows = (line.split() for line in result.stdout.splitlines())
+        table = {int(pid): int(parent) for pid, parent in rows}
+    except (TypeError, ValueError):
+        return {}, "process table snapshot was malformed"
+    if os.getpid() not in table:
+        return {}, "process table snapshot was incomplete"
+    return table, None
+
+
+def _process_table() -> tuple[dict[int, int], str | None]:
     table: dict[int, int] = {}
     if Path("/proc").is_dir():
         for entry in Path("/proc").iterdir():
             if not entry.name.isdecimal():
                 continue
             try:
-                for line in (entry / "status").read_text(
-                    encoding="utf-8", errors="replace"
-                ).splitlines():
+                for line in (entry / "status").read_text(encoding="utf-8", errors="replace").splitlines():
                     if line.startswith("PPid:"):
                         table[int(entry.name)] = int(line.split()[1])
                         break
             except (OSError, ValueError, IndexError):
                 continue
-        return table
-    result = subprocess.run(
-        ["ps", "-axo", "pid=,ppid="],
-        stdin=subprocess.DEVNULL,
-        stdout=subprocess.PIPE,
-        stderr=subprocess.DEVNULL,
-        timeout=2,
-        check=False,
-        text=True,
-    )
-    if result.returncode == 0:
-        for line in result.stdout.splitlines():
-            try:
-                pid, parent = (int(value) for value in line.split())
-            except (ValueError, TypeError):
-                continue
-            table[pid] = parent
-    return table
+        return table, None
+    return _ps_process_table()
 
 
 def _descendants(root: int, table: Mapping[int, int]) -> set[int]:
@@ -100,10 +108,8 @@ def _pid_alive(pid: int) -> bool:
         try:
             state = status.read_text(encoding="utf-8").rsplit(")", 1)[1].split()[0]
             if state == "Z":
-                try:
+                with suppress(OSError):
                     os.waitpid(pid, os.WNOHANG)
-                except OSError:
-                    pass
                 return False
         except (OSError, IndexError):
             pass
@@ -111,11 +117,9 @@ def _pid_alive(pid: int) -> bool:
 
 
 def _process_group_exists(process_group: int) -> bool:
-    try:
+    with suppress(OSError):
         while os.waitpid(-process_group, os.WNOHANG)[0]:
             pass
-    except OSError:
-        pass
     try:
         os.killpg(process_group, 0)
     except ProcessLookupError:
@@ -127,10 +131,8 @@ def _process_group_exists(process_group: int) -> bool:
 
 def _signal_processes(processes: set[int], signal_number: int) -> None:
     for pid in processes:
-        try:
+        with suppress(OSError):
             os.kill(pid, signal_number)
-        except OSError:
-            pass
 
 
 def _wait_for_process_group(process_group: int, timeout: float) -> bool:
@@ -145,20 +147,12 @@ def _wait_for_process_group(process_group: int, timeout: float) -> bool:
 def _terminate_group(process_group: int, grace_seconds: float) -> CleanupOutcome:
     if not _process_group_exists(process_group):
         return CleanupOutcome(bounded=True)
-    try:
+    with suppress(OSError):
         os.killpg(process_group, signal.SIGTERM)
-    except ProcessLookupError:
-        return CleanupOutcome(bounded=True, descendants_found=True)
-    except OSError:
-        pass
     if _wait_for_process_group(process_group, grace_seconds):
         return CleanupOutcome(bounded=True, descendants_found=True)
-    try:
+    with suppress(OSError):
         os.killpg(process_group, signal.SIGKILL)
-    except ProcessLookupError:
-        return CleanupOutcome(bounded=True, descendants_found=True)
-    except OSError:
-        pass
     bounded = _wait_for_process_group(process_group, grace_seconds)
     return CleanupOutcome(
         bounded=bounded,
@@ -167,10 +161,24 @@ def _terminate_group(process_group: int, grace_seconds: float) -> CleanupOutcome
     )
 
 
+def _terminate_root(process: subprocess.Popen[bytes], grace_seconds: float) -> bool:
+    for signal_number in (signal.SIGTERM, signal.SIGKILL):
+        with suppress(OSError):
+            os.killpg(process.pid, signal_number)
+        try:
+            process.wait(timeout=grace_seconds)
+            return True
+        except subprocess.TimeoutExpired:
+            pass
+    return False
+
+
 class PosixProcessSupervisor:
     def __init__(self) -> None:
         self._known_descendants: dict[int, set[int]] = {}
         self._run_ids: dict[int, str] = {}
+        self._last_observation: dict[int, float] = {}
+        self._observation_errors: dict[int, str] = {}
 
     def resolve_application(
         self,
@@ -234,37 +242,43 @@ class PosixProcessSupervisor:
         )
         self._known_descendants[process.pid] = set()
         self._run_ids[process.pid] = run_id
-        self.observe(process)
+        self._observation_errors.pop(process.pid, None)
+        self._observe(process, force=True)
         return process
 
-    def observe(self, process: subprocess.Popen[bytes]) -> None:
-        table = _process_table()
+    def _observe(self, process: subprocess.Popen[bytes], *, force: bool) -> None:
+        now = time.monotonic()
+        if not force and now - self._last_observation.get(process.pid, 0) < PROCESS_TABLE_OBSERVATION_INTERVAL_SECONDS:
+            return
+        table, error = _process_table()
+        self._last_observation[process.pid] = time.monotonic()
         known = self._known_descendants.setdefault(process.pid, set())
-        known.update(_descendants(process.pid, table))
-        run_id = self._run_ids.get(process.pid)
-        if sys.platform.startswith("linux") and run_id is not None:
-            adopted = {
-                pid
-                for pid, parent in table.items()
-                if (
-                    parent == os.getpid()
+        if error is not None:
+            self._observation_errors.setdefault(process.pid, error)
+        else:
+            known.update(_descendants(process.pid, table))
+            run_id = self._run_ids.get(process.pid)
+            if sys.platform.startswith("linux") and run_id is not None:
+                known.update(
+                    pid
+                    for pid, parent in table.items()
+                    if parent == os.getpid()
                     and pid != process.pid
                     and _has_run_id(pid, run_id)
                 )
-            }
-            known.update(adopted)
 
-    def _terminate_detached(
-        self,
-        process: subprocess.Popen[bytes],
-        grace_seconds: float,
-    ) -> CleanupOutcome:
-        self.observe(process)
+    def observe(self, process: subprocess.Popen[bytes]) -> None:
+        self._observe(process, force=False)
+
+    def _terminate_detached(self, process: subprocess.Popen[bytes], grace_seconds: float) -> CleanupOutcome:
+        self._observe(process, force=True)
         candidates = self._known_descendants.pop(process.pid, set())
         self._run_ids.pop(process.pid, None)
+        self._last_observation.pop(process.pid, None)
+        observation_error = self._observation_errors.pop(process.pid, None)
         live = {pid for pid in candidates if _pid_alive(pid)}
         if not live:
-            return CleanupOutcome(bounded=True)
+            return CleanupOutcome(bounded=observation_error is None, error=observation_error)
         _signal_processes(live, signal.SIGTERM)
         deadline = time.monotonic() + grace_seconds
         while time.monotonic() < deadline and any(_pid_alive(pid) for pid in live):
@@ -281,11 +295,16 @@ class PosixProcessSupervisor:
                 os.waitpid(pid, os.WNOHANG)
             except (ChildProcessError, OSError):
                 pass
-        bounded = not any(_pid_alive(pid) for pid in live)
+        descendants_bounded = not any(_pid_alive(pid) for pid in live)
+        error = observation_error or (
+            None
+            if descendants_bounded
+            else "detached descendants survived bounded termination"
+        )
         return CleanupOutcome(
-            bounded=bounded,
+            bounded=descendants_bounded and error is None,
             descendants_found=True,
-            error=None if bounded else "detached descendants survived bounded termination",
+            error=error,
         )
 
     def terminate(
@@ -294,34 +313,15 @@ class PosixProcessSupervisor:
         *,
         grace_seconds: float,
     ) -> CleanupOutcome:
-        self.observe(process)
-        try:
-            os.killpg(process.pid, signal.SIGTERM)
-        except ProcessLookupError:
-            pass
-        except OSError:
-            pass
-        try:
-            process.wait(timeout=grace_seconds)
-        except subprocess.TimeoutExpired:
-            try:
-                os.killpg(process.pid, signal.SIGKILL)
-            except ProcessLookupError:
-                pass
-            except OSError:
-                pass
-            try:
-                process.wait(timeout=grace_seconds)
-            except subprocess.TimeoutExpired:
-                detached = self._terminate_detached(process, grace_seconds)
-                return CleanupOutcome(
-                    bounded=False,
-                    descendants_found=detached.descendants_found,
-                    error=(
-                        "command root process survived bounded process-group termination"
-                        + (f"; {detached.error}" if detached.error else "")
-                    ),
-                )
+        self._observe(process, force=True)
+        if not _terminate_root(process, grace_seconds):
+            detached = self._terminate_detached(process, grace_seconds)
+            error = "command root process survived bounded process-group termination"
+            return CleanupOutcome(
+                bounded=False,
+                descendants_found=detached.descendants_found,
+                error=error + (f"; {detached.error}" if detached.error else ""),
+            )
         descendants = _terminate_group(process.pid, grace_seconds)
         detached = self._terminate_detached(process, grace_seconds)
         return CleanupOutcome(
@@ -342,7 +342,7 @@ class PosixProcessSupervisor:
         # alive, even after the original process has exited. Give descendants that
         # were synchronously asked to stop one short bounded interval to disappear
         # naturally before classifying them as abandoned background work.
-        self.observe(process)
+        self._observe(process, force=True)
         if not _process_group_exists(process.pid):
             return self._terminate_detached(process, grace_seconds)
         natural_drain_seconds = min(