repoactive 0.3.0__tar.gz → 0.3.1.dev2__tar.gz

{repoactive-0.3.0 → repoactive-0.3.1.dev2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: repoactive
-Version: 0.3.0
+Version: 0.3.1.dev2
 Summary: Script-driven code changes with automated merge requests
 License-Expression: GPL-3.0-or-later
 License-File: LICENSE
@@ -56,6 +56,7 @@ and (with `--mode publish`) the full MR lifecycle.
   - [`[platform.<name>]`](#platformname)
   - [Config file locations](#config-file-locations)
   - [Variables passed to job commands](#variables-passed-to-job-commands)
+  - [Passing secrets to commands with `secret_env`](#passing-secrets-to-commands-with-secret_env)
   - [Overriding values on the command line](#overriding-values-on-the-command-line)
 - [Selecting jobs with tags](#selecting-jobs-with-tags)
   - [Keeping unmerged branches current](#keeping-unmerged-branches-current)
@@ -494,6 +495,14 @@ block.
   works. The command runs as `<shell> -c <command>`. Arguments are not
   allowed - the value must be a single binary.
 
+**Secrets:**
+
+- **`secret_env`** (default: `[]`) - Names of environment variables to mark
+  as secrets config-wide. Each marked name is stripped from every job
+  command's environment, but a job still reads one only by listing it in its
+  own `secret_env`; `job-defaults` marks names, it does not grant them. See
+  [Passing secrets to commands with `secret_env`](#passing-secrets-to-commands-with-secret_env).
+
 ### `[job.<name>]`
 
 The table key is the job's unique name; the branch is always
@@ -570,6 +579,14 @@ underscores.
   `*.toml` job fragments into `$RA_JOBS_DIR` instead of producing a diff.
   See [Generating jobs dynamically](#generating-jobs-dynamically).
 
+**Secrets:**
+
+- **`secret_env`** (default: `[]`) - Names of environment variables holding
+  secrets this job's command may read. A marked name is stripped from every
+  command's environment; only a job that lists it here has it injected back
+  for its own command. See
+  [Passing secrets to commands with `secret_env`](#passing-secrets-to-commands-with-secret_env).
+
 ### Stacking MRs
 
 When `depends_on` is set, `repoactive` starts the job's script from a
@@ -770,6 +787,66 @@ whether the file was discovered automatically or named with `-c`. A command
 whose value came from a `--set` override has no config file, so
 `RA_CONFIG_SOURCE_DIR` is unset for it.
 
+### Passing secrets to commands with `secret_env`
+
+A job command inherits `repoactive`'s environment, so a secret exported
+before the run (an LLM key, a registry token) reaches it. Left unmanaged
+that secret reaches **every** job, whether it needs it or not. `secret_env`
+scopes secrets to the jobs that ask for them.
+
+The field separates two things:
+
+- **Marking** - naming a variable in _any_ `secret_env` (a job's or
+  `[job-defaults]`') marks it a secret and **removes it from every job
+  command's environment**.
+- **Granting** - a job reads a marked secret back only by listing it in
+  **its own** `secret_env`. `[job-defaults].secret_env` marks names but
+  grants to no job.
+
+So a secret is present only in the jobs that name it, never ambient:
+
+```toml
+[job-defaults]
+secret_env = ["OPENAI_API_KEY"]
+
+[job.rewrite-docs]
+command = "./llm-rewrite.sh"
+title = "docs: rewrite with the LLM"
+secret_env = ["OPENAI_API_KEY"]
+
+[job.format]
+command = "treefmt"
+title = "format the tree"
+```
+
+Here `OPENAI_API_KEY` is marked in `[job-defaults]`, so `format` (which does
+not grant it) runs without it in its environment; only `rewrite-docs`, which
+lists it in its own `secret_env`, can read it.
+
+A job that grants a secret which is **unset** in `repoactive`'s environment
+fails at its start with a clear error, rather than letting the command fail
+obscurely later. Secret **values** must never be written in config - a
+`.repoactive.toml` is checked into the repo; `secret_env` names variables,
+your environment (or your CI's secret store) supplies the values. The `RA_`
+and `REPOACTIVE_` prefixes are reserved and rejected.
+
+A [generated job](#generating-jobs-dynamically) may only grant a secret that
+the static config already marks; an emitted `secret_env` naming a variable
+no static job or `[job-defaults]` marked is rejected. Mark such a secret up
+front in `[job-defaults].secret_env` (or on the generator), and the emitted
+jobs grant it.
+
+A platform's `token_env` is normally stripped from every command
+([the platform token](#platformname) is kept out of jobs). A job that
+genuinely needs it can opt back in by naming that variable in its own
+`secret_env`; only that job then receives it.
+
+> **Note:** `secret_env` scopes _which jobs_ can read a secret. It does not
+> yet redact secret values from a command's captured output, so a command
+> that echoes a secret it was granted can still write it into the commit
+> message. Avoid printing granted secrets. See
+> [ADR 0017](docs/adr/0017-secret-env-redaction.md).
+
 ### Overriding values on the command line
 
 `--set NAME=VALUE`/`-s` tweaks individual config values without editing a

{repoactive-0.3.0 → repoactive-0.3.1.dev2}/README.md

@@ -29,6 +29,7 @@ and (with `--mode publish`) the full MR lifecycle.
   - [`[platform.<name>]`](#platformname)
   - [Config file locations](#config-file-locations)
   - [Variables passed to job commands](#variables-passed-to-job-commands)
+  - [Passing secrets to commands with `secret_env`](#passing-secrets-to-commands-with-secret_env)
   - [Overriding values on the command line](#overriding-values-on-the-command-line)
 - [Selecting jobs with tags](#selecting-jobs-with-tags)
   - [Keeping unmerged branches current](#keeping-unmerged-branches-current)
@@ -467,6 +468,14 @@ block.
   works. The command runs as `<shell> -c <command>`. Arguments are not
   allowed - the value must be a single binary.
 
+**Secrets:**
+
+- **`secret_env`** (default: `[]`) - Names of environment variables to mark
+  as secrets config-wide. Each marked name is stripped from every job
+  command's environment, but a job still reads one only by listing it in its
+  own `secret_env`; `job-defaults` marks names, it does not grant them. See
+  [Passing secrets to commands with `secret_env`](#passing-secrets-to-commands-with-secret_env).
+
 ### `[job.<name>]`
 
 The table key is the job's unique name; the branch is always
@@ -543,6 +552,14 @@ underscores.
   `*.toml` job fragments into `$RA_JOBS_DIR` instead of producing a diff.
   See [Generating jobs dynamically](#generating-jobs-dynamically).
 
+**Secrets:**
+
+- **`secret_env`** (default: `[]`) - Names of environment variables holding
+  secrets this job's command may read. A marked name is stripped from every
+  command's environment; only a job that lists it here has it injected back
+  for its own command. See
+  [Passing secrets to commands with `secret_env`](#passing-secrets-to-commands-with-secret_env).
+
 ### Stacking MRs
 
 When `depends_on` is set, `repoactive` starts the job's script from a
@@ -743,6 +760,66 @@ whether the file was discovered automatically or named with `-c`. A command
 whose value came from a `--set` override has no config file, so
 `RA_CONFIG_SOURCE_DIR` is unset for it.
 
+### Passing secrets to commands with `secret_env`
+
+A job command inherits `repoactive`'s environment, so a secret exported
+before the run (an LLM key, a registry token) reaches it. Left unmanaged
+that secret reaches **every** job, whether it needs it or not. `secret_env`
+scopes secrets to the jobs that ask for them.
+
+The field separates two things:
+
+- **Marking** - naming a variable in _any_ `secret_env` (a job's or
+  `[job-defaults]`') marks it a secret and **removes it from every job
+  command's environment**.
+- **Granting** - a job reads a marked secret back only by listing it in
+  **its own** `secret_env`. `[job-defaults].secret_env` marks names but
+  grants to no job.
+
+So a secret is present only in the jobs that name it, never ambient:
+
+```toml
+[job-defaults]
+secret_env = ["OPENAI_API_KEY"]
+
+[job.rewrite-docs]
+command = "./llm-rewrite.sh"
+title = "docs: rewrite with the LLM"
+secret_env = ["OPENAI_API_KEY"]
+
+[job.format]
+command = "treefmt"
+title = "format the tree"
+```
+
+Here `OPENAI_API_KEY` is marked in `[job-defaults]`, so `format` (which does
+not grant it) runs without it in its environment; only `rewrite-docs`, which
+lists it in its own `secret_env`, can read it.
+
+A job that grants a secret which is **unset** in `repoactive`'s environment
+fails at its start with a clear error, rather than letting the command fail
+obscurely later. Secret **values** must never be written in config - a
+`.repoactive.toml` is checked into the repo; `secret_env` names variables,
+your environment (or your CI's secret store) supplies the values. The `RA_`
+and `REPOACTIVE_` prefixes are reserved and rejected.
+
+A [generated job](#generating-jobs-dynamically) may only grant a secret that
+the static config already marks; an emitted `secret_env` naming a variable
+no static job or `[job-defaults]` marked is rejected. Mark such a secret up
+front in `[job-defaults].secret_env` (or on the generator), and the emitted
+jobs grant it.
+
+A platform's `token_env` is normally stripped from every command
+([the platform token](#platformname) is kept out of jobs). A job that
+genuinely needs it can opt back in by naming that variable in its own
+`secret_env`; only that job then receives it.
+
+> **Note:** `secret_env` scopes _which jobs_ can read a secret. It does not
+> yet redact secret values from a command's captured output, so a command
+> that echoes a secret it was granted can still write it into the commit
+> message. Avoid printing granted secrets. See
+> [ADR 0017](docs/adr/0017-secret-env-redaction.md).
+
 ### Overriding values on the command line
 
 `--set NAME=VALUE`/`-s` tweaks individual config values without editing a

{repoactive-0.3.0 → repoactive-0.3.1.dev2}/pyproject.toml

@@ -10,7 +10,7 @@ only-include = ["src"]
 
 [project]
 name = "repoactive"
-version = "0.3.0"
+version = "0.3.1.dev2"
 description = "Script-driven code changes with automated merge requests"
 readme = "README.md"
 requires-python = ">=3.11"

{repoactive-0.3.0 → repoactive-0.3.1.dev2}/src/repoactive/config.py

@@ -33,6 +33,10 @@ logger = logging.getLogger(__name__)
 _DURATION_RE = re.compile(r"^(\d+)([smhdw])$")
 _JOB_NAME_RE = re.compile(r"^[A-Za-z0-9_-]+$")
 _TAG_RE = re.compile(r"^[A-Za-z0-9_-]+$")
+# Environment-variable name grammar for secret_env entries. The RA_ and
+# REPOACTIVE_ prefixes are repoactive's own (ADR 0016) and are rejected.
+_ENV_NAME_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
+_RESERVED_ENV_PREFIXES = ("RA_", "REPOACTIVE_")
 # An empty prefix is allowed (branch name == job name); hence * not +.
 _BRANCH_PREFIX_RE = re.compile(r"^(?!/)(?!.*//)[a-zA-Z0-9_\-/]*$")
 _DURATION_UNITS = {"s": "seconds", "m": "minutes", "h": "hours", "d": "days", "w": "weeks"}
@@ -91,6 +95,27 @@ class InvalidTagError(ValueError):
         super().__init__(f"invalid tag {tag!r}: only letters, digits, '-', and '_' are allowed")
 
 
+class InvalidSecretEnvNameError(ValueError):
+    """Raised when a secret_env entry is not a valid environment-variable name."""
+
+    def __init__(self, name: str) -> None:
+        super().__init__(
+            f"invalid secret_env name {name!r}: must be a valid environment-variable name "
+            "(letters, digits, and '_', not starting with a digit)"
+        )
+
+
+class ReservedSecretEnvNameError(ValueError):
+    """Raised when a secret_env entry uses a prefix reserved for repoactive."""
+
+    def __init__(self, name: str) -> None:
+        super().__init__(
+            f"invalid secret_env name {name!r}: the "
+            f"{' and '.join(_RESERVED_ENV_PREFIXES)} prefixes are reserved for repoactive "
+            "(see docs/adr/0016-injected-env-var-prefix.md)"
+        )
+
+
 class NewlineInTitleError(ValueError):
     """Raised when a title or prefix field contains a newline character."""
 
@@ -278,9 +303,19 @@ def _validate_single_line(value: str, info: ValidationInfo) -> str:
     return value
 
 
+def _validate_secret_env(value: list[str]) -> list[str]:
+    for name in value:
+        if not _ENV_NAME_RE.match(name):
+            raise InvalidSecretEnvNameError(name)
+        if name.startswith(_RESERVED_ENV_PREFIXES):
+            raise ReservedSecretEnvNameError(name)
+    return value
+
+
 _BranchPrefix = Annotated[str, AfterValidator(_validate_branch_prefix)]
 _Duration = Annotated[str, AfterValidator(_validate_duration)]
 _Shell = Annotated[str, AfterValidator(_validate_shell)]
+_SecretEnv = Annotated[list[str], AfterValidator(_validate_secret_env)]
 # Titles and title prefixes end up in commit subjects and MR titles, which are
 # single-line by nature.
 _SingleLine = Annotated[str, AfterValidator(_validate_single_line)]
@@ -306,6 +341,12 @@ class JobDefaults(BaseModel):
     # `<shell> -c <command>`. None means /bin/sh (the shell=True default). May be a
     # bare name resolved on PATH (e.g. "bash") or an absolute path.
     shell: _Shell | None = None
+    # Names of environment variables to mark as secrets config-wide: each is
+    # stripped from every job command's environment, but a job still reads one
+    # only by listing it in its own secret_env. Marks names, never grants them;
+    # deliberately not inherited into jobs like the defaulted fields above. See
+    # docs/adr/0017-secret-env-redaction.md.
+    secret_env: _SecretEnv = Field(default_factory=list)
 
 
 # Fields Job.resolve fills in from JobDefaults when the job does not set them
@@ -328,6 +369,12 @@ class Job(BaseModel):
 
     name: str
     command: str
+    # Names of environment variables holding secrets this job's command may read.
+    # A name listed in any secret_env (here, another job's, or job-defaults') is
+    # stripped from every command's base environment; only a job that lists a name
+    # here has that variable injected back for its own command. See
+    # docs/adr/0017-secret-env-redaction.md.
+    secret_env: _SecretEnv = Field(default_factory=list)
     title: _SingleLine
     description: str | None = None
     base_branch: str | None = None
@@ -589,6 +636,19 @@ class Config(BaseModel):
         """
         return {p.token_env for p in self.platforms}
 
+    def marked_secret_names(self) -> set[str]:
+        """Names marked as managed secrets: every secret_env in the merged config.
+
+        The union of ``job-defaults`` and every job's ``secret_env``. A marked
+        name is stripped from every job command's base environment; a job reads
+        it back only by listing it in its own ``secret_env`` (see
+        runner._run_command and docs/adr/0017-secret-env-redaction.md).
+        """
+        names = set(self.job_defaults.secret_env)
+        for job in self.jobs:
+            names.update(job.secret_env)
+        return names
+
 
 class ConfigShape(BaseModel):
     """Structural shape of a raw config source, validated before merging.

{repoactive-0.3.0 → repoactive-0.3.1.dev2}/src/repoactive/runner.py

@@ -130,6 +130,18 @@ class CommandError(RuntimeError):
         self.elapsed = elapsed
 
 
+class MissingSecretError(RuntimeError):
+    """A job granted a secret_env variable that is unset in repoactive's environment.
+
+    Raised before the command runs so the failure is legible (ADR 0017) instead
+    of surfacing as an obscure command error later.
+    """
+
+    def __init__(self, name: str) -> None:
+        super().__init__(f"requires secret {name}, not set")
+        self.name = name
+
+
 class GeneratedJobError(ValueError):
     """Raised when a generator emits an invalid job set.
 
@@ -236,10 +248,12 @@ class RunContext:
     plan: UpdatePlan = field(default_factory=UpdatePlan)
 
     @property
-    def secret_env_names(self) -> frozenset[str]:
-        # Token vars stripped from every job command's environment so a command
-        # cannot read the credential repoactive uses to push/create MRs.
-        return frozenset(self.config.token_env_names())
+    def stripped_env_names(self) -> frozenset[str]:
+        # Names removed from every job command's base environment: the platform
+        # tokens (ADR 0006) plus every marked secret (ADR 0017). A job reads a
+        # marked secret back only by granting it in its own secret_env; see
+        # _resolve_granted_secrets.
+        return frozenset(self.config.token_env_names() | self.config.marked_secret_names())
 
 
 def _compute_parents(job: Job, results: dict[str, JobResult]) -> list[str]:
@@ -268,22 +282,44 @@ def _kill_process_group(proc: subprocess.Popen[str]) -> None:
 
 
 def _command_env(
-    *, extra_env: dict[str, str] | None, secret_env_names: frozenset[str]
+    *,
+    extra_env: dict[str, str] | None,
+    stripped_env_names: frozenset[str],
+    granted_secret_env: dict[str, str] | None = None,
 ) -> dict[str, str]:
     """Build the environment a job command runs in.
 
     Starts from the inherited environment (so the command still sees PATH etc.),
-    drops the platform token variables (``secret_env_names``) so a command cannot read
-    the credential repoactive uses to push/create MRs, then layers on ``extra_env``
-    (e.g. RA_JOBS_DIR for a generator). See
-    docs/adr/0006-job-commands-are-trusted.md.
+    drops ``stripped_env_names`` (the platform tokens of ADR 0006 plus every
+    marked secret of ADR 0017), injects back only the secrets this job granted
+    (``granted_secret_env``), then layers on ``extra_env`` (the RA_* variables,
+    e.g. RA_JOBS_DIR for a generator) last so repoactive's own variables win.
     """
-    env = {k: v for k, v in os.environ.items() if k not in secret_env_names}
+    env = {k: v for k, v in os.environ.items() if k not in stripped_env_names}
+    if granted_secret_env:
+        env.update(granted_secret_env)
     if extra_env:
         env.update(extra_env)
     return env
 
 
+def _resolve_granted_secrets(job: Job) -> dict[str, str]:
+    """Values for the secrets ``job`` grants, read from repoactive's own environment.
+
+    Only the names in the job's own ``secret_env`` are granted; ``job-defaults``
+    marks names but grants to no job (ADR 0017). Raises MissingSecretError on the
+    first granted name that is unset, so a misconfigured job fails legibly before
+    its command runs rather than deep inside it.
+    """
+    granted: dict[str, str] = {}
+    for name in job.secret_env:
+        try:
+            granted[name] = os.environ[name]
+        except KeyError:
+            raise MissingSecretError(name) from None
+    return granted
+
+
 def _job_extra_env(job: Job, extra: dict[str, str] | None = None) -> dict[str, str]:
     """Extra environment for ``job``'s command: its name, branches, config dir, ``extra``.
 
@@ -375,15 +411,18 @@ def _run_command(
     job: Job,
     cwd: Path,
     *,
-    secret_env_names: frozenset[str] = frozenset(),
+    stripped_env_names: frozenset[str] = frozenset(),
     extra_env: dict[str, str] | None = None,
 ) -> CommandResult:
     start = time.monotonic()
+    # Fail before the command runs if a granted secret is unset (ADR 0017).
+    granted_secret_env = _resolve_granted_secrets(job)
     # The workspace is always cwd, but expose it explicitly so a command that
     # cd's elsewhere can still find the workspace repoactive prepared for it.
     env = _command_env(
         extra_env={**(extra_env or {}), RA_WORKSPACE_DIR_ENV: str(cwd)},
-        secret_env_names=secret_env_names,
+        stripped_env_names=stripped_env_names,
+        granted_secret_env=granted_secret_env,
     )
 
     # Stream the merged stdout/stderr line by line: keep the full output (needed
@@ -558,7 +597,7 @@ def run_job(
             command_result = _run_command(
                 job,
                 repo.cwd,
-                secret_env_names=ctx.secret_env_names,
+                stripped_env_names=ctx.stripped_env_names,
                 extra_env=_job_extra_env(job),
             )
         except CommandError:
@@ -606,16 +645,23 @@ def _load_job_specs(jobs_dir: Path) -> dict[str, dict]:
     return specs
 
 
-def _build_generated_job(
-    *, generator: Job, name: str, spec: dict, run_names: set[str], all_config_names: set[str]
+def _build_generated_job(  # noqa: PLR0913
+    *,
+    generator: Job,
+    name: str,
+    spec: dict,
+    run_names: set[str],
+    all_config_names: set[str],
+    marked_secret_names: frozenset[str],
 ) -> Job:
     """Build one emitted ``Job`` from its raw spec, applying inheritance.
 
     ``name`` is the spec's table key. The job inherits the (resolved) generator's
     tags, ``depends_on`` and the ``_INHERITED_FIELDS`` unless the spec overrides
     them, and records the generator in ``generated_by``. Raises GeneratedJobError
-    on a name colliding with an existing job, a nested generator, or a job that
-    fails validation.
+    on a name colliding with an existing job, a nested generator, a job that
+    fails validation, or a ``secret_env`` naming a secret the static config did
+    not already mark (``marked_secret_names``).
     """
     if name in run_names or name in all_config_names:
         raise GeneratedJobError(
@@ -637,13 +683,32 @@ def _build_generated_job(
     # meaningful config location is the generator's own config source.
     merged["config_source_dir"] = generator.config_source_dir
     try:
-        return Job.model_validate(merged)
+        job = Job.model_validate(merged)
     except ValidationError as e:
         raise GeneratedJobError(generator.name, f"emitted job {name!r} is invalid: {e}") from e
+    # A generated job may only grant secrets the static config already marked. The
+    # env strip set is derived from the static config (RunContext.stripped_env_names),
+    # so a secret first introduced by an emitted job would not be stripped from the
+    # other jobs' environments; requiring it be marked up front (e.g. in
+    # [job-defaults].secret_env or on the generator) keeps a secret out of every job
+    # that did not grant it. See docs/adr/0017-secret-env-redaction.md.
+    unmarked = sorted(set(job.secret_env) - marked_secret_names)
+    if unmarked:
+        raise GeneratedJobError(
+            generator.name,
+            f"emitted job {name!r} grants secret(s) not marked in the static config: "
+            f"{unmarked}; add them to [job-defaults].secret_env or the generator's secret_env",
+        )
+    return job
 
 
 def _build_generated_jobs(
-    *, generator: Job, specs: dict[str, dict], run_names: set[str], all_config_names: set[str]
+    *,
+    generator: Job,
+    specs: dict[str, dict],
+    run_names: set[str],
+    all_config_names: set[str],
+    marked_secret_names: frozenset[str] = frozenset(),
 ) -> list[Job]:
     """Turn a generator's raw specs into validated ``Job`` objects.
 
@@ -660,6 +725,7 @@ def _build_generated_jobs(
             spec=spec,
             run_names=run_names,
             all_config_names=all_config_names,
+            marked_secret_names=marked_secret_names,
         )
         for name, spec in specs.items()
     ]
@@ -872,7 +938,7 @@ def _run_generator_job(
             _run_command(
                 job,
                 repo.cwd,
-                secret_env_names=ctx.secret_env_names,
+                stripped_env_names=ctx.stripped_env_names,
                 extra_env=_job_extra_env(job, {RA_JOBS_DIR_ENV: str(jobs_dir)}),
             )
             specs = _load_job_specs(jobs_dir)
@@ -886,6 +952,7 @@ def _run_generator_job(
         specs=specs,
         run_names={j.name for j in ctx.selection.jobs},
         all_config_names={j.name for j in ctx.config.jobs},
+        marked_secret_names=frozenset(ctx.config.marked_secret_names()),
     )
 
     names = ", ".join(j.name for j in emitted) if emitted else "none"