pipelinemd 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- pipelinemd-0.1.0/.dockerignore +12 -0
- pipelinemd-0.1.0/.gitignore +26 -0
- pipelinemd-0.1.0/CHANGELOG.md +55 -0
- pipelinemd-0.1.0/Dockerfile +68 -0
- pipelinemd-0.1.0/LICENSE +21 -0
- pipelinemd-0.1.0/Makefile +130 -0
- pipelinemd-0.1.0/PKG-INFO +337 -0
- pipelinemd-0.1.0/README.md +287 -0
- pipelinemd-0.1.0/corpus/README.md +87 -0
- pipelinemd-0.1.0/corpus/corpus.jsonl +62 -0
- pipelinemd-0.1.0/corpus/traces/artifact-absolute-path.log +21 -0
- pipelinemd-0.1.0/corpus/traces/artifact-download-404.log +21 -0
- pipelinemd-0.1.0/corpus/traces/artifact-expired.log +22 -0
- pipelinemd-0.1.0/corpus/traces/artifact-no-matching-files.log +29 -0
- pipelinemd-0.1.0/corpus/traces/artifact-too-large.log +23 -0
- pipelinemd-0.1.0/corpus/traces/cache-extract-failed.log +23 -0
- pipelinemd-0.1.0/corpus/traces/cache-missing-first-run.log +23 -0
- pipelinemd-0.1.0/corpus/traces/cache-s3-credentials.log +22 -0
- pipelinemd-0.1.0/corpus/traces/civars-aws-expired-token.log +20 -0
- pipelinemd-0.1.0/corpus/traces/civars-aws-missing-credentials.log +20 -0
- pipelinemd-0.1.0/corpus/traces/civars-git-push-token-expired.log +21 -0
- pipelinemd-0.1.0/corpus/traces/civars-kube-unauthorized.log +20 -0
- pipelinemd-0.1.0/corpus/traces/civars-masked-empty-header.log +20 -0
- pipelinemd-0.1.0/corpus/traces/civars-npm-registry-401.log +23 -0
- pipelinemd-0.1.0/corpus/traces/civars-protected-var-missing.log +23 -0
- pipelinemd-0.1.0/corpus/traces/civars-registry-push-denied.log +21 -0
- pipelinemd-0.1.0/corpus/traces/flaky-apt-mirror-unreachable.log +23 -0
- pipelinemd-0.1.0/corpus/traces/flaky-connection-reset.log +21 -0
- pipelinemd-0.1.0/corpus/traces/flaky-dns-resolution.log +23 -0
- pipelinemd-0.1.0/corpus/traces/flaky-external-api-timeout.log +20 -0
- pipelinemd-0.1.0/corpus/traces/flaky-gitlab-502.log +22 -0
- pipelinemd-0.1.0/corpus/traces/flaky-runner-lost.log +9 -0
- pipelinemd-0.1.0/corpus/traces/flaky-service-not-ready.log +21 -0
- pipelinemd-0.1.0/corpus/traces/flaky-test-passes-on-retry.log +23 -0
- pipelinemd-0.1.0/corpus/traces/flaky-tls-unknown-authority.log +20 -0
- pipelinemd-0.1.0/corpus/traces/imagepull-arch-mismatch.log +20 -0
- pipelinemd-0.1.0/corpus/traces/imagepull-dind-tls-mismatch.log +20 -0
- pipelinemd-0.1.0/corpus/traces/imagepull-dockerhub-rate-limit.log +20 -0
- pipelinemd-0.1.0/corpus/traces/imagepull-manifest-unknown.log +20 -0
- pipelinemd-0.1.0/corpus/traces/imagepull-no-dind-service.log +20 -0
- pipelinemd-0.1.0/corpus/traces/imagepull-preparation-failed.log +9 -0
- pipelinemd-0.1.0/corpus/traces/imagepull-prepare-tag-missing.log +21 -0
- pipelinemd-0.1.0/corpus/traces/imagepull-private-no-login.log +20 -0
- pipelinemd-0.1.0/corpus/traces/imagepull-registry-unauthorized.log +20 -0
- pipelinemd-0.1.0/corpus/traces/imagepull-service-image-missing.log +9 -0
- pipelinemd-0.1.0/corpus/traces/runner-canceled-sigterm.log +21 -0
- pipelinemd-0.1.0/corpus/traces/runner-docker-daemon-down.log +15 -0
- pipelinemd-0.1.0/corpus/traces/runner-gradle-daemon-lost.log +21 -0
- pipelinemd-0.1.0/corpus/traces/runner-job-timeout.log +22 -0
- pipelinemd-0.1.0/corpus/traces/runner-log-limit.log +23 -0
- pipelinemd-0.1.0/corpus/traces/runner-no-space.log +21 -0
- pipelinemd-0.1.0/corpus/traces/runner-node-heap-oom.log +23 -0
- pipelinemd-0.1.0/corpus/traces/runner-none-available.log +10 -0
- pipelinemd-0.1.0/corpus/traces/runner-oom-killed.log +21 -0
- pipelinemd-0.1.0/corpus/traces/runner-system-failure.log +9 -0
- pipelinemd-0.1.0/corpus/traces/test-eslint-errors.log +26 -0
- pipelinemd-0.1.0/corpus/traces/test-go-fail.log +24 -0
- pipelinemd-0.1.0/corpus/traces/test-jest-snapshot.log +26 -0
- pipelinemd-0.1.0/corpus/traces/test-junit-surefire.log +23 -0
- pipelinemd-0.1.0/corpus/traces/test-phpunit-failure.log +30 -0
- pipelinemd-0.1.0/corpus/traces/test-pytest-assertion.log +28 -0
- pipelinemd-0.1.0/corpus/traces/test-rspec-failure.log +28 -0
- pipelinemd-0.1.0/corpus/traces/test-tsc-type-errors.log +22 -0
- pipelinemd-0.1.0/corpus/traces/test-vitest-failure.log +23 -0
- pipelinemd-0.1.0/corpus/traces/yaml-anchor-missing-script.log +21 -0
- pipelinemd-0.1.0/corpus/traces/yaml-empty-variable-expansion.log +21 -0
- pipelinemd-0.1.0/corpus/traces/yaml-invalid-include.log +21 -0
- pipelinemd-0.1.0/corpus/traces/yaml-no-visible-jobs.log +21 -0
- pipelinemd-0.1.0/corpus/traces/yaml-reference-tag-loop.log +21 -0
- pipelinemd-0.1.0/corpus/traces/yaml-trigger-invalid-config.log +22 -0
- pipelinemd-0.1.0/corpus/traces/yaml-unknown-stage.log +21 -0
- pipelinemd-0.1.0/corpus/traces/yaml-yamllint-indentation.log +23 -0
- pipelinemd-0.1.0/docs/architecture.md +306 -0
- pipelinemd-0.1.0/docs/evaluation.md +434 -0
- pipelinemd-0.1.0/docs/releasing.md +117 -0
- pipelinemd-0.1.0/examples/gitlab-ci-diagnose.yml +41 -0
- pipelinemd-0.1.0/pyproject.toml +88 -0
- pipelinemd-0.1.0/scripts/release_notes.py +107 -0
- pipelinemd-0.1.0/src/pipelinemd/__init__.py +43 -0
- pipelinemd-0.1.0/src/pipelinemd/__main__.py +10 -0
- pipelinemd-0.1.0/src/pipelinemd/assessment.py +94 -0
- pipelinemd-0.1.0/src/pipelinemd/cli.py +692 -0
- pipelinemd-0.1.0/src/pipelinemd/config.py +100 -0
- pipelinemd-0.1.0/src/pipelinemd/corpus.py +202 -0
- pipelinemd-0.1.0/src/pipelinemd/cost.py +188 -0
- pipelinemd-0.1.0/src/pipelinemd/diagnose/__init__.py +16 -0
- pipelinemd-0.1.0/src/pipelinemd/diagnose/citations.py +95 -0
- pipelinemd-0.1.0/src/pipelinemd/diagnose/claude.py +188 -0
- pipelinemd-0.1.0/src/pipelinemd/diagnose/prompt.py +235 -0
- pipelinemd-0.1.0/src/pipelinemd/distill/__init__.py +22 -0
- pipelinemd-0.1.0/src/pipelinemd/distill/ansi.py +61 -0
- pipelinemd-0.1.0/src/pipelinemd/distill/distiller.py +54 -0
- pipelinemd-0.1.0/src/pipelinemd/distill/extract.py +314 -0
- pipelinemd-0.1.0/src/pipelinemd/distill/redact.py +135 -0
- pipelinemd-0.1.0/src/pipelinemd/distill/trace.py +180 -0
- pipelinemd-0.1.0/src/pipelinemd/errors.py +45 -0
- pipelinemd-0.1.0/src/pipelinemd/evaluate.py +421 -0
- pipelinemd-0.1.0/src/pipelinemd/gitlab/__init__.py +14 -0
- pipelinemd-0.1.0/src/pipelinemd/gitlab/client.py +109 -0
- pipelinemd-0.1.0/src/pipelinemd/gitlab/http.py +150 -0
- pipelinemd-0.1.0/src/pipelinemd/gitlab/url.py +122 -0
- pipelinemd-0.1.0/src/pipelinemd/models.py +440 -0
- pipelinemd-0.1.0/src/pipelinemd/render/__init__.py +22 -0
- pipelinemd-0.1.0/src/pipelinemd/render/evidence.py +87 -0
- pipelinemd-0.1.0/src/pipelinemd/render/json_out.py +216 -0
- pipelinemd-0.1.0/src/pipelinemd/render/markdown.py +154 -0
- pipelinemd-0.1.0/src/pipelinemd/render/style.py +77 -0
- pipelinemd-0.1.0/src/pipelinemd/render/terminal.py +257 -0
- pipelinemd-0.1.0/src/pipelinemd/rules/__init__.py +6 -0
- pipelinemd-0.1.0/src/pipelinemd/rules/catalog.py +1316 -0
- pipelinemd-0.1.0/src/pipelinemd/rules/engine.py +210 -0
- pipelinemd-0.1.0/src/pipelinemd/taxonomy.py +288 -0
- pipelinemd-0.1.0/tests/__init__.py +1 -0
- pipelinemd-0.1.0/tests/conftest.py +39 -0
- pipelinemd-0.1.0/tests/fixtures/__init__.py +1 -0
- pipelinemd-0.1.0/tests/fixtures/secrets.py +77 -0
- pipelinemd-0.1.0/tests/fixtures/traces/command_not_found.log +26 -0
- pipelinemd-0.1.0/tests/fixtures/traces/docker_daemon_unreachable.log +27 -0
- pipelinemd-0.1.0/tests/fixtures/traces/git_submodule_auth.log +28 -0
- pipelinemd-0.1.0/tests/fixtures/traces/no_space_left.log +27 -0
- pipelinemd-0.1.0/tests/fixtures/traces/node_heap_oom.log +35 -0
- pipelinemd-0.1.0/tests/fixtures/traces/noisy_lint_failure.log +3039 -0
- pipelinemd-0.1.0/tests/fixtures/traces/npm_eresolve.log +77 -0
- pipelinemd-0.1.0/tests/fixtures/traces/npm_lockfile_out_of_sync.log +31 -0
- pipelinemd-0.1.0/tests/fixtures/traces/pytest_failures.log +43 -0
- pipelinemd-0.1.0/tests/test_ansi.py +56 -0
- pipelinemd-0.1.0/tests/test_assessment.py +254 -0
- pipelinemd-0.1.0/tests/test_citations.py +159 -0
- pipelinemd-0.1.0/tests/test_cli.py +774 -0
- pipelinemd-0.1.0/tests/test_config.py +78 -0
- pipelinemd-0.1.0/tests/test_corpus.py +238 -0
- pipelinemd-0.1.0/tests/test_cost.py +229 -0
- pipelinemd-0.1.0/tests/test_diagnose.py +315 -0
- pipelinemd-0.1.0/tests/test_distiller.py +89 -0
- pipelinemd-0.1.0/tests/test_engine.py +420 -0
- pipelinemd-0.1.0/tests/test_evaluate.py +426 -0
- pipelinemd-0.1.0/tests/test_extract.py +130 -0
- pipelinemd-0.1.0/tests/test_gitlab.py +248 -0
- pipelinemd-0.1.0/tests/test_models.py +233 -0
- pipelinemd-0.1.0/tests/test_redact.py +92 -0
- pipelinemd-0.1.0/tests/test_release.py +92 -0
- pipelinemd-0.1.0/tests/test_render.py +550 -0
- pipelinemd-0.1.0/tests/test_rules_catalog.py +54 -0
- pipelinemd-0.1.0/tests/test_taxonomy.py +289 -0
- pipelinemd-0.1.0/tests/test_trace.py +111 -0
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# The image build reads pyproject.toml, README.md, LICENSE and src/ and nothing
|
|
2
|
+
# else. Allow-listing them keeps the context small, keeps job traces in the
|
|
3
|
+
# corpus out of it, and means a stray local file can never end up in an image.
|
|
4
|
+
*
|
|
5
|
+
!pyproject.toml
|
|
6
|
+
!README.md
|
|
7
|
+
!LICENSE
|
|
8
|
+
!src/
|
|
9
|
+
|
|
10
|
+
# Even inside src/, never ship interpreter or tool droppings.
|
|
11
|
+
**/__pycache__
|
|
12
|
+
**/*.py[cod]
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
# Python
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*.egg-info/
|
|
5
|
+
.eggs/
|
|
6
|
+
build/
|
|
7
|
+
dist/
|
|
8
|
+
.venv/
|
|
9
|
+
venv/
|
|
10
|
+
|
|
11
|
+
# Tooling
|
|
12
|
+
.pytest_cache/
|
|
13
|
+
.mypy_cache/
|
|
14
|
+
.ruff_cache/
|
|
15
|
+
.coverage
|
|
16
|
+
coverage.xml
|
|
17
|
+
htmlcov/
|
|
18
|
+
|
|
19
|
+
# Editors / OS
|
|
20
|
+
.idea/
|
|
21
|
+
.vscode/
|
|
22
|
+
.DS_Store
|
|
23
|
+
|
|
24
|
+
# Local scratch - never commit real job traces, they can carry secrets
|
|
25
|
+
*.local.log
|
|
26
|
+
scratch/
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to pipelinemd are recorded here.
|
|
4
|
+
|
|
5
|
+
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and the project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
How a section becomes a release is in [docs/releasing.md](docs/releasing.md).
|
|
8
|
+
|
|
9
|
+
## [Unreleased]
|
|
10
|
+
|
|
11
|
+
## [0.1.0] - 2026-10-06
|
|
12
|
+
|
|
13
|
+
The first release.
|
|
14
|
+
|
|
15
|
+
### Added
|
|
16
|
+
|
|
17
|
+
- `pipelinemd diagnose` takes a GitLab job or pipeline URL, or `--project`
|
|
18
|
+
with `--pipeline`/`--job`, and with no target at all diagnoses the pipeline
|
|
19
|
+
it is running in. On a pipeline it picks the failed job, names the rest, and
|
|
20
|
+
`--all-jobs` diagnoses every one. Credentials come from `--token`,
|
|
21
|
+
`$PIPELINEMD_TOKEN`, `$GITLAB_TOKEN`, `$GITLAB_PRIVATE_TOKEN` or
|
|
22
|
+
`$CI_JOB_TOKEN`.
|
|
23
|
+
- `pipelinemd distill` distils a saved trace, or one on stdin, fully offline.
|
|
24
|
+
- A deterministic distiller: replays ANSI escapes and `\r`/`\b` overwrites,
|
|
25
|
+
parses GitLab sections, strips runner timestamps, scores every line against
|
|
26
|
+
failure signals and dampeners, and spends a fixed line budget on the windows
|
|
27
|
+
that explain the outcome, always keeping the runner's verdict at the tail.
|
|
28
|
+
- Redaction of credential-shaped text (GitLab, GitHub, AWS and Slack tokens,
|
|
29
|
+
JWTs, URL credentials, `Authorization:` headers, secret flags and
|
|
30
|
+
assignments, PEM private keys) before evidence reaches the prompt, the
|
|
31
|
+
terminal or the JSON output.
|
|
32
|
+
- A catalog of 59 known CI failure signatures, each with an explanation and
|
|
33
|
+
fixes, browsable with `pipelinemd rules` and `pipelinemd explain`.
|
|
34
|
+
- Every report is placed in one of seven v1 failure classes (`yaml`,
|
|
35
|
+
`ci_vars`, `image_pull`, `cache_artifact`, `test`, `runner`, `flaky`) with a
|
|
36
|
+
fix type. Flaky comes from GitLab's retry history: another attempt of the
|
|
37
|
+
same job passing on the same commit.
|
|
38
|
+
- An optional Claude diagnosis (the `[llm]` extra, `ANTHROPIC_API_KEY`) that
|
|
39
|
+
reads only the distilled evidence and must cite evidence lines; a diagnosis
|
|
40
|
+
citing nothing real is rejected, and invented line numbers are shown beside
|
|
41
|
+
one that cites a mix. A failed call is never fatal.
|
|
42
|
+
- An analysis-level confidence capped at its weakest signal, with a **needs
|
|
43
|
+
human review** flag at low, and an estimated cost from the API's own token
|
|
44
|
+
counts, totalled across a run with `--all-jobs`.
|
|
45
|
+
- Terminal, Markdown and JSON output. JSON is versioned by `schema_version`.
|
|
46
|
+
- A labelled corpus of 62 failure traces across the v1 taxonomy, and
|
|
47
|
+
`pipelinemd eval` (`make eval`, `make gate`) scoring rule@1, class accuracy,
|
|
48
|
+
evidence hit rate and exit-code accuracy, with thresholds that fail CI on a
|
|
49
|
+
regression.
|
|
50
|
+
- A Docker image, `ghcr.io/rbalukja15/pipelinemd`, for amd64 and arm64, so a
|
|
51
|
+
GitLab CI job can diagnose a failed pipeline without a pip install, and a
|
|
52
|
+
release workflow that publishes it and the PyPI package as one version.
|
|
53
|
+
|
|
54
|
+
[Unreleased]: https://github.com/rbalukja15/pipelinemd/compare/v0.1.0...HEAD
|
|
55
|
+
[0.1.0]: https://github.com/rbalukja15/pipelinemd/releases/tag/v0.1.0
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# The pipelinemd image: the CLI with the Claude diagnosis layer, ready to name
|
|
2
|
+
# in a CI job's `image:` so a failed pipeline is diagnosed without a pip install
|
|
3
|
+
# - which is exactly when PyPI being slow or unreachable would hurt most.
|
|
4
|
+
#
|
|
5
|
+
# Two stages. The builder turns the source tree into wheels for pipelinemd and
|
|
6
|
+
# its [llm] dependencies; the runtime stage installs from those wheels alone,
|
|
7
|
+
# so the published image carries no build backend, no source tree and no pip
|
|
8
|
+
# cache.
|
|
9
|
+
|
|
10
|
+
ARG PYTHON_VERSION=3.12
|
|
11
|
+
|
|
12
|
+
FROM python:${PYTHON_VERSION}-slim AS builder
|
|
13
|
+
|
|
14
|
+
ENV PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
|
15
|
+
PIP_NO_CACHE_DIR=1
|
|
16
|
+
|
|
17
|
+
WORKDIR /src
|
|
18
|
+
|
|
19
|
+
# Only what the build backend reads. Copying the whole context would make every
|
|
20
|
+
# edit to a test or a doc invalidate this layer.
|
|
21
|
+
COPY pyproject.toml README.md LICENSE ./
|
|
22
|
+
COPY src ./src
|
|
23
|
+
|
|
24
|
+
RUN pip wheel --wheel-dir /wheels ".[llm]"
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
FROM python:${PYTHON_VERSION}-slim
|
|
28
|
+
|
|
29
|
+
# Set from the release tag by the release workflow and from __version__ by
|
|
30
|
+
# `make image`; a bare `docker build` says "dev".
|
|
31
|
+
ARG VERSION=dev
|
|
32
|
+
|
|
33
|
+
LABEL org.opencontainers.image.title="pipelinemd" \
|
|
34
|
+
org.opencontainers.image.description="GitLab CI/CD failure doctor - diagnoses failed pipelines and suggests fixes." \
|
|
35
|
+
org.opencontainers.image.source="https://github.com/rbalukja15/pipelinemd" \
|
|
36
|
+
org.opencontainers.image.licenses="MIT" \
|
|
37
|
+
org.opencontainers.image.version="${VERSION}"
|
|
38
|
+
|
|
39
|
+
ENV PYTHONDONTWRITEBYTECODE=1 \
|
|
40
|
+
PYTHONUNBUFFERED=1 \
|
|
41
|
+
PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
|
42
|
+
PIP_NO_CACHE_DIR=1
|
|
43
|
+
|
|
44
|
+
# The wheels are bind-mounted rather than copied, so they never become a layer
|
|
45
|
+
# of their own. --no-index keeps pip off the network entirely: what gets
|
|
46
|
+
# installed is exactly what the builder resolved.
|
|
47
|
+
RUN --mount=type=bind,from=builder,source=/wheels,target=/wheels \
|
|
48
|
+
pip install --no-index --find-links /wheels --root-user-action=ignore "pipelinemd[llm]"
|
|
49
|
+
|
|
50
|
+
# A fixed numeric uid, not just a name: Kubernetes runners with runAsNonRoot
|
|
51
|
+
# can only verify a number. GitLab's Docker executor makes the build directory
|
|
52
|
+
# writable for whichever user the image declares.
|
|
53
|
+
RUN useradd --create-home --uid 10001 --user-group pipelinemd
|
|
54
|
+
USER 10001:10001
|
|
55
|
+
WORKDIR /home/pipelinemd
|
|
56
|
+
|
|
57
|
+
# GitLab runs job scripts through a shell, so a job using this image overrides
|
|
58
|
+
# the entrypoint with `entrypoint: [""]`; the slim base keeps /bin/sh for that.
|
|
59
|
+
# Everywhere else the image is the CLI itself: `docker run <image> distill -`.
|
|
60
|
+
ENTRYPOINT ["pipelinemd"]
|
|
61
|
+
CMD ["--help"]
|
|
62
|
+
|
|
63
|
+
# `docker stop` (and compose and Kubernetes) send the image's stop signal.
|
|
64
|
+
# Python running as PID 1 ignores SIGTERM but turns SIGINT into
|
|
65
|
+
# KeyboardInterrupt, which the CLI already answers with "interrupted" and exit
|
|
66
|
+
# 130 - so a stop takes a fraction of a second instead of waiting 10 s for
|
|
67
|
+
# SIGKILL.
|
|
68
|
+
STOPSIGNAL SIGINT
|
pipelinemd-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 rbalukja15
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
# Developer entry points. Everything here runs offline.
|
|
2
|
+
#
|
|
3
|
+
# Tools are invoked through the interpreter rather than by bare name: a
|
|
4
|
+
# standalone `pytest` or `mypy` on PATH may belong to a different environment
|
|
5
|
+
# than the one pipelinemd is installed into, and then the suite fails to import
|
|
6
|
+
# the package it is meant to be testing. CI calls these same targets, so that
|
|
7
|
+
# fix applies there too and the two cannot drift.
|
|
8
|
+
PYTHON ?= python3
|
|
9
|
+
PYTEST_ARGS ?=
|
|
10
|
+
|
|
11
|
+
# The regression floor for `make gate`. rule@1 is 55/55 today, so 0.92 is 51/55
|
|
12
|
+
# - the gate trips on the fifth regression. That headroom exists for one
|
|
13
|
+
# reason: the corpus is meant to grow with observed traces, which will be
|
|
14
|
+
# harder than the authored ones, and a gate that goes red the moment someone
|
|
15
|
+
# commits a real failing log discourages the contribution this project most
|
|
16
|
+
# needs. Single-case regressions stay visible either way, since `make eval`
|
|
17
|
+
# names every miss.
|
|
18
|
+
#
|
|
19
|
+
# Adding hard traces will eventually push the rate under this floor, and the
|
|
20
|
+
# right response is to lower it in a commit that says why. That is the point:
|
|
21
|
+
# it makes the number move in review rather than silently.
|
|
22
|
+
MIN_RULE_ACCURACY ?= 0.92
|
|
23
|
+
|
|
24
|
+
# The class floor. Class is the input to the fix type, and yaml_patch is what
|
|
25
|
+
# #24's MR generator acts on, so a regression here is the one with the sharpest
|
|
26
|
+
# downside - it should fail a build, not just move a printed number. Class is
|
|
27
|
+
# 51/62 today; 0.78 is 49/62, so the gate trips on the third regression. Same
|
|
28
|
+
# rule as above: lowering it in a commit that says why is the intended response
|
|
29
|
+
# to adding hard traces, not a failure.
|
|
30
|
+
MIN_CLASS_ACCURACY ?= 0.78
|
|
31
|
+
|
|
32
|
+
# Known gaps are excluded from every rate, so a rule that starts firing on one
|
|
33
|
+
# moves no number. Zero tolerance here: the catalog growing a confident wrong
|
|
34
|
+
# answer where it used to stay silent is a regression worth failing over.
|
|
35
|
+
MAX_GAP_FALSE_POSITIVES ?= 0
|
|
36
|
+
|
|
37
|
+
# The image `make image` builds and `make image-smoke` runs. PACKAGE_VERSION
|
|
38
|
+
# is read from the same line hatch stamps the wheel with, so the smoke test
|
|
39
|
+
# checks the image reports the version the package claims. Not plain VERSION:
|
|
40
|
+
# that name is common enough in a CI environment to be picked up by accident.
|
|
41
|
+
IMAGE ?= pipelinemd:dev
|
|
42
|
+
PACKAGE_VERSION ?= $(shell sed -n 's/^__version__ = "\(.*\)"$$/\1/p' src/pipelinemd/__init__.py)
|
|
43
|
+
SMOKE_TRACE ?= corpus/traces/runner-oom-killed.log
|
|
44
|
+
|
|
45
|
+
.PHONY: help install lint format typecheck test eval gate check dist image image-smoke \
|
|
46
|
+
release-notes clean
|
|
47
|
+
|
|
48
|
+
help:
|
|
49
|
+
@echo "install editable install with dev extras"
|
|
50
|
+
@echo "lint ruff check + ruff format --check"
|
|
51
|
+
@echo "format ruff format (rewrites files)"
|
|
52
|
+
@echo "typecheck mypy --strict"
|
|
53
|
+
@echo "test pytest"
|
|
54
|
+
@echo "eval score the deterministic pipeline against corpus/"
|
|
55
|
+
@echo "gate eval, failing below rule@1 $(MIN_RULE_ACCURACY) or class $(MIN_CLASS_ACCURACY)"
|
|
56
|
+
@echo "check lint + typecheck + test + gate, CI's test job"
|
|
57
|
+
@echo "dist sdist + wheel into dist/, then twine check (needs build, twine)"
|
|
58
|
+
@echo "image build the Docker image as $(IMAGE) (needs docker)"
|
|
59
|
+
@echo "image-smoke run $(IMAGE) the ways CI and GitLab will (needs docker)"
|
|
60
|
+
@echo "release-notes check \$$TAG against the version and changelog, print its notes"
|
|
61
|
+
|
|
62
|
+
install:
|
|
63
|
+
$(PYTHON) -m pip install -e ".[dev]"
|
|
64
|
+
|
|
65
|
+
lint:
|
|
66
|
+
$(PYTHON) -m ruff check src tests scripts
|
|
67
|
+
$(PYTHON) -m ruff format --check src tests scripts
|
|
68
|
+
|
|
69
|
+
format:
|
|
70
|
+
$(PYTHON) -m ruff format src tests scripts
|
|
71
|
+
|
|
72
|
+
typecheck:
|
|
73
|
+
$(PYTHON) -m mypy
|
|
74
|
+
|
|
75
|
+
test:
|
|
76
|
+
$(PYTHON) -m pytest $(PYTEST_ARGS)
|
|
77
|
+
|
|
78
|
+
eval:
|
|
79
|
+
$(PYTHON) -m pipelinemd eval
|
|
80
|
+
|
|
81
|
+
gate:
|
|
82
|
+
$(PYTHON) -m pipelinemd eval \
|
|
83
|
+
--min-rule-accuracy $(MIN_RULE_ACCURACY) \
|
|
84
|
+
--min-class-accuracy $(MIN_CLASS_ACCURACY) \
|
|
85
|
+
--max-gap-false-positives $(MAX_GAP_FALSE_POSITIVES)
|
|
86
|
+
|
|
87
|
+
check: lint typecheck test gate
|
|
88
|
+
|
|
89
|
+
# --strict turns twine's warnings into failures: PyPI will not let a version be
|
|
90
|
+
# uploaded twice, so a broken long description is cheaper to catch here.
|
|
91
|
+
dist:
|
|
92
|
+
rm -rf dist
|
|
93
|
+
$(PYTHON) -m build
|
|
94
|
+
$(PYTHON) -m twine check --strict dist/*
|
|
95
|
+
|
|
96
|
+
# Not part of `check`: that has to run anywhere Python does, and these need a
|
|
97
|
+
# Docker daemon. CI runs them in a job of their own. buildx with --load works
|
|
98
|
+
# with both the default builder and a docker-container one, and leaves the
|
|
99
|
+
# result where `docker run` can find it.
|
|
100
|
+
image:
|
|
101
|
+
docker buildx build --load --build-arg VERSION=$(PACKAGE_VERSION) -t $(IMAGE) .
|
|
102
|
+
|
|
103
|
+
# Each line is a promise the README makes about the image. The output is
|
|
104
|
+
# captured before it is searched, so a container that fails after printing
|
|
105
|
+
# the right text still fails the target.
|
|
106
|
+
image-smoke:
|
|
107
|
+
@echo "--version reports $(PACKAGE_VERSION)"
|
|
108
|
+
@out=$$(docker run --rm $(IMAGE) --version) && echo "$$out" && \
|
|
109
|
+
test "$$out" = "pipelinemd $(PACKAGE_VERSION)"
|
|
110
|
+
@echo "the default command runs"
|
|
111
|
+
docker run --rm $(IMAGE) > /dev/null
|
|
112
|
+
@echo "distill - reads a trace on stdin, offline, and finds its rule and evidence"
|
|
113
|
+
@out=$$(docker run --rm -i --network none $(IMAGE) distill --color never - < $(SMOKE_TRACE)) && \
|
|
114
|
+
echo "$$out" | grep -F "runner.oom-killed" && echo "$$out" | grep -E "^Evidence +[0-9]+ of"
|
|
115
|
+
@echo "the [llm] extra is installed"
|
|
116
|
+
docker run --rm --entrypoint python $(IMAGE) -c "import anthropic; print(anthropic.__version__)"
|
|
117
|
+
@echo "it does not run as root"
|
|
118
|
+
@uid=$$(docker run --rm --entrypoint id $(IMAGE) -u) && echo "uid $$uid" && test "$$uid" != 0
|
|
119
|
+
@echo "a shell script runs with the entrypoint cleared, as GitLab runs one"
|
|
120
|
+
docker run --rm --entrypoint "" $(IMAGE) sh -c 'pipelinemd --version'
|
|
121
|
+
|
|
122
|
+
# The tag comes from the environment, never the command line, and is only ever
|
|
123
|
+
# expanded by the shell inside quotes: a tag name is text someone else chose,
|
|
124
|
+
# and make would paste it into the recipe unquoted.
|
|
125
|
+
release-notes:
|
|
126
|
+
@$(PYTHON) scripts/release_notes.py "$$TAG"
|
|
127
|
+
|
|
128
|
+
clean:
|
|
129
|
+
rm -rf .pytest_cache .mypy_cache .ruff_cache .coverage htmlcov dist build
|
|
130
|
+
find . -name __pycache__ -type d -prune -exec rm -rf {} +
|
|
@@ -0,0 +1,337 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: pipelinemd
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: GitLab CI/CD failure doctor - diagnoses failed pipelines and suggests fixes.
|
|
5
|
+
Project-URL: Homepage, https://github.com/rbalukja15/pipelinemd
|
|
6
|
+
Project-URL: Source, https://github.com/rbalukja15/pipelinemd
|
|
7
|
+
Project-URL: Issues, https://github.com/rbalukja15/pipelinemd/issues
|
|
8
|
+
Author: rbalukja15
|
|
9
|
+
License: MIT License
|
|
10
|
+
|
|
11
|
+
Copyright (c) 2026 rbalukja15
|
|
12
|
+
|
|
13
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
14
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
15
|
+
in the Software without restriction, including without limitation the rights
|
|
16
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
17
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
18
|
+
furnished to do so, subject to the following conditions:
|
|
19
|
+
|
|
20
|
+
The above copyright notice and this permission notice shall be included in all
|
|
21
|
+
copies or substantial portions of the Software.
|
|
22
|
+
|
|
23
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
24
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
25
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
26
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
27
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
28
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
29
|
+
SOFTWARE.
|
|
30
|
+
License-File: LICENSE
|
|
31
|
+
Keywords: cd,ci,claude,diagnostics,gitlab,logs,pipeline
|
|
32
|
+
Classifier: Development Status :: 3 - Alpha
|
|
33
|
+
Classifier: Environment :: Console
|
|
34
|
+
Classifier: Intended Audience :: Developers
|
|
35
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
36
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
37
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
38
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
39
|
+
Classifier: Topic :: Software Development :: Build Tools
|
|
40
|
+
Classifier: Topic :: System :: Logging
|
|
41
|
+
Requires-Python: >=3.11
|
|
42
|
+
Provides-Extra: dev
|
|
43
|
+
Requires-Dist: mypy>=1.11; extra == 'dev'
|
|
44
|
+
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
|
|
45
|
+
Requires-Dist: pytest>=8.0; extra == 'dev'
|
|
46
|
+
Requires-Dist: ruff>=0.6; extra == 'dev'
|
|
47
|
+
Provides-Extra: llm
|
|
48
|
+
Requires-Dist: anthropic>=0.40; extra == 'llm'
|
|
49
|
+
Description-Content-Type: text/markdown
|
|
50
|
+
|
|
51
|
+
# pipelinemd
|
|
52
|
+
|
|
53
|
+
[](https://github.com/rbalukja15/pipelinemd/actions/workflows/ci.yml)
|
|
54
|
+
[](https://github.com/rbalukja15/pipelinemd/blob/main/LICENSE)
|
|
55
|
+
|
|
56
|
+
**GitLab CI/CD failure doctor** — takes a failed pipeline, works out what
|
|
57
|
+
actually broke, and tells you how to fix it.
|
|
58
|
+
|
|
59
|
+
A failed job's log is a terminal recording, not a report. It can run to tens of
|
|
60
|
+
thousands of lines, most of them progress bars that rewrote themselves five
|
|
61
|
+
hundred times, and the one line that matters is somewhere in the middle.
|
|
62
|
+
pipelinemd does two things about that:
|
|
63
|
+
|
|
64
|
+
1. **A deterministic distiller** replays the trace the way a terminal would,
|
|
65
|
+
strips the noise, scores every line for failure-likeness, and keeps only the
|
|
66
|
+
regions that explain the outcome — then matches them against a catalog of
|
|
67
|
+
**59 known CI failure signatures**, each with a real fix.
|
|
68
|
+
Every report is placed in one of seven **v1 failure classes** — `yaml`,
|
|
69
|
+
`ci_vars`, `image_pull`, `cache_artifact`, `test`, `runner`, `flaky` — with
|
|
70
|
+
the kind of fix it wants. Flaky comes from GitLab's retry history, not from
|
|
71
|
+
a guess: if another attempt of the same job passed on the same commit, it
|
|
72
|
+
says so.
|
|
73
|
+
2. **An optional Claude diagnosis** reads only that distilled evidence and
|
|
74
|
+
names the root cause, separating the actual fault from its fallout — and
|
|
75
|
+
must cite the evidence lines it relied on. A diagnosis that cites nothing
|
|
76
|
+
real is rejected rather than reported.
|
|
77
|
+
|
|
78
|
+
Every report says how far to trust it and what it cost. The **confidence** is
|
|
79
|
+
never more than the weakest signal behind it; at low, the report says **needs
|
|
80
|
+
human review** and why. The **cost** is $0 for rules alone, and an estimate
|
|
81
|
+
from the API's own token counts when Claude was asked.
|
|
82
|
+
|
|
83
|
+
The first half needs no API key, no model, and **no third-party packages at
|
|
84
|
+
all**. The second is the upgrade.
|
|
85
|
+
|
|
86
|
+
```
|
|
87
|
+
$ pipelinemd diagnose https://gitlab.com/acme/web/-/jobs/98765
|
|
88
|
+
|
|
89
|
+
pipelinemd build (test) #98765
|
|
90
|
+
acme/web · ref main · exit code 1 · 47s
|
|
91
|
+
ERROR: Job failed: exit code 1
|
|
92
|
+
test → code_patch · high · from npm.lockfile-out-of-sync
|
|
93
|
+
confidence high
|
|
94
|
+
est. cost $0.0519 · claude-opus-5 · 5,210 in / 1,034 out tokens
|
|
95
|
+
|
|
96
|
+
Diagnosis high confidence · dependency
|
|
97
|
+
|
|
98
|
+
npm ci refused to install because package-lock.json no longer matches package.json.
|
|
99
|
+
|
|
100
|
+
Line 41203 shows npm rejecting the install outright rather than resolving it.
|
|
101
|
+
The lockfile still pins react@17 while package.json now asks for ^18 (line
|
|
102
|
+
41211), which is the change that broke the pair.
|
|
103
|
+
|
|
104
|
+
Suggested fixes
|
|
105
|
+
1. Regenerate the lockfile and commit it
|
|
106
|
+
|
|
107
|
+
npm install
|
|
108
|
+
git add package-lock.json && git commit -m 'Regenerate lockfile'
|
|
109
|
+
|
|
110
|
+
Rule matches
|
|
111
|
+
● npm.lockfile-out-of-sync package-lock.json is out of sync with package.json L41203
|
|
112
|
+
|
|
113
|
+
Evidence 9 of 41,284 lines · 100.0% reduced
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
---
|
|
117
|
+
|
|
118
|
+
## Install
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
pip install pipelinemd # core: distiller + rules, zero dependencies
|
|
122
|
+
pip install 'pipelinemd[llm]' # adds the Claude diagnosis layer
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Requires Python 3.11+.
|
|
126
|
+
|
|
127
|
+
Or skip Python altogether. The image has the `[llm]` extra installed, runs as
|
|
128
|
+
an unprivileged user, and is built for amd64 and arm64:
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
docker run --rm ghcr.io/rbalukja15/pipelinemd:0.1 --version
|
|
132
|
+
docker run --rm -i ghcr.io/rbalukja15/pipelinemd:0.1 distill - < build.log
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
The image runs as uid 10001, so to save a report redirect stdout rather than
|
|
136
|
+
passing `-o` into a mounted directory, which that user may not be allowed to
|
|
137
|
+
write to:
|
|
138
|
+
|
|
139
|
+
```bash
|
|
140
|
+
docker run --rm -i ghcr.io/rbalukja15/pipelinemd:0.1 distill - < build.log > report.txt
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
On rootful Linux Docker, `--user "$(id -u):$(id -g)"` also lets `-o` write into
|
|
144
|
+
a mount; rootless Docker and Podman map that uid elsewhere, so redirect there.
|
|
145
|
+
|
|
146
|
+
Tags follow the package: `0.1.0` exactly, `0.1` for the latest patch release,
|
|
147
|
+
and `latest`. The image and the PyPI package are published by one workflow
|
|
148
|
+
from one tag, so a version means the same thing in both
|
|
149
|
+
([docs/releasing.md](https://github.com/rbalukja15/pipelinemd/blob/main/docs/releasing.md)).
|
|
150
|
+
|
|
151
|
+
## Use
|
|
152
|
+
|
|
153
|
+
### Diagnose a failed job or pipeline
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
pipelinemd diagnose https://gitlab.com/acme/web/-/jobs/98765
|
|
157
|
+
pipelinemd diagnose https://gitlab.com/acme/web/-/pipelines/12345 # picks the failed job
|
|
158
|
+
pipelinemd diagnose --project acme/web --pipeline 12345
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
Give it a pipeline and it finds the failed jobs for you; with more than one it
|
|
162
|
+
diagnoses the first and names the rest (`--all-jobs` does them all).
|
|
163
|
+
|
|
164
|
+
Credentials come from `--token`, `$PIPELINEMD_TOKEN`, `$GITLAB_TOKEN`,
|
|
165
|
+
`$GITLAB_PRIVATE_TOKEN` or `$CI_JOB_TOKEN` — in that order, with the right
|
|
166
|
+
header for each. Public projects need no token at all.
|
|
167
|
+
|
|
168
|
+
### Distil a log you already have — fully offline
|
|
169
|
+
|
|
170
|
+
```bash
|
|
171
|
+
pipelinemd distill build.log
|
|
172
|
+
kubectl logs job/ci-run | pipelinemd distill -
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
No network, no model, no key. Useful on its own for shrinking a log before
|
|
176
|
+
pasting it anywhere.
|
|
177
|
+
|
|
178
|
+
### Score it against the corpus
|
|
179
|
+
|
|
180
|
+
```bash
|
|
181
|
+
make eval
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
Runs the distiller and rule engine over 62 labelled traces in [`corpus/`](https://github.com/rbalukja15/pipelinemd/blob/main/corpus/README.md)
|
|
185
|
+
and reports rule@1, evidence hit rate and exit-code accuracy per failure class.
|
|
186
|
+
Offline and deterministic. Results and their caveats: [docs/evaluation.md](https://github.com/rbalukja15/pipelinemd/blob/main/docs/evaluation.md).
|
|
187
|
+
|
|
188
|
+
`make gate` is the same run with CI's thresholds, and runs on every push — a
|
|
189
|
+
catalog regression fails the build rather than being noticed later.
|
|
190
|
+
|
|
191
|
+
### Browse the rule catalog
|
|
192
|
+
|
|
193
|
+
```bash
|
|
194
|
+
pipelinemd rules # all 59, grouped by category
|
|
195
|
+
pipelinemd rules --category dependency
|
|
196
|
+
pipelinemd rules --search docker
|
|
197
|
+
pipelinemd explain npm.eresolve # one rule in full, patterns included
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
### Inside GitLab CI
|
|
201
|
+
|
|
202
|
+
Run with no target at all and it diagnoses the pipeline it is running in:
|
|
203
|
+
|
|
204
|
+
```yaml
|
|
205
|
+
diagnose:
|
|
206
|
+
stage: .post
|
|
207
|
+
image:
|
|
208
|
+
name: ghcr.io/rbalukja15/pipelinemd:0.1
|
|
209
|
+
entrypoint: [""]
|
|
210
|
+
when: on_failure
|
|
211
|
+
script:
|
|
212
|
+
- pipelinemd diagnose --format markdown -o diagnosis.md
|
|
213
|
+
artifacts:
|
|
214
|
+
when: always
|
|
215
|
+
paths: [diagnosis.md]
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
Nothing is installed when the job runs, so a failure is diagnosed even while
|
|
219
|
+
PyPI is slow or unreachable. The `entrypoint: [""]` matters: the image's
|
|
220
|
+
entrypoint is the CLI, and GitLab needs a shell to run `script:`.
|
|
221
|
+
|
|
222
|
+
`CI_JOB_TOKEN` and `CI_PIPELINE_ID` are picked up automatically. Set
|
|
223
|
+
`ANTHROPIC_API_KEY` as a masked CI variable to enable the diagnosis layer.
|
|
224
|
+
|
|
225
|
+
Any image with Python 3.11+ works too, at the cost of an install on every
|
|
226
|
+
failure:
|
|
227
|
+
|
|
228
|
+
```yaml
|
|
229
|
+
diagnose:
|
|
230
|
+
stage: .post
|
|
231
|
+
image: python:3.12-slim
|
|
232
|
+
when: on_failure
|
|
233
|
+
script:
|
|
234
|
+
- pip install 'pipelinemd[llm]'
|
|
235
|
+
- pipelinemd diagnose --format markdown -o diagnosis.md
|
|
236
|
+
artifacts:
|
|
237
|
+
when: always
|
|
238
|
+
paths: [diagnosis.md]
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
A ready-made job is in [`examples/gitlab-ci-diagnose.yml`](https://github.com/rbalukja15/pipelinemd/blob/main/examples/gitlab-ci-diagnose.yml).
|
|
242
|
+
|
|
243
|
+
## Output formats
|
|
244
|
+
|
|
245
|
+
| `--format` | For |
|
|
246
|
+
| --- | --- |
|
|
247
|
+
| `terminal` (default) | Reading it yourself. Colour honours `NO_COLOR` and non-TTY output. |
|
|
248
|
+
| `markdown` | Pasting into a merge request or issue. Collapsible evidence. |
|
|
249
|
+
| `json` | Other tooling. Versioned via `schema_version`. One job is one report object; several (`--all-jobs`) are `{"reports": [...], "cost": {...}}`, with the run's total cost beside them. |
|
|
250
|
+
|
|
251
|
+
## How the distiller works
|
|
252
|
+
|
|
253
|
+
Each stage is pure, independently testable, and does one thing:
|
|
254
|
+
|
|
255
|
+
| Stage | What it removes or adds |
|
|
256
|
+
| --- | --- |
|
|
257
|
+
| **ANSI** | Colour codes, cursor moves, erase sequences, OSC hyperlinks. |
|
|
258
|
+
| **Overwrites** | Replays `\r` and `\b` positionally — a progress bar that rewrote one line 500 times collapses to its final frame. |
|
|
259
|
+
| **Sections** | Parses `section_start`/`section_end` markers into structured spans with durations, and attributes every line to the section it ran in. |
|
|
260
|
+
| **Timestamps** | Strips the runner's optional per-line RFC3339 prefix. |
|
|
261
|
+
| **Redaction** | Masks credential-shaped substrings *before* anything leaves the process. |
|
|
262
|
+
| **Scoring** | Weights every line against 27 failure signals and 5 dampeners, so `0 failed, 12 passed` does not outrank the real fault. |
|
|
263
|
+
| **Windowing** | Grows context around strong signals, merges overlapping windows, spends a fixed line budget on the best, and always keeps the tail — the runner writes its verdict there. |
|
|
264
|
+
| **Collapsing** | Folds runs of near-identical lines into one entry with a repeat count. |
|
|
265
|
+
|
|
266
|
+
Same trace in, same evidence out — which is what makes it cheap to cache and
|
|
267
|
+
safe to assert on in tests.
|
|
268
|
+
|
|
269
|
+
## Redaction
|
|
270
|
+
|
|
271
|
+
pipelinemd can send evidence to an LLM, and you will paste its output into
|
|
272
|
+
merge requests. GitLab masks *known* CI variables; anything a build tool prints
|
|
273
|
+
itself does not get that protection. So before evidence reaches the prompt, the
|
|
274
|
+
terminal, or the JSON output, these shapes are masked:
|
|
275
|
+
|
|
276
|
+
GitLab tokens (`glpat-`, `glrt-`, …) · GitHub tokens · AWS access key ids ·
|
|
277
|
+
Slack tokens · JWTs · credentials embedded in URLs · `Authorization:` headers ·
|
|
278
|
+
`--password`/`--token` style flags · `KEY=value` where the key names a secret ·
|
|
279
|
+
PEM private key blocks.
|
|
280
|
+
|
|
281
|
+
Values that only look secret-shaped (`SECRET=false`, `***`) are left alone.
|
|
282
|
+
This reduces exposure; it is not a guarantee — treat traces from untrusted
|
|
283
|
+
pipelines accordingly.
|
|
284
|
+
|
|
285
|
+
## Exit codes
|
|
286
|
+
|
|
287
|
+
| Code | Meaning |
|
|
288
|
+
| --- | --- |
|
|
289
|
+
| `0` | A report was produced. |
|
|
290
|
+
| `2` | Usage error — bad URL, missing argument, unknown rule. |
|
|
291
|
+
| `3` | GitLab error — unreachable, unauthorised, not found. |
|
|
292
|
+
| `4` | Nothing to diagnose — the pipeline has no failed jobs. |
|
|
293
|
+
|
|
294
|
+
A failed Claude call is **not** fatal: pipelinemd warns on stderr and reports
|
|
295
|
+
the deterministic findings anyway.
|
|
296
|
+
|
|
297
|
+
For how the pieces fit together, see [docs/architecture.md](https://github.com/rbalukja15/pipelinemd/blob/main/docs/architecture.md);
|
|
298
|
+
for how a version reaches PyPI and the image, [docs/releasing.md](https://github.com/rbalukja15/pipelinemd/blob/main/docs/releasing.md).
|
|
299
|
+
|
|
300
|
+
## Design notes
|
|
301
|
+
|
|
302
|
+
- **The core has no dependencies.** Not "few" — none. It installs into any
|
|
303
|
+
runner image without dragging a tree behind it, which matters for a tool
|
|
304
|
+
whose whole job is to run in someone else's broken build.
|
|
305
|
+
- **Rules first, model second.** Every rule fires without a network call. The
|
|
306
|
+
model is asked to do only what rules cannot: decide which of several signals
|
|
307
|
+
is the cause and which is the consequence.
|
|
308
|
+
- **The model never sees a raw trace.** It sees distilled, redacted evidence
|
|
309
|
+
plus what the rules already concluded — which keeps requests small and cheap,
|
|
310
|
+
and keeps the model's effort on the judgement call.
|
|
311
|
+
- **A diagnosis must point at something.** Every diagnosis cites line numbers,
|
|
312
|
+
and every cited number is resolved against the excerpt the model was shown.
|
|
313
|
+
Cite nothing real and the diagnosis is rejected; cite a mix and the invented
|
|
314
|
+
numbers are printed alongside it. Confident prose is easy; a claim you can
|
|
315
|
+
check is the product.
|
|
316
|
+
- **Confidence is earned, not asserted.** An analysis is only as confident as
|
|
317
|
+
its weakest part — the rule, the model's own rating — and loses a level for
|
|
318
|
+
each sign of trouble. It stays high/medium/low: a decimal would claim a
|
|
319
|
+
calibration nothing here has, and `make eval` reports how often each level
|
|
320
|
+
is right instead.
|
|
321
|
+
- **The distiller is pure.** No clock, no network, no randomness.
|
|
322
|
+
|
|
323
|
+
## Contributing a rule
|
|
324
|
+
|
|
325
|
+
Rules live in `src/pipelinemd/rules/catalog.py`. A good one is narrow: anchor
|
|
326
|
+
`patterns` to text the tool actually prints, write `explanation` as *why this
|
|
327
|
+
happens*, and make each entry of `fixes` something someone can do. Add a
|
|
328
|
+
fixture under `tests/fixtures/traces/` and assert the rule fires on it.
|
|
329
|
+
|
|
330
|
+
Then give it a v1 class in `RULE_CLASS` in `src/pipelinemd/taxonomy.py` — the
|
|
331
|
+
suite fails until you do. Choose from what the failure *is*, and only use
|
|
332
|
+
`flaky` if it is transient whatever the environment; anything that is merely
|
|
333
|
+
often transient is flaky only when retry history says so.
|
|
334
|
+
|
|
335
|
+
## License
|
|
336
|
+
|
|
337
|
+
MIT — see [LICENSE](https://github.com/rbalukja15/pipelinemd/blob/main/LICENSE).
|