copilot-guardian 0.2.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. package/.github/workflows/ci.yml +53 -0
  2. package/.test-output-run-abstain/guardian.report.json +8 -0
  3. package/CHANGELOG.md +602 -0
  4. package/CONTRIBUTING.md +28 -0
  5. package/LICENSE +21 -0
  6. package/README.md +205 -0
  7. package/SECURITY.md +150 -0
  8. package/dist/cli.js +384 -0
  9. package/dist/cli.js.map +1 -0
  10. package/dist/engine/analyze.js +294 -0
  11. package/dist/engine/analyze.js.map +1 -0
  12. package/dist/engine/async-exec.js +314 -0
  13. package/dist/engine/async-exec.js.map +1 -0
  14. package/dist/engine/auto-apply.js +424 -0
  15. package/dist/engine/auto-apply.js.map +1 -0
  16. package/dist/engine/context-enhancer.js +141 -0
  17. package/dist/engine/context-enhancer.js.map +1 -0
  18. package/dist/engine/debug.js +77 -0
  19. package/dist/engine/debug.js.map +1 -0
  20. package/dist/engine/eval.js +437 -0
  21. package/dist/engine/eval.js.map +1 -0
  22. package/dist/engine/github.js +191 -0
  23. package/dist/engine/github.js.map +1 -0
  24. package/dist/engine/mcp.js +217 -0
  25. package/dist/engine/mcp.js.map +1 -0
  26. package/dist/engine/patch_options.js +474 -0
  27. package/dist/engine/patch_options.js.map +1 -0
  28. package/dist/engine/run.js +124 -0
  29. package/dist/engine/run.js.map +1 -0
  30. package/dist/engine/util.js +167 -0
  31. package/dist/engine/util.js.map +1 -0
  32. package/dist/ui/dashboard.js +81 -0
  33. package/dist/ui/dashboard.js.map +1 -0
  34. package/docs/ARCHITECTURE.md +292 -0
  35. package/docs/Logo.png +0 -0
  36. package/docs/screenshots/05-hypothesis-dashboard.png +0 -0
  37. package/docs/screenshots/07-patch-spectrum.png +0 -0
  38. package/docs/screenshots/final-demo.gif +0 -0
  39. package/examples/demo-failure/.github/workflows/ci.yml +23 -0
  40. package/examples/demo-failure/README.md +93 -0
  41. package/examples/demo-failure/package.json +9 -0
  42. package/examples/demo-failure/test/require-api-url.js +10 -0
  43. package/jest.config.cjs +35 -0
  44. package/package.json +39 -0
  45. package/prompts/analysis.v2.txt +62 -0
  46. package/prompts/debug.followup.v1.txt +18 -0
  47. package/prompts/patch.options.v1.txt +47 -0
  48. package/prompts/patch.simple.v1.txt +12 -0
  49. package/prompts/quality.v1.txt +25 -0
  50. package/schemas/analysis.schema.json +65 -0
  51. package/schemas/patch_options.schema.json +23 -0
  52. package/schemas/quality.schema.json +12 -0
  53. package/src/cli.ts +417 -0
  54. package/src/engine/analyze.ts +412 -0
  55. package/src/engine/async-exec.ts +384 -0
  56. package/src/engine/auto-apply.ts +516 -0
  57. package/src/engine/context-enhancer.ts +176 -0
  58. package/src/engine/debug.ts +91 -0
  59. package/src/engine/eval.ts +546 -0
  60. package/src/engine/github.ts +223 -0
  61. package/src/engine/mcp.ts +267 -0
  62. package/src/engine/patch_options.ts +604 -0
  63. package/src/engine/run.ts +154 -0
  64. package/src/engine/util.ts +195 -0
  65. package/src/ui/dashboard.ts +90 -0
  66. package/test-sdk.mjs +51 -0
  67. package/tests/auto_heal_branch_safety.test.ts +76 -0
  68. package/tests/github_redaction_failclosed.test.ts +24 -0
  69. package/tests/mocks/copilot-sdk.mock.ts +15 -0
  70. package/tests/quality_guard_regression_matrix.test.ts +432 -0
  71. package/tests/run_abstain_policy.test.ts +83 -0
  72. package/tsconfig.json +17 -0
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Flamehaven
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,205 @@
1
+ <div align="center">
2
+
3
+ <img src="docs/Logo.png" alt="Copilot Guardian Logo" width="400"/>
4
+
5
+ # Copilot Guardian
6
+
7
+ Deterministic safety layer for Copilot-driven CI healing.
8
+
9
+ [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg?style=flat-square)](https://opensource.org/licenses/MIT)
10
+ [![CI](https://github.com/flamehaven01/copilot-guardian/actions/workflows/ci.yml/badge.svg)](https://github.com/flamehaven01/copilot-guardian/actions/workflows/ci.yml)
11
+ [![Version: 0.2.5](https://img.shields.io/badge/version-0.2.5-blue.svg?style=flat-square)](https://github.com/flamehaven01/copilot-guardian/releases)
12
+ [![Release: v0.2.5](https://img.shields.io/badge/release-v0.2.5-0A66C2.svg?style=flat-square)](https://github.com/flamehaven01/copilot-guardian/releases/tag/v0.2.5)
13
+ [![Copilot CLI Challenge](https://img.shields.io/badge/GitHub-Copilot_Challenge-181717.svg?style=flat-square&logo=github&logoColor=white)](https://dev.to/challenges/github-2026-01-21)
14
+ [![TypeScript](https://img.shields.io/badge/TypeScript-5.3-3178C6.svg?style=flat-square&logo=typescript&logoColor=white)](https://www.typescriptlang.org/)
15
+ [![MCP](https://img.shields.io/badge/MCP-Enabled-FF5722.svg?style=flat-square)](https://modelcontextprotocol.io/)
16
+
17
+ [Why Challenge](#why-this-is-a-copilot-cli-challenge-submission) • [Judge Quick Test](#judge-quick-test-90-seconds) • [Final GIF](#final-gif-slot-submission-finalization) • [Quick Start](#quick-start) • [How It Works](#how-it-works) • [Docs](#documentation-links)
18
+
19
+ </div>
20
+
21
+ ---
22
+
23
+ ## Why This Is a Copilot CLI Challenge Submission
24
+
25
+ This project demonstrates five advanced Copilot usage patterns under real CI failures:
26
+
27
+ 1. Multi-hypothesis reasoning with explicit confidence and evidence
28
+ 2. Patch synthesis across conservative, balanced, and aggressive strategies
29
+ 3. Deterministic fail-closed guardrails against slop and bypass patterns
30
+ 4. MCP-enriched context to improve diagnosis quality
31
+ 5. Transparent artifact trail (`analysis.json`, raw responses, patch index)
32
+
33
+ Runtime clarification:
34
+ - Production path uses `@github/copilot-sdk`
35
+ - CLI fallback is for local experimentation only
36
+
37
+ ---
38
+
39
+ ## Judge Quick Test (90 seconds)
40
+
41
+ ```bash
42
+ copilot-guardian run \
43
+ --repo flamehaven01/copilot-guardian \
44
+ --last-failed \
45
+ --show-options \
46
+ --fast \
47
+ --max-log-chars 20000
48
+ ```
49
+
50
+ Expected:
51
+ 1. Structured diagnosis in `analysis.json`
52
+ 2. Patch spectrum in `patch_options.json`
53
+ 3. Safety verdicts in `quality_review.*.json`
54
+
55
+ For extended trace mode (slower), add `--show-reasoning`.
56
+
57
+ ---
58
+
59
+ ## Final GIF Slot (Submission Finalization)
60
+
61
+ Final demo artifact:
62
+
63
+ ![Judge Quick Test Demo](docs/screenshots/final-demo.gif)
64
+
65
+ Runtime: 3m43s, Profile: --fast --max-log-chars 20000 (reasoning hidden for stable demo)
66
+
67
+ ---
68
+
69
+ ## Forced Abstain Policy (NOT PATCHABLE)
70
+
71
+ Guardian intentionally abstains for non-patchable failure classes such as:
72
+ - `401/403` auth failures
73
+ - token permission errors
74
+ - API rate-limit or infra-unavailable patterns
75
+
76
+ When abstaining, `abstain.report.json` is emitted and patch generation is skipped.
77
+
78
+ ---
79
+
80
+ ## Copilot Challenge Showcase: Five Advanced Usage Patterns
81
+
82
+ 1. Multi-turn structured reasoning
83
+ 2. Schema-constrained JSON outputs
84
+ 3. Risk-calibrated generation
85
+ 4. Independent validation loop
86
+ 5. Fail-closed enforcement
87
+
88
+ Why this matters: AI slop in CI can produce green-looking but unsafe results.
89
+
90
+ ---
91
+
92
+ ## Quick Start
93
+
94
+ ### Prerequisites
95
+
96
+ - Node.js >=18
97
+ - GitHub CLI (`gh`) authenticated
98
+ - GitHub Copilot subscription (SDK access)
99
+
100
+ ### Installation
101
+
102
+ ```bash
103
+ npm install -g copilot-guardian
104
+ # or
105
+ npx copilot-guardian --help
106
+ ```
107
+
108
+ ### Core Commands
109
+
110
+ ```bash
111
+ # Stable demo profile
112
+ copilot-guardian run \
113
+ --repo owner/repo \
114
+ --last-failed \
115
+ --show-options \
116
+ --fast \
117
+ --max-log-chars 20000
118
+
119
+ # Analysis only
120
+ copilot-guardian analyze \
121
+ --repo owner/repo \
122
+ --run-id <run_id> \
123
+ --fast \
124
+ --max-log-chars 20000
125
+
126
+ # Evaluate multiple failed runs
127
+ copilot-guardian eval \
128
+ --repo owner/repo \
129
+ --failed-limit 5 \
130
+ --fast \
131
+ --max-log-chars 50000
132
+
133
+ # Interactive follow-up
134
+ copilot-guardian debug \
135
+ --repo owner/repo \
136
+ --last-failed
137
+ ```
138
+
139
+ ---
140
+
141
+ ## How It Works
142
+
143
+ Full architecture: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
144
+
145
+ ```mermaid
146
+ graph TB
147
+ A[GitHub Actions Failure] --> B[Guardian CLI]
148
+ B --> C[Context Fetch]
149
+ C --> D[Multi-Hypothesis Analysis]
150
+ D --> E[Copilot SDK]
151
+ E --> F[Patch Strategies]
152
+ F --> G[Deterministic Quality Guard]
153
+ G --> H{GO?}
154
+ H -->|NO_GO| I[Reject and Re-diagnose]
155
+ H -->|GO| J[Patch Candidate]
156
+ J --> K[Safe Branch PR or Auto-Heal]
157
+ ```
158
+
159
+ ### Key Modules
160
+
161
+ | Layer | Module | Purpose |
162
+ |---|---|---|
163
+ | Detection | `src/engine/github.ts` | Collect failure context |
164
+ | Intelligence | `src/engine/analyze.ts` | Multi-hypothesis diagnosis |
165
+ | Decision | `src/engine/patch_options.ts` | Strategy generation |
166
+ | Validation | Deterministic + model review | Slop and bypass control |
167
+ | Action | `src/engine/auto-apply.ts` | Safe branch/PR workflow |
168
+
169
+ ---
170
+
171
+ ## Output Files
172
+
173
+ Artifacts are generated under `.copilot-guardian/`:
174
+
175
+ | File | Purpose |
176
+ |---|---|
177
+ | `analysis.json` | Diagnosis + selected hypothesis |
178
+ | `reasoning_trace.json` | Hypothesis trace |
179
+ | `patch_options.json` | Strategy index + verdicts |
180
+ | `fix.*.patch` | Patch files |
181
+ | `quality_review.*.json` | Per-strategy quality results |
182
+ | `abstain.report.json` | Forced abstain classification |
183
+ | `copilot.*.raw.txt` | Raw model output snapshots |
184
+
185
+ ---
186
+
187
+ ## Documentation Links
188
+
189
+ - Architecture: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
190
+ - Demo walkthrough: [examples/demo-failure/README.md](examples/demo-failure/README.md)
191
+ - Changelog: [CHANGELOG.md](CHANGELOG.md)
192
+ - Security: [SECURITY.md](SECURITY.md)
193
+ - Contributing: [CONTRIBUTING.md](CONTRIBUTING.md)
194
+
195
+ ## License
196
+
197
+ MIT License. See [LICENSE](LICENSE).
198
+
199
+ ## Credits
200
+
201
+ Built by Flamehaven (Yun) for the [GitHub Copilot CLI Challenge](https://dev.to/challenges/github-2026-01-21).
202
+
203
+ ---
204
+
205
+ Trust is built on receipts.
package/SECURITY.md ADDED
@@ -0,0 +1,150 @@
1
+ # Security Policy
2
+
3
+ ## Supported Versions
4
+
5
+ | Version | Supported |
6
+ | ------- | ------------------ |
7
+ | 0.2.x | :white_check_mark: |
8
+
9
+ ## Security Philosophy
10
+
11
+ Copilot Guardian is designed with security-first principles:
12
+
13
+ ### 1. Secret Redaction
14
+ All logs and context sent to AI models are automatically sanitized:
15
+ - GitHub tokens (`ghp_*`, `ghs_*`, `gho_*`)
16
+ - Bearer tokens
17
+ - API keys
18
+ - Passwords in error messages
19
+
20
+ ### 2. Local-First Processing
21
+ - All analysis happens locally via GitHub Copilot CLI
22
+ - No data is sent to external servers (except GitHub's Copilot API via authenticated CLI)
23
+ - Full audit trail maintained in `.copilot-guardian/` directory
24
+
25
+ ### 3. Transparency
26
+ - All raw inputs and outputs are saved (`.raw.txt` files)
27
+ - Users can inspect exactly what data was sent to AI models
28
+ - No hidden telemetry or data collection
29
+
30
+ ## Reporting a Vulnerability
31
+
32
+ If you discover a security vulnerability in Copilot Guardian, please:
33
+
34
+ 1. **DO NOT** open a public issue
35
+ 2. Email: info@flamehaven.space
36
+ 3. Include:
37
+ - Description of the vulnerability
38
+ - Steps to reproduce
39
+ - Potential impact
40
+ - Suggested fix (if any)
41
+
42
+ ### Response Timeline
43
+ - **24 hours**: Initial acknowledgment
44
+ - **7 days**: Detailed assessment and action plan
45
+ - **30 days**: Fix implementation and disclosure
46
+
47
+ ## Security Best Practices for Users
48
+
49
+ ### Authentication
50
+ ```bash
51
+ # Ensure GitHub CLI is authenticated
52
+ gh auth status
53
+
54
+ # Use token with minimal required scopes
55
+ # Required: repo, workflow
56
+ ```
57
+
58
+ ### Environment Variables
59
+ ```bash
60
+ # Never commit .env files
61
+ # Use GitHub Secrets for CI/CD environments
62
+ ```
63
+
64
+ ### Patch Review
65
+ ```bash
66
+ # Always review patches before applying
67
+ copilot-guardian fix --interactive
68
+
69
+ # Use Conservative mode for production
70
+ # Review the "Quality Verdict" before accepting
71
+ ```
72
+
73
+ ## Known Limitations
74
+
75
+ ### 1. GitHub CLI Security
76
+ - Guardian inherits gh CLI's authentication model
77
+ - Ensure `gh` is up to date: `gh version`
78
+
79
+ ### 2. AI Model Limitations
80
+ - LLMs can hallucinate - always review patches
81
+ - Use the Anti-Slop quality checks
82
+ - Test patches in non-production environments first
83
+
84
+ ### 3. Rate Limits
85
+ - GitHub API rate limits apply
86
+ - Copilot API rate limits apply
87
+ - Guardian implements exponential backoff
88
+
89
+ ## Security Audit Trail
90
+
91
+ Every Guardian run creates:
92
+ ```
93
+ .copilot-guardian/
94
+ ├── [timestamp]-context.raw.txt # What was sent to AI
95
+ ├── [timestamp]-analysis.json # AI response (structured)
96
+ └── [timestamp]-patches.json # Generated patches
97
+ ```
98
+
99
+ This enables:
100
+ - Post-incident forensics
101
+ - Compliance audits
102
+ - Privacy verification
103
+
104
+ ## Data Privacy
105
+
106
+ ### What Guardian Collects
107
+ - GitHub Actions logs (for analysis)
108
+ - Repository metadata (via gh CLI)
109
+ - Source code context (when using MCP)
110
+
111
+ ### What Guardian DOES NOT Collect
112
+ - User credentials
113
+ - Unrelated source code
114
+ - Personal information
115
+ - Telemetry or usage statistics
116
+
117
+ ### Data Retention
118
+ - All data is stored locally in `.copilot-guardian/`
119
+ - User controls retention (can delete directory)
120
+ - No cloud storage or external databases
121
+
122
+ ## Compliance
123
+
124
+ ### GDPR
125
+ - All processing is local
126
+ - No data transfer to third parties (except GitHub Copilot API via user's authenticated session)
127
+ - User has full control and right to erasure
128
+
129
+ ### Enterprise Use
130
+ - Compatible with GitHub Enterprise
131
+ - Works within corporate firewalls
132
+ - No external dependencies beyond GitHub APIs
133
+
134
+ ## Security Updates
135
+
136
+ Subscribe to security advisories:
137
+ ```bash
138
+ gh repo subscribe flamehaven01/copilot-guardian --alerts
139
+ ```
140
+
141
+ ## Contact
142
+
143
+ - Security Issues: info@flamehaven.space
144
+ - General Issues: [GitHub Issues](https://github.com/flamehaven01/copilot-guardian/issues)
145
+ - Documentation: [docs/](./docs/)
146
+
147
+ ---
148
+
149
+ **Last Updated**: 2026-02-12
150
+ **Security Policy Version**: 0.2.5