copilot-guardian 0.2.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/ci.yml +53 -0
- package/.test-output-run-abstain/guardian.report.json +8 -0
- package/CHANGELOG.md +602 -0
- package/CONTRIBUTING.md +28 -0
- package/LICENSE +21 -0
- package/README.md +205 -0
- package/SECURITY.md +150 -0
- package/dist/cli.js +384 -0
- package/dist/cli.js.map +1 -0
- package/dist/engine/analyze.js +294 -0
- package/dist/engine/analyze.js.map +1 -0
- package/dist/engine/async-exec.js +314 -0
- package/dist/engine/async-exec.js.map +1 -0
- package/dist/engine/auto-apply.js +424 -0
- package/dist/engine/auto-apply.js.map +1 -0
- package/dist/engine/context-enhancer.js +141 -0
- package/dist/engine/context-enhancer.js.map +1 -0
- package/dist/engine/debug.js +77 -0
- package/dist/engine/debug.js.map +1 -0
- package/dist/engine/eval.js +437 -0
- package/dist/engine/eval.js.map +1 -0
- package/dist/engine/github.js +191 -0
- package/dist/engine/github.js.map +1 -0
- package/dist/engine/mcp.js +217 -0
- package/dist/engine/mcp.js.map +1 -0
- package/dist/engine/patch_options.js +474 -0
- package/dist/engine/patch_options.js.map +1 -0
- package/dist/engine/run.js +124 -0
- package/dist/engine/run.js.map +1 -0
- package/dist/engine/util.js +167 -0
- package/dist/engine/util.js.map +1 -0
- package/dist/ui/dashboard.js +81 -0
- package/dist/ui/dashboard.js.map +1 -0
- package/docs/ARCHITECTURE.md +292 -0
- package/docs/Logo.png +0 -0
- package/docs/screenshots/05-hypothesis-dashboard.png +0 -0
- package/docs/screenshots/07-patch-spectrum.png +0 -0
- package/docs/screenshots/final-demo.gif +0 -0
- package/examples/demo-failure/.github/workflows/ci.yml +23 -0
- package/examples/demo-failure/README.md +93 -0
- package/examples/demo-failure/package.json +9 -0
- package/examples/demo-failure/test/require-api-url.js +10 -0
- package/jest.config.cjs +35 -0
- package/package.json +39 -0
- package/prompts/analysis.v2.txt +62 -0
- package/prompts/debug.followup.v1.txt +18 -0
- package/prompts/patch.options.v1.txt +47 -0
- package/prompts/patch.simple.v1.txt +12 -0
- package/prompts/quality.v1.txt +25 -0
- package/schemas/analysis.schema.json +65 -0
- package/schemas/patch_options.schema.json +23 -0
- package/schemas/quality.schema.json +12 -0
- package/src/cli.ts +417 -0
- package/src/engine/analyze.ts +412 -0
- package/src/engine/async-exec.ts +384 -0
- package/src/engine/auto-apply.ts +516 -0
- package/src/engine/context-enhancer.ts +176 -0
- package/src/engine/debug.ts +91 -0
- package/src/engine/eval.ts +546 -0
- package/src/engine/github.ts +223 -0
- package/src/engine/mcp.ts +267 -0
- package/src/engine/patch_options.ts +604 -0
- package/src/engine/run.ts +154 -0
- package/src/engine/util.ts +195 -0
- package/src/ui/dashboard.ts +90 -0
- package/test-sdk.mjs +51 -0
- package/tests/auto_heal_branch_safety.test.ts +76 -0
- package/tests/github_redaction_failclosed.test.ts +24 -0
- package/tests/mocks/copilot-sdk.mock.ts +15 -0
- package/tests/quality_guard_regression_matrix.test.ts +432 -0
- package/tests/run_abstain_policy.test.ts +83 -0
- package/tsconfig.json +17 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Flamehaven
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,205 @@
|
|
|
1
|
+
<div align="center">
|
|
2
|
+
|
|
3
|
+
<img src="docs/Logo.png" alt="Copilot Guardian Logo" width="400"/>
|
|
4
|
+
|
|
5
|
+
# Copilot Guardian
|
|
6
|
+
|
|
7
|
+
Deterministic safety layer for Copilot-driven CI healing.
|
|
8
|
+
|
|
9
|
+
[](https://opensource.org/licenses/MIT)
|
|
10
|
+
[](https://github.com/flamehaven01/copilot-guardian/actions/workflows/ci.yml)
|
|
11
|
+
[](https://github.com/flamehaven01/copilot-guardian/releases)
|
|
12
|
+
[](https://github.com/flamehaven01/copilot-guardian/releases/tag/v0.2.5)
|
|
13
|
+
[](https://dev.to/challenges/github-2026-01-21)
|
|
14
|
+
[](https://www.typescriptlang.org/)
|
|
15
|
+
[](https://modelcontextprotocol.io/)
|
|
16
|
+
|
|
17
|
+
[Why Challenge](#why-this-is-a-copilot-cli-challenge-submission) • [Judge Quick Test](#judge-quick-test-90-seconds) • [Final GIF](#final-gif-slot-submission-finalization) • [Quick Start](#quick-start) • [How It Works](#how-it-works) • [Docs](#documentation-links)
|
|
18
|
+
|
|
19
|
+
</div>
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Why This Is a Copilot CLI Challenge Submission
|
|
24
|
+
|
|
25
|
+
This project demonstrates five advanced Copilot usage patterns under real CI failures:
|
|
26
|
+
|
|
27
|
+
1. Multi-hypothesis reasoning with explicit confidence and evidence
|
|
28
|
+
2. Patch synthesis across conservative, balanced, and aggressive strategies
|
|
29
|
+
3. Deterministic fail-closed guardrails against slop and bypass patterns
|
|
30
|
+
4. MCP-enriched context to improve diagnosis quality
|
|
31
|
+
5. Transparent artifact trail (`analysis.json`, raw responses, patch index)
|
|
32
|
+
|
|
33
|
+
Runtime clarification:
|
|
34
|
+
- Production path uses `@github/copilot-sdk`
|
|
35
|
+
- CLI fallback is for local experimentation only
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## Judge Quick Test (90 seconds)
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
copilot-guardian run \
|
|
43
|
+
--repo flamehaven01/copilot-guardian \
|
|
44
|
+
--last-failed \
|
|
45
|
+
--show-options \
|
|
46
|
+
--fast \
|
|
47
|
+
--max-log-chars 20000
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Expected:
|
|
51
|
+
1. Structured diagnosis in `analysis.json`
|
|
52
|
+
2. Patch spectrum in `patch_options.json`
|
|
53
|
+
3. Safety verdicts in `quality_review.*.json`
|
|
54
|
+
|
|
55
|
+
For extended trace mode (slower), add `--show-reasoning`.
|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
## Final GIF Slot (Submission Finalization)
|
|
60
|
+
|
|
61
|
+
Final demo artifact:
|
|
62
|
+
|
|
63
|
+

|
|
64
|
+
|
|
65
|
+
Runtime: 3m43s, Profile: --fast --max-log-chars 20000 (reasoning hidden for stable demo)
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## Forced Abstain Policy (NOT PATCHABLE)
|
|
70
|
+
|
|
71
|
+
Guardian intentionally abstains for non-patchable failure classes such as:
|
|
72
|
+
- `401/403` auth failures
|
|
73
|
+
- token permission errors
|
|
74
|
+
- API rate-limit or infra-unavailable patterns
|
|
75
|
+
|
|
76
|
+
When abstaining, `abstain.report.json` is emitted and patch generation is skipped.
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## Copilot Challenge Showcase: Five Advanced Usage Patterns
|
|
81
|
+
|
|
82
|
+
1. Multi-turn structured reasoning
|
|
83
|
+
2. Schema-constrained JSON outputs
|
|
84
|
+
3. Risk-calibrated generation
|
|
85
|
+
4. Independent validation loop
|
|
86
|
+
5. Fail-closed enforcement
|
|
87
|
+
|
|
88
|
+
Why this matters: AI slop in CI can produce green-looking but unsafe results.
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
## Quick Start
|
|
93
|
+
|
|
94
|
+
### Prerequisites
|
|
95
|
+
|
|
96
|
+
- Node.js >=18
|
|
97
|
+
- GitHub CLI (`gh`) authenticated
|
|
98
|
+
- GitHub Copilot subscription (SDK access)
|
|
99
|
+
|
|
100
|
+
### Installation
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
npm install -g copilot-guardian
|
|
104
|
+
# or
|
|
105
|
+
npx copilot-guardian --help
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
### Core Commands
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
# Stable demo profile
|
|
112
|
+
copilot-guardian run \
|
|
113
|
+
--repo owner/repo \
|
|
114
|
+
--last-failed \
|
|
115
|
+
--show-options \
|
|
116
|
+
--fast \
|
|
117
|
+
--max-log-chars 20000
|
|
118
|
+
|
|
119
|
+
# Analysis only
|
|
120
|
+
copilot-guardian analyze \
|
|
121
|
+
--repo owner/repo \
|
|
122
|
+
--run-id <run_id> \
|
|
123
|
+
--fast \
|
|
124
|
+
--max-log-chars 20000
|
|
125
|
+
|
|
126
|
+
# Evaluate multiple failed runs
|
|
127
|
+
copilot-guardian eval \
|
|
128
|
+
--repo owner/repo \
|
|
129
|
+
--failed-limit 5 \
|
|
130
|
+
--fast \
|
|
131
|
+
--max-log-chars 50000
|
|
132
|
+
|
|
133
|
+
# Interactive follow-up
|
|
134
|
+
copilot-guardian debug \
|
|
135
|
+
--repo owner/repo \
|
|
136
|
+
--last-failed
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## How It Works
|
|
142
|
+
|
|
143
|
+
Full architecture: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
|
|
144
|
+
|
|
145
|
+
```mermaid
|
|
146
|
+
graph TB
|
|
147
|
+
A[GitHub Actions Failure] --> B[Guardian CLI]
|
|
148
|
+
B --> C[Context Fetch]
|
|
149
|
+
C --> D[Multi-Hypothesis Analysis]
|
|
150
|
+
D --> E[Copilot SDK]
|
|
151
|
+
E --> F[Patch Strategies]
|
|
152
|
+
F --> G[Deterministic Quality Guard]
|
|
153
|
+
G --> H{GO?}
|
|
154
|
+
H -->|NO_GO| I[Reject and Re-diagnose]
|
|
155
|
+
H -->|GO| J[Patch Candidate]
|
|
156
|
+
J --> K[Safe Branch PR or Auto-Heal]
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
### Key Modules
|
|
160
|
+
|
|
161
|
+
| Layer | Module | Purpose |
|
|
162
|
+
|---|---|---|
|
|
163
|
+
| Detection | `src/engine/github.ts` | Collect failure context |
|
|
164
|
+
| Intelligence | `src/engine/analyze.ts` | Multi-hypothesis diagnosis |
|
|
165
|
+
| Decision | `src/engine/patch_options.ts` | Strategy generation |
|
|
166
|
+
| Validation | Deterministic + model review | Slop and bypass control |
|
|
167
|
+
| Action | `src/engine/auto-apply.ts` | Safe branch/PR workflow |
|
|
168
|
+
|
|
169
|
+
---
|
|
170
|
+
|
|
171
|
+
## Output Files
|
|
172
|
+
|
|
173
|
+
Artifacts are generated under `.copilot-guardian/`:
|
|
174
|
+
|
|
175
|
+
| File | Purpose |
|
|
176
|
+
|---|---|
|
|
177
|
+
| `analysis.json` | Diagnosis + selected hypothesis |
|
|
178
|
+
| `reasoning_trace.json` | Hypothesis trace |
|
|
179
|
+
| `patch_options.json` | Strategy index + verdicts |
|
|
180
|
+
| `fix.*.patch` | Patch files |
|
|
181
|
+
| `quality_review.*.json` | Per-strategy quality results |
|
|
182
|
+
| `abstain.report.json` | Forced abstain classification |
|
|
183
|
+
| `copilot.*.raw.txt` | Raw model output snapshots |
|
|
184
|
+
|
|
185
|
+
---
|
|
186
|
+
|
|
187
|
+
## Documentation Links
|
|
188
|
+
|
|
189
|
+
- Architecture: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
|
|
190
|
+
- Demo walkthrough: [examples/demo-failure/README.md](examples/demo-failure/README.md)
|
|
191
|
+
- Changelog: [CHANGELOG.md](CHANGELOG.md)
|
|
192
|
+
- Security: [SECURITY.md](SECURITY.md)
|
|
193
|
+
- Contributing: [CONTRIBUTING.md](CONTRIBUTING.md)
|
|
194
|
+
|
|
195
|
+
## License
|
|
196
|
+
|
|
197
|
+
MIT License. See [LICENSE](LICENSE).
|
|
198
|
+
|
|
199
|
+
## Credits
|
|
200
|
+
|
|
201
|
+
Built by Flamehaven (Yun) for the [GitHub Copilot CLI Challenge](https://dev.to/challenges/github-2026-01-21).
|
|
202
|
+
|
|
203
|
+
---
|
|
204
|
+
|
|
205
|
+
Trust is built on receipts.
|
package/SECURITY.md
ADDED
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
# Security Policy
|
|
2
|
+
|
|
3
|
+
## Supported Versions
|
|
4
|
+
|
|
5
|
+
| Version | Supported |
|
|
6
|
+
| ------- | ------------------ |
|
|
7
|
+
| 0.2.x | :white_check_mark: |
|
|
8
|
+
|
|
9
|
+
## Security Philosophy
|
|
10
|
+
|
|
11
|
+
Copilot Guardian is designed with security-first principles:
|
|
12
|
+
|
|
13
|
+
### 1. Secret Redaction
|
|
14
|
+
All logs and context sent to AI models are automatically sanitized:
|
|
15
|
+
- GitHub tokens (`ghp_*`, `ghs_*`, `gho_*`)
|
|
16
|
+
- Bearer tokens
|
|
17
|
+
- API keys
|
|
18
|
+
- Passwords in error messages
|
|
19
|
+
|
|
20
|
+
### 2. Local-First Processing
|
|
21
|
+
- All analysis happens locally via GitHub Copilot CLI
|
|
22
|
+
- No data is sent to external servers (except GitHub's Copilot API via authenticated CLI)
|
|
23
|
+
- Full audit trail maintained in `.copilot-guardian/` directory
|
|
24
|
+
|
|
25
|
+
### 3. Transparency
|
|
26
|
+
- All raw inputs and outputs are saved (`.raw.txt` files)
|
|
27
|
+
- Users can inspect exactly what data was sent to AI models
|
|
28
|
+
- No hidden telemetry or data collection
|
|
29
|
+
|
|
30
|
+
## Reporting a Vulnerability
|
|
31
|
+
|
|
32
|
+
If you discover a security vulnerability in Copilot Guardian, please:
|
|
33
|
+
|
|
34
|
+
1. **DO NOT** open a public issue
|
|
35
|
+
2. Email: info@flamehaven.space
|
|
36
|
+
3. Include:
|
|
37
|
+
- Description of the vulnerability
|
|
38
|
+
- Steps to reproduce
|
|
39
|
+
- Potential impact
|
|
40
|
+
- Suggested fix (if any)
|
|
41
|
+
|
|
42
|
+
### Response Timeline
|
|
43
|
+
- **24 hours**: Initial acknowledgment
|
|
44
|
+
- **7 days**: Detailed assessment and action plan
|
|
45
|
+
- **30 days**: Fix implementation and disclosure
|
|
46
|
+
|
|
47
|
+
## Security Best Practices for Users
|
|
48
|
+
|
|
49
|
+
### Authentication
|
|
50
|
+
```bash
|
|
51
|
+
# Ensure GitHub CLI is authenticated
|
|
52
|
+
gh auth status
|
|
53
|
+
|
|
54
|
+
# Use token with minimal required scopes
|
|
55
|
+
# Required: repo, workflow
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
### Environment Variables
|
|
59
|
+
```bash
|
|
60
|
+
# Never commit .env files
|
|
61
|
+
# Use GitHub Secrets for CI/CD environments
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
### Patch Review
|
|
65
|
+
```bash
|
|
66
|
+
# Always review patches before applying
|
|
67
|
+
copilot-guardian fix --interactive
|
|
68
|
+
|
|
69
|
+
# Use Conservative mode for production
|
|
70
|
+
# Review the "Quality Verdict" before accepting
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
## Known Limitations
|
|
74
|
+
|
|
75
|
+
### 1. GitHub CLI Security
|
|
76
|
+
- Guardian inherits gh CLI's authentication model
|
|
77
|
+
- Ensure `gh` is up to date: `gh version`
|
|
78
|
+
|
|
79
|
+
### 2. AI Model Limitations
|
|
80
|
+
- LLMs can hallucinate - always review patches
|
|
81
|
+
- Use the Anti-Slop quality checks
|
|
82
|
+
- Test patches in non-production environments first
|
|
83
|
+
|
|
84
|
+
### 3. Rate Limits
|
|
85
|
+
- GitHub API rate limits apply
|
|
86
|
+
- Copilot API rate limits apply
|
|
87
|
+
- Guardian implements exponential backoff
|
|
88
|
+
|
|
89
|
+
## Security Audit Trail
|
|
90
|
+
|
|
91
|
+
Every Guardian run creates:
|
|
92
|
+
```
|
|
93
|
+
.copilot-guardian/
|
|
94
|
+
├── [timestamp]-context.raw.txt # What was sent to AI
|
|
95
|
+
├── [timestamp]-analysis.json # AI response (structured)
|
|
96
|
+
└── [timestamp]-patches.json # Generated patches
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
This enables:
|
|
100
|
+
- Post-incident forensics
|
|
101
|
+
- Compliance audits
|
|
102
|
+
- Privacy verification
|
|
103
|
+
|
|
104
|
+
## Data Privacy
|
|
105
|
+
|
|
106
|
+
### What Guardian Collects
|
|
107
|
+
- GitHub Actions logs (for analysis)
|
|
108
|
+
- Repository metadata (via gh CLI)
|
|
109
|
+
- Source code context (when using MCP)
|
|
110
|
+
|
|
111
|
+
### What Guardian DOES NOT Collect
|
|
112
|
+
- User credentials
|
|
113
|
+
- Unrelated source code
|
|
114
|
+
- Personal information
|
|
115
|
+
- Telemetry or usage statistics
|
|
116
|
+
|
|
117
|
+
### Data Retention
|
|
118
|
+
- All data is stored locally in `.copilot-guardian/`
|
|
119
|
+
- User controls retention (can delete directory)
|
|
120
|
+
- No cloud storage or external databases
|
|
121
|
+
|
|
122
|
+
## Compliance
|
|
123
|
+
|
|
124
|
+
### GDPR
|
|
125
|
+
- All processing is local
|
|
126
|
+
- No data transfer to third parties (except GitHub Copilot API via user's authenticated session)
|
|
127
|
+
- User has full control and right to erasure
|
|
128
|
+
|
|
129
|
+
### Enterprise Use
|
|
130
|
+
- Compatible with GitHub Enterprise
|
|
131
|
+
- Works within corporate firewalls
|
|
132
|
+
- No external dependencies beyond GitHub APIs
|
|
133
|
+
|
|
134
|
+
## Security Updates
|
|
135
|
+
|
|
136
|
+
Subscribe to security advisories:
|
|
137
|
+
```bash
|
|
138
|
+
gh repo subscribe flamehaven01/copilot-guardian --alerts
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
## Contact
|
|
142
|
+
|
|
143
|
+
- Security Issues: info@flamehaven.space
|
|
144
|
+
- General Issues: [GitHub Issues](https://github.com/flamehaven01/copilot-guardian/issues)
|
|
145
|
+
- Documentation: [docs/](./docs/)
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
**Last Updated**: 2026-02-12
|
|
150
|
+
**Security Policy Version**: 0.2.5
|