@canivel/ralph 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/ralph/PROMPT_build.md +126 -0
- package/.agents/ralph/agents.sh +15 -0
- package/.agents/ralph/config.sh +25 -0
- package/.agents/ralph/log-activity.sh +15 -0
- package/.agents/ralph/loop.sh +1001 -0
- package/.agents/ralph/references/CONTEXT_ENGINEERING.md +126 -0
- package/.agents/ralph/references/GUARDRAILS.md +174 -0
- package/AGENTS.md +20 -0
- package/README.md +266 -0
- package/bin/ralph +766 -0
- package/diagram.svg +55 -0
- package/examples/commands.md +46 -0
- package/package.json +39 -0
- package/ralph.webp +0 -0
- package/skills/commit/SKILL.md +219 -0
- package/skills/commit/references/commit_examples.md +292 -0
- package/skills/dev-browser/SKILL.md +211 -0
- package/skills/dev-browser/bun.lock +443 -0
- package/skills/dev-browser/package-lock.json +2988 -0
- package/skills/dev-browser/package.json +31 -0
- package/skills/dev-browser/references/scraping.md +155 -0
- package/skills/dev-browser/scripts/start-relay.ts +32 -0
- package/skills/dev-browser/scripts/start-server.ts +117 -0
- package/skills/dev-browser/server.sh +24 -0
- package/skills/dev-browser/src/client.ts +474 -0
- package/skills/dev-browser/src/index.ts +287 -0
- package/skills/dev-browser/src/relay.ts +731 -0
- package/skills/dev-browser/src/snapshot/__tests__/snapshot.test.ts +223 -0
- package/skills/dev-browser/src/snapshot/browser-script.ts +877 -0
- package/skills/dev-browser/src/snapshot/index.ts +14 -0
- package/skills/dev-browser/src/snapshot/inject.ts +13 -0
- package/skills/dev-browser/src/types.ts +34 -0
- package/skills/dev-browser/tsconfig.json +36 -0
- package/skills/dev-browser/vitest.config.ts +12 -0
- package/skills/prd/SKILL.md +235 -0
- package/tests/agent-loops.mjs +79 -0
- package/tests/agent-ping.mjs +39 -0
- package/tests/audit.md +56 -0
- package/tests/cli-smoke.mjs +47 -0
- package/tests/real-agents.mjs +127 -0
package/diagram.svg
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 520" role="img" aria-label="Ralph architecture diagram">
|
|
2
|
+
<style>
|
|
3
|
+
.bg { fill: #0f1218; }
|
|
4
|
+
.box { fill: #141a23; stroke: #e6e6e6; stroke-width: 1.5; }
|
|
5
|
+
.line { stroke: #e6e6e6; stroke-width: 1.5; }
|
|
6
|
+
.text { fill: #f3f3f3; font-family: Menlo, Consolas, Monaco, monospace; font-size: 16px; }
|
|
7
|
+
.small { font-size: 13px; opacity: 0.9; }
|
|
8
|
+
.muted { opacity: 0.7; }
|
|
9
|
+
</style>
|
|
10
|
+
|
|
11
|
+
<rect class="bg" x="0" y="0" width="1000" height="520" rx="0" />
|
|
12
|
+
|
|
13
|
+
<!-- Top boxes -->
|
|
14
|
+
<rect class="box" x="390" y="30" width="220" height="44" rx="0" />
|
|
15
|
+
<text class="text" x="500" y="58" text-anchor="middle">ralph CLI</text>
|
|
16
|
+
|
|
17
|
+
<rect class="box" x="240" y="114" width="520" height="48" rx="0" />
|
|
18
|
+
<text class="text small muted" x="500" y="144" text-anchor="middle">agent CLI: codex | claude | droid</text>
|
|
19
|
+
|
|
20
|
+
<!-- Arrow from top to agent (gap matches lower arrows) -->
|
|
21
|
+
<line class="line" x1="500" y1="74" x2="500" y2="96" />
|
|
22
|
+
<polygon points="496,96 500,104 504,96" fill="#e6e6e6" />
|
|
23
|
+
|
|
24
|
+
<!-- Branch lines (gap for heads) -->
|
|
25
|
+
<line class="line" x1="500" y1="162" x2="500" y2="200" />
|
|
26
|
+
<line class="line" x1="500" y1="200" x2="280" y2="200" />
|
|
27
|
+
<line class="line" x1="500" y1="200" x2="720" y2="200" />
|
|
28
|
+
<line class="line" x1="280" y1="200" x2="280" y2="214" />
|
|
29
|
+
<line class="line" x1="720" y1="200" x2="720" y2="214" />
|
|
30
|
+
<polygon points="276,214 280,222 284,214" fill="#e6e6e6" />
|
|
31
|
+
<polygon points="716,214 720,222 724,214" fill="#e6e6e6" />
|
|
32
|
+
|
|
33
|
+
<!-- Bottom boxes -->
|
|
34
|
+
<rect class="box" x="90" y="226" width="380" height="230" rx="0" />
|
|
35
|
+
<text class="text" x="110" y="256">.agents/ralph/</text>
|
|
36
|
+
<text class="text small muted" x="110" y="278">local overrides (optional)</text>
|
|
37
|
+
<text class="text small" x="110" y="306">loop.sh</text>
|
|
38
|
+
<text class="text small" x="110" y="328">PROMPT_build.md</text>
|
|
39
|
+
<text class="text small" x="110" y="350">references/</text>
|
|
40
|
+
<text class="text small" x="110" y="372">log-activity.sh</text>
|
|
41
|
+
<text class="text small" x="110" y="394">config.sh (optional)</text>
|
|
42
|
+
|
|
43
|
+
<rect class="box" x="530" y="226" width="380" height="230" rx="0" />
|
|
44
|
+
<text class="text" x="550" y="256">.ralph/</text>
|
|
45
|
+
<text class="text small muted" x="550" y="278">state + logs</text>
|
|
46
|
+
<text class="text small" x="550" y="306">errors.log</text>
|
|
47
|
+
<text class="text small" x="550" y="328">progress.md</text>
|
|
48
|
+
<text class="text small" x="550" y="350">guardrails.md</text>
|
|
49
|
+
<text class="text small" x="550" y="372">activity.log</text>
|
|
50
|
+
<text class="text small" x="550" y="394">runs/</text>
|
|
51
|
+
|
|
52
|
+
<text class="text small muted" x="500" y="500" text-anchor="middle">
|
|
53
|
+
Local templates override bundled defaults. State persists between runs.
|
|
54
|
+
</text>
|
|
55
|
+
</svg>
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Ralph CLI Examples
|
|
2
|
+
|
|
3
|
+
Basic usage:
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
ralph prd "A lightweight uptime monitor (Hono app), deployed on Cloudflare, with email alerts via AWS SES"
|
|
7
|
+
ralph build 1 # one Ralph run
|
|
8
|
+
ralph build 1 --no-commit # one Ralph run
|
|
9
|
+
ralph overview
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
Agent override:
|
|
13
|
+
|
|
14
|
+
```bash
|
|
15
|
+
ralph ping --agent=codex # check agent is installed + responsive
|
|
16
|
+
ralph build 1 --agent=codex # one Ralph run
|
|
17
|
+
ralph build 1 --agent=claude # one Ralph run
|
|
18
|
+
ralph build 1 --agent=droid # one Ralph run
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
PRD overrides:
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
ralph prd "..." --out .agents/tasks/prd-api.json
|
|
25
|
+
ralph build 1 --prd .agents/tasks/prd-api.json # one Ralph run
|
|
26
|
+
ralph overview --prd .agents/tasks/prd-api.json
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
Progress override:
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
ralph build 1 --progress .ralph/progress-api.md # one Ralph run
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Install templates:
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
ralph install
|
|
39
|
+
ralph install --force
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Install skills:
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
ralph install --skills
|
|
46
|
+
```
|
package/package.json
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@canivel/ralph",
|
|
3
|
+
"version": "0.2.0",
|
|
4
|
+
"description": "Minimal, file-based agent loop for autonomous coding. Fork with improved Claude support and first-run configuration.",
|
|
5
|
+
"bin": {
|
|
6
|
+
"ralph": "bin/ralph"
|
|
7
|
+
},
|
|
8
|
+
"scripts": {
|
|
9
|
+
"test": "node tests/cli-smoke.mjs && node tests/agent-loops.mjs",
|
|
10
|
+
"test:real": "node tests/real-agents.mjs",
|
|
11
|
+
"test:ping": "node tests/agent-ping.mjs"
|
|
12
|
+
},
|
|
13
|
+
"dependencies": {
|
|
14
|
+
"@clack/prompts": "^0.7.0"
|
|
15
|
+
},
|
|
16
|
+
"repository": {
|
|
17
|
+
"type": "git",
|
|
18
|
+
"url": "git+https://github.com/canivel/ralph.git"
|
|
19
|
+
},
|
|
20
|
+
"keywords": [
|
|
21
|
+
"ai",
|
|
22
|
+
"agent",
|
|
23
|
+
"coding",
|
|
24
|
+
"autonomous",
|
|
25
|
+
"claude",
|
|
26
|
+
"codex",
|
|
27
|
+
"prd",
|
|
28
|
+
"cli"
|
|
29
|
+
],
|
|
30
|
+
"author": "Danilo Canivel",
|
|
31
|
+
"bugs": {
|
|
32
|
+
"url": "https://github.com/canivel/ralph/issues"
|
|
33
|
+
},
|
|
34
|
+
"homepage": "https://github.com/canivel/ralph#readme",
|
|
35
|
+
"license": "MIT",
|
|
36
|
+
"engines": {
|
|
37
|
+
"node": ">=18.0.0"
|
|
38
|
+
}
|
|
39
|
+
}
|
package/ralph.webp
ADDED
|
Binary file
|
|
@@ -0,0 +1,219 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: commit
|
|
3
|
+
description: Write conventional commit messages with type, scope, and subject when the user wants to commit changes or save work.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Git Commit
|
|
7
|
+
|
|
8
|
+
Creates git commits following Conventional Commits format with proper type, scope, and subject.
|
|
9
|
+
|
|
10
|
+
## Quick Start
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
# 1. Stage changes
|
|
14
|
+
git add <files> # or: git add -A
|
|
15
|
+
|
|
16
|
+
# 2. Create commit (branch commit format)
|
|
17
|
+
git commit -m "type(scope): subject
|
|
18
|
+
|
|
19
|
+
Body explaining HOW and WHY.
|
|
20
|
+
Reference: Task X.Y, Req N"
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Commit Types
|
|
24
|
+
|
|
25
|
+
### Regular Branch Commits (During Development)
|
|
26
|
+
|
|
27
|
+
**Format**: `type(scope): subject`
|
|
28
|
+
|
|
29
|
+
| Type | Purpose |
|
|
30
|
+
|------|---------|
|
|
31
|
+
| `feat` | New feature or functionality |
|
|
32
|
+
| `fix` | Bug fix or issue resolution |
|
|
33
|
+
| `refactor` | Code refactoring without behavior change |
|
|
34
|
+
| `perf` | Performance improvements |
|
|
35
|
+
| `test` | Test additions or modifications |
|
|
36
|
+
| `ci` | CI/CD configuration changes |
|
|
37
|
+
| `docs` | Documentation updates |
|
|
38
|
+
| `chore` | Maintenance, dependencies, tooling |
|
|
39
|
+
| `style` | Code formatting, linting (non-functional) |
|
|
40
|
+
| `security` | Security vulnerability fixes or hardening |
|
|
41
|
+
|
|
42
|
+
### Scope (Required, kebab-case)
|
|
43
|
+
|
|
44
|
+
Examples: `validation`, `auth`, `cookie-service`, `template`, `config`, `tests`, `api`
|
|
45
|
+
|
|
46
|
+
### Subject Line Rules
|
|
47
|
+
|
|
48
|
+
- Max 50 characters after colon
|
|
49
|
+
- Present tense imperative: add, implement, fix, improve, enhance, refactor, remove, prevent
|
|
50
|
+
- NO period at the end
|
|
51
|
+
- Specific and descriptive - state WHAT, not WHY
|
|
52
|
+
|
|
53
|
+
## Core Workflow
|
|
54
|
+
|
|
55
|
+
### 1. Review Changes
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
git status
|
|
59
|
+
git diff --staged # if already staged
|
|
60
|
+
git diff # if not staged
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
### 2. Stage Files
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
git add <specific-files> # preferred
|
|
67
|
+
# or
|
|
68
|
+
git add -A # all changes
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
**NEVER commit**:
|
|
72
|
+
- `.env`, `credentials.json`, secrets
|
|
73
|
+
- `node_modules/`, `__pycache__/`, `.venv/`
|
|
74
|
+
- Large binary files without explicit approval
|
|
75
|
+
|
|
76
|
+
### 3. Create Commit
|
|
77
|
+
|
|
78
|
+
**Simple change**:
|
|
79
|
+
```bash
|
|
80
|
+
git commit -m "fix(auth): use hmac.compare_digest for secure comparison"
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
**Complex change (with body)**:
|
|
84
|
+
```bash
|
|
85
|
+
git commit -m "$(cat <<'EOF'
|
|
86
|
+
feat(validation): add URLValidator with domain whitelist
|
|
87
|
+
|
|
88
|
+
Implement URLValidator class supporting:
|
|
89
|
+
- Domain whitelist enforcement (youtube.com, youtu.be)
|
|
90
|
+
- Dangerous scheme blocking (javascript, data, file)
|
|
91
|
+
- URL parsing with embedded credentials handling
|
|
92
|
+
|
|
93
|
+
Addresses Requirement 31: Input validation
|
|
94
|
+
Part of Task 5.1: Input Validation Utilities
|
|
95
|
+
EOF
|
|
96
|
+
)"
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
### 4. Verify Commit
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
git log -1 --format="%h %s"
|
|
103
|
+
git show --stat HEAD
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
## Body Format (Recommended for Complex Changes)
|
|
107
|
+
|
|
108
|
+
```
|
|
109
|
+
<blank line>
|
|
110
|
+
Explain HOW and WHY the change was made.
|
|
111
|
+
- Use bullet points for multiple items
|
|
112
|
+
- Wrap at 72 characters
|
|
113
|
+
|
|
114
|
+
Reference: Task X.Y
|
|
115
|
+
Addresses: Req N
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
## Git Trailers
|
|
119
|
+
|
|
120
|
+
| Trailer | Purpose |
|
|
121
|
+
|---------|---------|
|
|
122
|
+
| `Fixes #N` | Links and closes issue on merge |
|
|
123
|
+
| `Closes #N` | Same as Fixes |
|
|
124
|
+
| `Co-authored-by: Name <email>` | Credit co-contributors |
|
|
125
|
+
|
|
126
|
+
Place trailers at end of body after blank line. See `references/commit_examples.md` for examples.
|
|
127
|
+
|
|
128
|
+
## Breaking Changes
|
|
129
|
+
|
|
130
|
+
For incompatible API/behavior changes, use `!` after scope OR `BREAKING CHANGE:` footer:
|
|
131
|
+
|
|
132
|
+
```
|
|
133
|
+
feat(api)!: change response format to JSON:API
|
|
134
|
+
|
|
135
|
+
BREAKING CHANGE: Response envelope changed from `{ data }` to `{ data: { type, id, attributes } }`.
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
Triggers major version bump in semantic-release.
|
|
139
|
+
|
|
140
|
+
## Merge Commits (PR Closure)
|
|
141
|
+
|
|
142
|
+
For PRs, use extended description with sections:
|
|
143
|
+
|
|
144
|
+
```bash
|
|
145
|
+
gh pr create --title "feat(security): implement input validation (Task 5)" --body "$(cat <<'EOF'
|
|
146
|
+
## Summary
|
|
147
|
+
- Input validation utilities (URLValidator, FormatValidator)
|
|
148
|
+
- Secure template processor with path traversal prevention
|
|
149
|
+
- API key authentication middleware
|
|
150
|
+
|
|
151
|
+
## Task Breakdown
|
|
152
|
+
Task 5.1: Input Validation - URLValidator, FormatValidator
|
|
153
|
+
Task 5.2: Template Processing - Path traversal prevention
|
|
154
|
+
Task 5.3: API Key Auth - Multi-key support, excluded paths
|
|
155
|
+
Task 5.4: Security Tests - 102 path traversal tests
|
|
156
|
+
|
|
157
|
+
## Requirements Covered
|
|
158
|
+
Req 7, Req 9, Req 31, Req 33
|
|
159
|
+
|
|
160
|
+
## Test Coverage
|
|
161
|
+
- All 473 tests passing
|
|
162
|
+
- Coverage: 93%
|
|
163
|
+
- Pre-commit checks: passing
|
|
164
|
+
EOF
|
|
165
|
+
)"
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
## Integration with Other Skills
|
|
169
|
+
|
|
170
|
+
### From github-pr-review
|
|
171
|
+
|
|
172
|
+
When fixing review comments, use this format:
|
|
173
|
+
|
|
174
|
+
```bash
|
|
175
|
+
git commit -m "fix(scope): address review comment #ID
|
|
176
|
+
|
|
177
|
+
Brief explanation of what was wrong and how it's fixed.
|
|
178
|
+
Addresses review comment #123456789."
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
### From github-pr-creation
|
|
182
|
+
|
|
183
|
+
Before creating PR, ensure all commits follow this format. The PR skill will:
|
|
184
|
+
1. Analyze commits for proper format
|
|
185
|
+
2. Extract types for PR labels
|
|
186
|
+
3. Build PR description from commit bodies
|
|
187
|
+
|
|
188
|
+
## Important Rules
|
|
189
|
+
|
|
190
|
+
- **ALWAYS** include scope in parentheses
|
|
191
|
+
- **ALWAYS** use present tense imperative verb
|
|
192
|
+
- **NEVER** end subject with period
|
|
193
|
+
- **NEVER** commit secrets or credentials
|
|
194
|
+
- **NEVER** use generic messages ("update code", "fix bug", "changes")
|
|
195
|
+
- **NEVER** exceed 50 chars in subject line
|
|
196
|
+
- Group related changes -> single focused commit
|
|
197
|
+
|
|
198
|
+
## Examples
|
|
199
|
+
|
|
200
|
+
**Good**:
|
|
201
|
+
```
|
|
202
|
+
feat(validation): add URLValidator with domain whitelist
|
|
203
|
+
fix(auth): use hmac.compare_digest for secure key comparison
|
|
204
|
+
refactor(template): consolidate filename sanitization logic
|
|
205
|
+
test(security): add 102 path traversal prevention tests
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
**Bad**:
|
|
209
|
+
```
|
|
210
|
+
update validation code # no type, no scope, vague
|
|
211
|
+
feat: add stuff # missing scope, too vague
|
|
212
|
+
fix(auth): fix bug # circular, not specific
|
|
213
|
+
chore: make changes # missing scope, vague
|
|
214
|
+
feat(security): improve things. # has period, vague
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
## References
|
|
218
|
+
|
|
219
|
+
- `references/commit_examples.md` - Extended examples by type
|
|
@@ -0,0 +1,292 @@
|
|
|
1
|
+
# Commit Examples by Type
|
|
2
|
+
|
|
3
|
+
Extended examples for each commit type with body content.
|
|
4
|
+
|
|
5
|
+
## feat - New Features
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
feat(validation): add URLValidator with domain whitelist
|
|
9
|
+
|
|
10
|
+
Implement URLValidator class supporting:
|
|
11
|
+
- Domain whitelist enforcement (youtube.com, youtu.be, m.youtube.com)
|
|
12
|
+
- Dangerous scheme blocking (javascript, data, file)
|
|
13
|
+
- URL parsing with embedded credentials handling
|
|
14
|
+
- Port number validation (1-65535)
|
|
15
|
+
|
|
16
|
+
Addresses Requirement 31: Input validation
|
|
17
|
+
Part of Task 5.1: Input Validation Utilities
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
```
|
|
21
|
+
feat(api): add video metadata endpoint
|
|
22
|
+
|
|
23
|
+
New GET /api/v1/videos/{id}/metadata endpoint:
|
|
24
|
+
- Returns title, duration, formats, thumbnails
|
|
25
|
+
- Supports format filtering via query params
|
|
26
|
+
- Implements caching with 5-minute TTL
|
|
27
|
+
|
|
28
|
+
Part of Task 6.2: API Endpoints
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
## fix - Bug Fixes
|
|
32
|
+
|
|
33
|
+
```
|
|
34
|
+
fix(auth): use hmac.compare_digest for secure key comparison
|
|
35
|
+
|
|
36
|
+
Replace direct string equality with hmac.compare_digest to prevent
|
|
37
|
+
timing attacks on API key validation. Ensures constant-time comparison
|
|
38
|
+
regardless of key length or content.
|
|
39
|
+
|
|
40
|
+
Addresses security best practice for sensitive data comparison
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
```
|
|
44
|
+
fix(download): handle network timeout during video fetch
|
|
45
|
+
|
|
46
|
+
Add retry logic with exponential backoff for network failures:
|
|
47
|
+
- Max 3 attempts with delays [2, 4, 8] seconds
|
|
48
|
+
- Classify retriable errors (5xx, timeout, connection)
|
|
49
|
+
- Log each retry attempt with remaining count
|
|
50
|
+
|
|
51
|
+
Fixes issue where downloads would fail silently on flaky connections
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
## refactor - Code Improvements
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
refactor(template): consolidate filename sanitization logic
|
|
58
|
+
|
|
59
|
+
Extract common sanitization patterns into helper methods:
|
|
60
|
+
- Path traversal prevention (.., /, absolute paths)
|
|
61
|
+
- Special character removal (control chars, null bytes)
|
|
62
|
+
- Windows reserved name handling (CON, PRN, LPT1-9, etc)
|
|
63
|
+
|
|
64
|
+
Improves code maintainability and reduces duplication
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
```
|
|
68
|
+
refactor(providers): extract common yt-dlp options builder
|
|
69
|
+
|
|
70
|
+
Move duplicated option building from get_info/download to
|
|
71
|
+
_build_base_options helper. Reduces code duplication and ensures
|
|
72
|
+
consistent option handling across all provider methods.
|
|
73
|
+
|
|
74
|
+
No behavior change, pure refactoring
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## test - Test Changes
|
|
78
|
+
|
|
79
|
+
```
|
|
80
|
+
test(security): add 102 path traversal prevention tests
|
|
81
|
+
|
|
82
|
+
Comprehensive test coverage for TemplateProcessor including:
|
|
83
|
+
- Basic path traversal attempts (.., /)
|
|
84
|
+
- URL-encoded variants (%2e%2e, %2f)
|
|
85
|
+
- Unicode/UTF-8 bypass attempts
|
|
86
|
+
- Windows edge cases (backslashes, drive letters)
|
|
87
|
+
|
|
88
|
+
Part of Task 5.4: Security Test Suite
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
```
|
|
92
|
+
test(validation): add parametrized URL validation tests
|
|
93
|
+
|
|
94
|
+
Add 25 test cases covering:
|
|
95
|
+
- Valid YouTube URL formats (watch, shorts, embed, youtu.be)
|
|
96
|
+
- Invalid domains (vimeo, dailymotion)
|
|
97
|
+
- Malformed URLs (no scheme, wrong port)
|
|
98
|
+
- Edge cases (trailing slashes, query params)
|
|
99
|
+
|
|
100
|
+
Coverage for URLValidator: 98%
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
## perf - Performance
|
|
104
|
+
|
|
105
|
+
```
|
|
106
|
+
perf(cache): implement LRU eviction for metadata cache
|
|
107
|
+
|
|
108
|
+
Replace dict-based cache with LRU implementation:
|
|
109
|
+
- Max 1000 entries with automatic eviction
|
|
110
|
+
- 40% memory reduction under high load
|
|
111
|
+
- Sub-millisecond lookup times maintained
|
|
112
|
+
|
|
113
|
+
Addresses memory growth issue in long-running instances
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
## security - Security Fixes
|
|
117
|
+
|
|
118
|
+
```
|
|
119
|
+
security(cookie): validate cookie file integrity before use
|
|
120
|
+
|
|
121
|
+
Add SHA256 checksum verification for cookie files:
|
|
122
|
+
- Compute hash on first load, store in memory
|
|
123
|
+
- Verify hash before each use
|
|
124
|
+
- Reject modified files with clear error message
|
|
125
|
+
|
|
126
|
+
Prevents use of tampered cookie files
|
|
127
|
+
Addresses Requirement 33: Security validation
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
## ci - CI/CD Changes
|
|
131
|
+
|
|
132
|
+
```
|
|
133
|
+
ci(github): add security scanning to PR workflow
|
|
134
|
+
|
|
135
|
+
Enable Bandit security scanner in GitHub Actions:
|
|
136
|
+
- Run on all Python files
|
|
137
|
+
- Fail on HIGH/CRITICAL findings
|
|
138
|
+
- Cache virtualenv for faster runs
|
|
139
|
+
|
|
140
|
+
Part of Task 15.3: Basic security validation
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
## docs - Documentation
|
|
144
|
+
|
|
145
|
+
```
|
|
146
|
+
docs(api): add OpenAPI description for download endpoint
|
|
147
|
+
|
|
148
|
+
Document /api/v1/download endpoint:
|
|
149
|
+
- Request body schema with format options
|
|
150
|
+
- Response codes (200, 400, 401, 404, 500)
|
|
151
|
+
- Example requests and responses
|
|
152
|
+
|
|
153
|
+
Improves API documentation for consumers
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
## chore - Maintenance
|
|
157
|
+
|
|
158
|
+
```
|
|
159
|
+
chore(deps): update yt-dlp to 2024.12.06
|
|
160
|
+
|
|
161
|
+
Update yt-dlp from 2024.11.15 to 2024.12.06:
|
|
162
|
+
- Fixes YouTube throttling detection
|
|
163
|
+
- Adds support for new Instagram format
|
|
164
|
+
- Improves error messages for geo-blocked content
|
|
165
|
+
|
|
166
|
+
No breaking changes expected
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
## style - Formatting
|
|
170
|
+
|
|
171
|
+
```
|
|
172
|
+
style(providers): apply black formatting to youtube.py
|
|
173
|
+
|
|
174
|
+
Apply black formatter with 88 char line length.
|
|
175
|
+
No functional changes, formatting only.
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
## Merge Commit Examples
|
|
179
|
+
|
|
180
|
+
### Feature Branch to Develop
|
|
181
|
+
|
|
182
|
+
```
|
|
183
|
+
Merge pull request #5 from fvadicamo/feature/input-validation-security
|
|
184
|
+
|
|
185
|
+
feat(security): implement input validation and security (Task 5)
|
|
186
|
+
|
|
187
|
+
Merges comprehensive security implementation (Task 5) into develop:
|
|
188
|
+
- Input validation utilities (URLValidator, FormatValidator, ParameterValidator)
|
|
189
|
+
- Secure template processor with path traversal prevention
|
|
190
|
+
- API key authentication middleware with multi-key support
|
|
191
|
+
- 473 tests with 93% coverage
|
|
192
|
+
|
|
193
|
+
Task 5.1: Input Validation Utilities
|
|
194
|
+
- URLValidator: Domain whitelist (youtube.com, youtu.be), dangerous scheme blocking
|
|
195
|
+
- FormatValidator: yt-dlp format ID validation with regex and selectors
|
|
196
|
+
- ParameterValidator: Audio quality/format and language code validation
|
|
197
|
+
|
|
198
|
+
Task 5.2: Template Processor
|
|
199
|
+
- Path traversal prevention (.., /, absolute paths, URL encoding)
|
|
200
|
+
- Filename sanitization (illegal chars, control chars, null bytes)
|
|
201
|
+
- Windows reserved names handling (CON, PRN, AUX, NUL, COM1-9, LPT1-9)
|
|
202
|
+
- Collision handling with numeric suffix, max length 200 chars
|
|
203
|
+
|
|
204
|
+
Task 5.3: API Key Authentication
|
|
205
|
+
- APIKeyAuth class with multi-key support
|
|
206
|
+
- Excluded paths for health/doc endpoints
|
|
207
|
+
- Secure hashing for logging (SHA256 first 8 chars)
|
|
208
|
+
- FastAPI dependency injection integration
|
|
209
|
+
|
|
210
|
+
Task 5.4: Security Tests
|
|
211
|
+
- 102 path traversal prevention tests with edge cases
|
|
212
|
+
- URL validation tests with malicious inputs
|
|
213
|
+
- API key authentication and credential tests
|
|
214
|
+
- Sensitive data redaction verification
|
|
215
|
+
|
|
216
|
+
Requirements Covered:
|
|
217
|
+
- Req 7: Output template processing with security
|
|
218
|
+
- Req 9: API key authentication
|
|
219
|
+
- Req 31: Input validation
|
|
220
|
+
- Req 33: Security (secure comparison, log redaction)
|
|
221
|
+
|
|
222
|
+
Test Coverage:
|
|
223
|
+
- All 473 tests passing
|
|
224
|
+
- Coverage: 93% (exceeds 80% minimum)
|
|
225
|
+
- Pre-commit checks: all passing
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
### Develop to Main (Release)
|
|
229
|
+
|
|
230
|
+
```
|
|
231
|
+
Merge pull request #10 from fvadicamo/develop
|
|
232
|
+
|
|
233
|
+
release: v0.1.0 - MVP with YouTube provider
|
|
234
|
+
|
|
235
|
+
First stable release with core functionality:
|
|
236
|
+
- YouTube video info, formats, download, audio extraction
|
|
237
|
+
- Cookie-based authentication for age-restricted content
|
|
238
|
+
- API key authentication
|
|
239
|
+
- Input validation and security hardening
|
|
240
|
+
- 500+ tests with 92% coverage
|
|
241
|
+
|
|
242
|
+
Breaking Changes: None (initial release)
|
|
243
|
+
|
|
244
|
+
Features:
|
|
245
|
+
- GET /api/v1/info - Video metadata
|
|
246
|
+
- GET /api/v1/formats - Available formats
|
|
247
|
+
- POST /api/v1/download - Video/audio download
|
|
248
|
+
- Cookie file support for authenticated requests
|
|
249
|
+
|
|
250
|
+
Documentation:
|
|
251
|
+
- API documentation at /docs (Swagger UI)
|
|
252
|
+
- OpenAPI spec at /openapi.json
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
## Commits with Trailers
|
|
256
|
+
|
|
257
|
+
### Single Issue
|
|
258
|
+
```
|
|
259
|
+
fix(validation): prevent XSS in user input
|
|
260
|
+
|
|
261
|
+
Escape HTML entities before rendering.
|
|
262
|
+
|
|
263
|
+
Fixes #78
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
### Multiple Issues + Co-author
|
|
267
|
+
```
|
|
268
|
+
fix(auth): resolve session and token issues
|
|
269
|
+
|
|
270
|
+
- Fix session expiry not triggering logout
|
|
271
|
+
- Fix token refresh race condition
|
|
272
|
+
|
|
273
|
+
Fixes #101
|
|
274
|
+
Fixes #103
|
|
275
|
+
Co-authored-by: Bob <bob@example.com>
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
## Breaking Changes
|
|
279
|
+
|
|
280
|
+
### With ! Notation
|
|
281
|
+
```
|
|
282
|
+
feat(api)!: migrate to v2 endpoints
|
|
283
|
+
|
|
284
|
+
BREAKING CHANGE: /api/v1/* endpoints removed. Update base URL to /api/v2/.
|
|
285
|
+
```
|
|
286
|
+
|
|
287
|
+
### Config Breaking Change
|
|
288
|
+
```
|
|
289
|
+
chore(config)!: rename environment variables
|
|
290
|
+
|
|
291
|
+
BREAKING CHANGE: DATABASE_URL -> APP_DATABASE_URL, API_KEY -> APP_API_KEY
|
|
292
|
+
```
|