better-dsh 0.2.4 → 0.2.5

package/docs/80_upstream-docs-and-skills/hermes-dev-skill/src/docs/user-guide/skills/bundled/software-development/software-development-test-driven-development.md

@@ -0,0 +1,380 @@
+---
+title: "Test Driven Development — TDD: enforce RED-GREEN-REFACTOR, tests before code"
+sidebar_label: "Test Driven Development"
+description: "TDD: enforce RED-GREEN-REFACTOR, tests before code"
+---
+
+{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
+
+# Test Driven Development
+
+TDD: enforce RED-GREEN-REFACTOR, tests before code.
+
+## Skill metadata
+
+| | |
+|---|---|
+| Source | Bundled (installed by default) |
+| Path | `skills/software-development\test-driven-development` |
+| Version | `1.1.0` |
+| Author | Hermes Agent (adapted from obra/superpowers) |
+| License | MIT |
+| Platforms | linux, macos, windows |
+| Tags | `testing`, `tdd`, `development`, `quality`, `red-green-refactor` |
+| Related skills | [`systematic-debugging`](/docs/user-guide/skills/bundled/software-development/software-development-systematic-debugging), [`subagent-driven-development`](/docs/user-guide/skills/optional/software-development/software-development-subagent-driven-development) |
+
+## Reference: full SKILL.md
+
+:::info
+The following is the complete skill definition that Hermes loads when this skill is triggered. This is what the agent sees as instructions when the skill is active.
+:::
+
+# Test-Driven Development (TDD)
+
+## Overview
+
+Write the test first. Watch it fail. Write minimal code to pass.
+
+**Core principle:** If you didn't watch the test fail, you don't know if it tests the right thing.
+
+**Violating the letter of the rules is violating the spirit of the rules.**
+
+## When to Use
+
+**Always:**
+- New features
+- Bug fixes
+- Refactoring
+- Behavior changes
+
+**Exceptions (ask the user first):**
+- Throwaway prototypes
+- Generated code
+- Configuration files
+
+Thinking "skip TDD just this once"? Stop. That's rationalization.
+
+## The Iron Law
+
+```
+NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST
+```
+
+Write code before the test? Delete it. Start over.
+
+**No exceptions:**
+- Don't keep it as "reference"
+- Don't "adapt" it while writing tests
+- Don't look at it
+- Delete means delete
+
+Implement fresh from tests. Period.
+
+## Red-Green-Refactor Cycle
+
+### RED — Write Failing Test
+
+Write one minimal test showing what should happen.
+
+**Good test:**
+```python
+def test_retries_failed_operations_3_times():
+    attempts = 0
+    def operation():
+        nonlocal attempts
+        attempts += 1
+        if attempts < 3:
+            raise Exception('fail')
+        return 'success'
+
+    result = retry_operation(operation)
+
+    assert result == 'success'
+    assert attempts == 3
+```
+Clear name, tests real behavior, one thing.
+
+**Bad test:**
+```python
+def test_retry_works():
+    mock = MagicMock()
+    mock.side_effect = [Exception(), Exception(), 'success']
+    result = retry_operation(mock)
+    assert result == 'success'  # What about retry count? Timing?
+```
+Vague name, tests mock not real code.
+
+**Requirements:**
+- One behavior per test
+- Clear descriptive name ("and" in name? Split it)
+- Real code, not mocks (unless truly unavoidable)
+- Name describes behavior, not implementation
+
+### Verify RED — Watch It Fail
+
+**MANDATORY. Never skip.**
+
+```bash
+# Use terminal tool to run the specific test
+pytest tests/test_feature.py::test_specific_behavior -v
+```
+
+Confirm:
+- Test fails (not errors from typos)
+- Failure message is expected
+- Fails because the feature is missing
+
+**Test passes immediately?** You're testing existing behavior. Fix the test.
+
+**Test errors?** Fix the error, re-run until it fails correctly.
+
+### GREEN — Minimal Code
+
+Write the simplest code to pass the test. Nothing more.
+
+**Good:**
+```python
+def add(a, b):
+    return a + b  # Nothing extra
+```
+
+**Bad:**
+```python
+def add(a, b):
+    result = a + b
+    logging.info(f"Adding {a} + {b} = {result}")  # Extra!
+    return result
+```
+
+Don't add features, refactor other code, or "improve" beyond the test.
+
+**Cheating is OK in GREEN:**
+- Hardcode return values
+- Copy-paste
+- Duplicate code
+- Skip edge cases
+
+We'll fix it in REFACTOR.
+
+### Verify GREEN — Watch It Pass
+
+**MANDATORY.**
+
+```bash
+# Run the specific test
+pytest tests/test_feature.py::test_specific_behavior -v
+
+# Then run ALL tests to check for regressions
+pytest tests/ -q
+```
+
+Confirm:
+- Test passes
+- Other tests still pass
+- Output pristine (no errors, warnings)
+
+**Test fails?** Fix the code, not the test.
+
+**Other tests fail?** Fix regressions now.
+
+### REFACTOR — Clean Up
+
+After green only:
+- Remove duplication
+- Improve names
+- Extract helpers
+- Simplify expressions
+
+Keep tests green throughout. Don't add behavior.
+
+**If tests fail during refactor:** Undo immediately. Take smaller steps.
+
+### Repeat
+
+Next failing test for next behavior. One cycle at a time.
+
+## Avoid Horizontal Slices
+
+Do **not** write all tests first and then all implementation. That is horizontal slicing: RED becomes "write a pile of imagined tests" and GREEN becomes "make the pile pass." It produces brittle tests because the tests are designed before the implementation has taught you what behavior and interface actually matter.
+
+Use vertical tracer bullets instead:
+
+```text
+WRONG:
+  RED:   test1, test2, test3, test4
+  GREEN: impl1, impl2, impl3, impl4
+
+RIGHT:
+  RED→GREEN: test1→impl1
+  RED→GREEN: test2→impl2
+  RED→GREEN: test3→impl3
+```
+
+A tracer bullet is one end-to-end behavior slice. It proves the path works, teaches you about the interface, and keeps each next test grounded in what you just learned.
+
+## Why Order Matters
+
+**"I'll write tests after to verify it works"**
+
+Tests written after code pass immediately. Passing immediately proves nothing:
+- Might test the wrong thing
+- Might test implementation, not behavior
+- Might miss edge cases you forgot
+- You never saw it catch the bug
+
+Test-first forces you to see the test fail, proving it actually tests something.
+
+**"I already manually tested all the edge cases"**
+
+Manual testing is ad-hoc. You think you tested everything but:
+- No record of what you tested
+- Can't re-run when code changes
+- Easy to forget cases under pressure
+- "It worked when I tried it" ≠ comprehensive
+
+Automated tests are systematic. They run the same way every time.
+
+**"Deleting X hours of work is wasteful"**
+
+Sunk cost fallacy. The time is already gone. Your choice now:
+- Delete and rewrite with TDD (high confidence)
+- Keep it and add tests after (low confidence, likely bugs)
+
+The "waste" is keeping code you can't trust.
+
+**"TDD is dogmatic, being pragmatic means adapting"**
+
+TDD IS pragmatic:
+- Finds bugs before commit (faster than debugging after)
+- Prevents regressions (tests catch breaks immediately)
+- Documents behavior (tests show how to use code)
+- Enables refactoring (change freely, tests catch breaks)
+
+"Pragmatic" shortcuts = debugging in production = slower.
+
+**"Tests after achieve the same goals — it's spirit not ritual"**
+
+No. Tests-after answer "What does this do?" Tests-first answer "What should this do?"
+
+Tests-after are biased by your implementation. You test what you built, not what's required. Tests-first force edge case discovery before implementing.
+
+## Common Rationalizations
+
+| Excuse | Reality |
+|--------|---------|
+| "Too simple to test" | Simple code breaks. Test takes 30 seconds. |
+| "I'll test after" | Tests passing immediately prove nothing. |
+| "Tests after achieve same goals" | Tests-after = "what does this do?" Tests-first = "what should this do?" |
+| "Already manually tested" | Ad-hoc ≠ systematic. No record, can't re-run. |
+| "Deleting X hours is wasteful" | Sunk cost fallacy. Keeping unverified code is technical debt. |
+| "Keep as reference, write tests first" | You'll adapt it. That's testing after. Delete means delete. |
+| "Need to explore first" | Fine. Throw away exploration, start with TDD. |
+| "Test hard = design unclear" | Listen to the test. Hard to test = hard to use. |
+| "TDD will slow me down" | TDD faster than debugging. Pragmatic = test-first. |
+| "Manual test faster" | Manual doesn't prove edge cases. You'll re-test every change. |
+| "Existing code has no tests" | You're improving it. Add tests for the code you touch. |
+
+## Red Flags — STOP and Start Over
+
+If you catch yourself doing any of these, delete the code and restart with TDD:
+
+- Code before test
+- Test after implementation
+- Test passes immediately on first run
+- Can't explain why test failed
+- Tests added "later"
+- Rationalizing "just this once"
+- "I already manually tested it"
+- "Tests after achieve the same purpose"
+- "Keep as reference" or "adapt existing code"
+- "Already spent X hours, deleting is wasteful"
+- "TDD is dogmatic, I'm being pragmatic"
+- "This is different because..."
+
+**All of these mean: Delete code. Start over with TDD.**
+
+## Verification Checklist
+
+Before marking work complete:
+
+- [ ] Every new function/method has a test
+- [ ] Watched each test fail before implementing
+- [ ] Each test failed for expected reason (feature missing, not typo)
+- [ ] Wrote minimal code to pass each test
+- [ ] All tests pass
+- [ ] Output pristine (no errors, warnings)
+- [ ] Tests use real code (mocks only if unavoidable)
+- [ ] Edge cases and errors covered
+
+Can't check all boxes? You skipped TDD. Start over.
+
+## When Stuck
+
+| Problem | Solution |
+|---------|----------|
+| Don't know how to test | Write the wished-for API. Write the assertion first. Ask the user. |
+| Test too complicated | Design too complicated. Simplify the interface. |
+| Must mock everything | Code too coupled. Use dependency injection. |
+| Test setup huge | Extract helpers. Still complex? Simplify the design. |
+
+## Hermes Agent Integration
+
+### Running Tests
+
+Use the `terminal` tool to run tests at each step:
+
+```python
+# RED — verify failure
+terminal("pytest tests/test_feature.py::test_name -v")
+
+# GREEN — verify pass
+terminal("pytest tests/test_feature.py::test_name -v")
+
+# Full suite — verify no regressions
+terminal("pytest tests/ -q")
+```
+
+### With delegate_task
+
+When dispatching subagents for implementation, enforce TDD in the goal:
+
+```python
+delegate_task(
+    goal="Implement [feature] using strict TDD",
+    context="""
+    Follow test-driven-development skill:
+    1. Write failing test FIRST
+    2. Run test to verify it fails
+    3. Write minimal code to pass
+    4. Run test to verify it passes
+    5. Refactor if needed
+    6. Commit
+
+    Project test command: pytest tests/ -q
+    Project structure: [describe relevant files]
+    """,
+    toolsets=['terminal', 'file']
+)
+```
+
+### With systematic-debugging
+
+Bug found? Write failing test reproducing it. Follow TDD cycle. The test proves the fix and prevents regression.
+
+Never fix bugs without a test.
+
+## Testing Anti-Patterns
+
+- **Testing mock behavior instead of real behavior** — mocks should verify interactions, not replace the system under test
+- **Testing implementation details** — test behavior/results, not internal method calls
+- **Happy path only** — always test edge cases, errors, and boundaries
+- **Brittle tests** — tests should verify behavior, not structure; refactoring shouldn't break them
+
+## Final Rule
+
+```
+Production code → test exists and failed first
+Otherwise → not TDD
+```
+
+No exceptions without the user's explicit permission.

package/docs/80_upstream-docs-and-skills/hermes-dev-skill/src/docs/user-guide/skills/bundled/web/web-blocked-page-recovery.md

@@ -0,0 +1,155 @@
+---
+title: "Blocked Page Recovery — Use when a fetch fails: 403/429, paywall, WAF, bot wall"
+sidebar_label: "Blocked Page Recovery"
+description: "Use when a fetch fails: 403/429, paywall, WAF, bot wall"
+---
+
+{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
+
+# Blocked Page Recovery
+
+Use when a fetch fails: 403/429, paywall, WAF, bot wall.
+
+## Skill metadata
+
+| | |
+|---|---|
+| Source | Bundled (installed by default) |
+| Path | `skills/web\blocked-page-recovery` |
+| Version | `1.0.0` |
+| Author | Hermes Agent |
+| License | MIT |
+| Platforms | linux, macos, windows |
+| Tags | `Research`, `Archives`, `Wayback`, `Paywall`, `WAF`, `Fallback` |
+| Related skills | [`grounded-citations`](/docs/user-guide/skills/bundled/research/research-grounded-citations) |
+
+## Reference: full SKILL.md
+
+:::info
+The following is the complete skill definition that Hermes loads when this skill is triggered. This is what the agent sees as instructions when the skill is active.
+:::
+
+# Blocked-Page Recovery
+
+When a page won't fetch — 403/429, Cloudflare "Just a moment...", a paywall,
+or a bot-detection interstitial — don't give up and don't loop on the same
+URL. Third-party services often hold a **copy** of the page. Work down this
+ladder, cheapest first.
+
+## The ladder
+
+```
+1. Wayback Machine  — archive.org "available" API  (snapshot + timestamp)
+2. archive.today    — domain rotation: archive.ph → .md → .li → .is
+3. Jina Reader      — only if JINA_API_KEY is set  (live server-side render)
+4. API-first pivot  — look for /api/, /graphql, .json, or RSS on the same host
+5. Real browser     — browser tool as the last, most expensive resort
+```
+
+Run it in one shot with the bundled script:
+
+```bash
+python3 scripts/recover_page.py "https://example.com/blocked-article" --json
+```
+
+The script tries each route in order, validates every body (see "Fake
+successes" below), and prints the first genuine hit with its provenance.
+
+## Provenance discipline (non-negotiable)
+
+Every recovered copy carries a provenance you MUST preserve when citing:
+
+| Route | Provenance | How to cite |
+|-------|-----------|-------------|
+| Wayback / archive.today | `snapshot` | Cite WITH the snapshot date: "as archived 2026-08-06". Never present a snapshot as the live page — it may be stale. |
+| Jina Reader | `live` | Server-side re-render of the live page; cite normally. |
+| Live fetch / browser | `live` | Cite normally. |
+
+If the user needs *current* data (prices, availability, breaking news), a
+snapshot is context, not an answer — say so explicitly and note its age.
+
+## Manual routes
+
+### 1. Wayback Machine (best provenance, try first)
+
+```bash
+# Discovery: returns closest snapshot URL + timestamp as JSON
+curl -sL "https://archive.org/wayback/available?url={URL}"
+# Then fetch archived_snapshots.closest.url
+```
+
+For enumerating many snapshots (or recovering deleted pages), the CDX index:
+
+```bash
+curl -sL "https://web.archive.org/cdx/search/cdx?url={URL}&output=json&limit=10"
+```
+
+CDX intermittently returns 503 under load — if it does, fall back to the
+`available` API; don't retry-hammer it.
+
+Works for: any publicly crawled URL. Fails for: robots-blocked sites,
+never-crawled URLs, JS-only SPAs (snapshots don't render).
+
+### 2. archive.today (paywalls, deleted content)
+
+User-submitted archives — often has paywalled news articles Wayback lacks.
+Rate-limits aggressively (429) and rotates domains, so iterate:
+
+```bash
+for d in archive.ph archive.md archive.li archive.is; do
+  curl -sL --max-time 20 "https://$d/newest/{URL}" -o /tmp/page.html \
+    -w "%{http_code}" && break
+done
+```
+
+**Validate the body, not the status code** — a 429 still ships several KB of
+rate-limit HTML that looks like a success to a size check alone.
+
+### 3. Jina Reader (requires JINA_API_KEY)
+
+`r.jina.ai` re-renders the live page in a real browser server-side and
+returns markdown. Anonymous access is dead (401 → Turnstile); a key is
+required:
+
+```bash
+curl -s -H "Authorization: Bearer $JINA_API_KEY" "https://r.jina.ai/{URL}"
+```
+
+Handles JS SPAs that archives can't. Skip this route entirely when the env
+var is unset.
+
+### 4. API-first pivot
+
+WAFs protect the HTML surface far more aggressively than the data endpoints
+behind it. After 2-3 blocked attempts on a site, stop fighting the HTML and
+look for:
+
+- `/api/...`, `/graphql`, or `.json` variants of the page URL
+- An RSS/Atom feed (`/feed`, `/rss`, `<link rel="alternate">` in any copy
+  you did recover)
+- A sitemap (`/sitemap.xml`) revealing canonical URLs that may not be gated
+
+## Fake successes — routes that LIE
+
+These return HTTP 200 with a plausible body that is NOT the page. The script
+rejects them automatically; reject them manually too:
+
+- **Google Cache is dead** (since mid-2024). `webcache.googleusercontent.com`
+  returns 200 + tens of KB, but it's a Google Search interstitial with a JS
+  redirect, not a cache. Never use it.
+- **AMP caches** (`*.cdn.ampproject.org`) mostly return a ~300-byte
+  `<title>Redirecting</title>` meta-refresh stub pointing back at the
+  original (blocked) URL. Treating that as success creates a fetch loop.
+- **Rate-limit bodies**: archive.today 429 pages are multi-KB HTML. Check for
+  the target's actual content (title words, expected strings), not just size.
+
+Detection heuristics the script applies: body under a per-route byte floor;
+meta-refresh/JS-redirect stubs whose target is the original host; interstitial
+titles ("Just a moment", "Redirecting", "Google Search", "Attention Required").
+
+## Proxy relays: don't
+
+Generic "web proxy" relays are man-in-the-middle by construction. Never send
+cookies or Authorization headers through one, and don't use them for anything
+the user will rely on — provenance is unverifiable. Prefer archives, which at
+least timestamp their copies.