vibecodeagent 0.0.0-stage → 0.1.65

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (109) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +46 -2
  3. package/THIRD-PARTY-NOTICES.txt +1427 -0
  4. package/dist/agen.js +138920 -0
  5. package/dist/bin.js +1245 -0
  6. package/dist/daemon.js +163701 -0
  7. package/dist/node_modules/node-ensure/index.js +10 -0
  8. package/dist/node_modules/node-ensure/package.json +31 -0
  9. package/dist/node_modules/pdf-parse/lib/pdf-parse.js +102 -0
  10. package/dist/node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.js +18538 -0
  11. package/dist/node_modules/pdf-parse/lib/pdf.js/v1.10.100/build/pdf.worker.js +45507 -0
  12. package/dist/node_modules/pdf-parse/package.json +49 -0
  13. package/dist/node_modules/sql.js/dist/sql-wasm.js +186 -0
  14. package/dist/node_modules/sql.js/dist/sql-wasm.wasm +0 -0
  15. package/dist/node_modules/sql.js/package.json +58 -0
  16. package/dist/resources/skills/builtin/.gitkeep +0 -0
  17. package/dist/resources/skills/builtin/NOTICE.md +35 -0
  18. package/dist/resources/skills/builtin/README.md +74 -0
  19. package/dist/resources/skills/builtin/backend-engineering/SKILL.md +102 -0
  20. package/dist/resources/skills/builtin/backend-engineering/references/api-contracts.md +320 -0
  21. package/dist/resources/skills/builtin/backend-engineering/references/auth-and-secrets.md +268 -0
  22. package/dist/resources/skills/builtin/backend-engineering/references/concurrency-and-jobs.md +386 -0
  23. package/dist/resources/skills/builtin/backend-engineering/references/data-modelling.md +367 -0
  24. package/dist/resources/skills/builtin/backend-engineering/references/deploy-and-config.md +228 -0
  25. package/dist/resources/skills/builtin/backend-engineering/references/migrations.md +320 -0
  26. package/dist/resources/skills/builtin/backend-engineering/references/observability.md +301 -0
  27. package/dist/resources/skills/builtin/backend-engineering/references/plan-template.md +152 -0
  28. package/dist/resources/skills/builtin/backend-engineering/references/review-audit.md +213 -0
  29. package/dist/resources/skills/builtin/backend-engineering/references/security.md +251 -0
  30. package/dist/resources/skills/builtin/backend-engineering/references/stack-go.md +238 -0
  31. package/dist/resources/skills/builtin/backend-engineering/references/stack-node.md +274 -0
  32. package/dist/resources/skills/builtin/backend-engineering/references/stack-python.md +296 -0
  33. package/dist/resources/skills/builtin/backend-engineering/references/testing-and-perf.md +298 -0
  34. package/dist/resources/skills/builtin/backend-engineering/references/validation-and-errors.md +266 -0
  35. package/dist/resources/skills/builtin/dasar-vibecode/SKILL.md +312 -0
  36. package/dist/resources/skills/builtin/excel-maker/SKILL.md +71 -0
  37. package/dist/resources/skills/builtin/frontend-design/SKILL.md +131 -0
  38. package/dist/resources/skills/builtin/frontend-design/data/app-interface.csv +33 -0
  39. package/dist/resources/skills/builtin/frontend-design/data/catalog-summary.json +78 -0
  40. package/dist/resources/skills/builtin/frontend-design/data/charts.csv +26 -0
  41. package/dist/resources/skills/builtin/frontend-design/data/colors.csv +193 -0
  42. package/dist/resources/skills/builtin/frontend-design/data/data-provenance.json +1348 -0
  43. package/dist/resources/skills/builtin/frontend-design/data/google-font-licenses.json +20185 -0
  44. package/dist/resources/skills/builtin/frontend-design/data/google-fonts.csv +1935 -0
  45. package/dist/resources/skills/builtin/frontend-design/data/icons.csv +106 -0
  46. package/dist/resources/skills/builtin/frontend-design/data/landing.csv +35 -0
  47. package/dist/resources/skills/builtin/frontend-design/data/motion.csv +18 -0
  48. package/dist/resources/skills/builtin/frontend-design/data/phosphor-icons-upstream.json +32582 -0
  49. package/dist/resources/skills/builtin/frontend-design/data/products.csv +193 -0
  50. package/dist/resources/skills/builtin/frontend-design/data/react-performance.csv +45 -0
  51. package/dist/resources/skills/builtin/frontend-design/data/stacks/angular.csv +51 -0
  52. package/dist/resources/skills/builtin/frontend-design/data/stacks/astro.csv +54 -0
  53. package/dist/resources/skills/builtin/frontend-design/data/stacks/avalonia.csv +57 -0
  54. package/dist/resources/skills/builtin/frontend-design/data/stacks/flutter.csv +53 -0
  55. package/dist/resources/skills/builtin/frontend-design/data/stacks/html-tailwind.csv +60 -0
  56. package/dist/resources/skills/builtin/frontend-design/data/stacks/javafx.csv +76 -0
  57. package/dist/resources/skills/builtin/frontend-design/data/stacks/jetpack-compose.csv +53 -0
  58. package/dist/resources/skills/builtin/frontend-design/data/stacks/laravel.csv +51 -0
  59. package/dist/resources/skills/builtin/frontend-design/data/stacks/nextjs.csv +62 -0
  60. package/dist/resources/skills/builtin/frontend-design/data/stacks/nuxt-ui.csv +71 -0
  61. package/dist/resources/skills/builtin/frontend-design/data/stacks/nuxtjs.csv +68 -0
  62. package/dist/resources/skills/builtin/frontend-design/data/stacks/react-native.csv +52 -0
  63. package/dist/resources/skills/builtin/frontend-design/data/stacks/react.csv +62 -0
  64. package/dist/resources/skills/builtin/frontend-design/data/stacks/shadcn.csv +69 -0
  65. package/dist/resources/skills/builtin/frontend-design/data/stacks/svelte.csv +56 -0
  66. package/dist/resources/skills/builtin/frontend-design/data/stacks/swiftui.csv +51 -0
  67. package/dist/resources/skills/builtin/frontend-design/data/stacks/threejs.csv +54 -0
  68. package/dist/resources/skills/builtin/frontend-design/data/stacks/uno.csv +60 -0
  69. package/dist/resources/skills/builtin/frontend-design/data/stacks/uwp.csv +56 -0
  70. package/dist/resources/skills/builtin/frontend-design/data/stacks/vue.csv +50 -0
  71. package/dist/resources/skills/builtin/frontend-design/data/stacks/winui.csv +60 -0
  72. package/dist/resources/skills/builtin/frontend-design/data/stacks/wpf.csv +57 -0
  73. package/dist/resources/skills/builtin/frontend-design/data/styles.csv +89 -0
  74. package/dist/resources/skills/builtin/frontend-design/data/typography.csv +75 -0
  75. package/dist/resources/skills/builtin/frontend-design/data/ui-reasoning.csv +193 -0
  76. package/dist/resources/skills/builtin/frontend-design/data/ux-guidelines.csv +120 -0
  77. package/dist/resources/skills/builtin/frontend-design/references/animation-patterns.md +705 -0
  78. package/dist/resources/skills/builtin/frontend-design/references/app-polish.md +117 -0
  79. package/dist/resources/skills/builtin/frontend-design/references/apple-fluid.md +278 -0
  80. package/dist/resources/skills/builtin/frontend-design/references/art-direction.md +280 -0
  81. package/dist/resources/skills/builtin/frontend-design/references/case-studies.md +599 -0
  82. package/dist/resources/skills/builtin/frontend-design/references/craft.md +128 -0
  83. package/dist/resources/skills/builtin/frontend-design/references/expo.md +251 -0
  84. package/dist/resources/skills/builtin/frontend-design/references/layout-systems.md +533 -0
  85. package/dist/resources/skills/builtin/frontend-design/references/libraries.md +72 -0
  86. package/dist/resources/skills/builtin/frontend-design/references/motion.md +222 -0
  87. package/dist/resources/skills/builtin/frontend-design/references/picker.md +197 -0
  88. package/dist/resources/skills/builtin/frontend-design/references/plan-template.md +73 -0
  89. package/dist/resources/skills/builtin/frontend-design/references/prototype.md +85 -0
  90. package/dist/resources/skills/builtin/frontend-design/references/recipes-expo.md +385 -0
  91. package/dist/resources/skills/builtin/frontend-design/references/recipes-web.md +421 -0
  92. package/dist/resources/skills/builtin/frontend-design/references/review-audit.md +107 -0
  93. package/dist/resources/skills/builtin/frontend-design/references/sonner.md +143 -0
  94. package/dist/resources/skills/builtin/frontend-design/references/technical-stack.md +543 -0
  95. package/dist/resources/skills/builtin/frontend-design/references/theming.md +60 -0
  96. package/dist/resources/skills/builtin/frontend-design/references/typography.md +393 -0
  97. package/dist/resources/skills/builtin/frontend-design/references/ux-rules.md +256 -0
  98. package/dist/resources/skills/builtin/frontend-design/references/vocabulary.md +169 -0
  99. package/dist/resources/skills/builtin/frontend-design/scripts/core.py +993 -0
  100. package/dist/resources/skills/builtin/frontend-design/scripts/design_system.py +1643 -0
  101. package/dist/resources/skills/builtin/frontend-design/scripts/reasoning_contract.py +123 -0
  102. package/dist/resources/skills/builtin/frontend-design/scripts/search.py +171 -0
  103. package/dist/resources/skills/builtin/frontend-design/scripts/validate_data.py +1091 -0
  104. package/dist/resources/skills/builtin/kerja-agentik/SKILL.md +117 -0
  105. package/dist/resources/skills/builtin/pdf-maker/SKILL.md +51 -0
  106. package/dist/resources/skills/builtin/tinjau-keamanan/SKILL.md +53 -0
  107. package/dist/resources/skills/builtin/word-maker/SKILL.md +58 -0
  108. package/dist/resources/skills/builtin/write-swift/SKILL.md +388 -0
  109. package/package.json +35 -4
@@ -0,0 +1,251 @@
1
+ # Security
2
+
3
+ This file owns the attacks that arrive through your own API: injection, SSRF, CORS, CSRF, rate limiting, and the authorization bug classes that return `200`. It is the canonical source for the SSRF rule set, the CORS allowlist shape, the CSRF decision, and the `429` contract.
4
+
5
+ Neighbours, so you don't half-solve a problem here: credentials themselves — cookie attributes, session ids, token lifetimes, password hashing, OAuth, RLS, and the IDOR fix — are [auth-and-secrets.md](auth-and-secrets.md), and every value it states is restated here identically or linked, never re-derived. Turning bytes into typed values, rejecting unknown fields, and what a response body may not contain are [validation-and-errors.md](validation-and-errors.md). The status-code table and error envelope are [api-contracts.md](api-contracts.md). Retry/backoff execution against a `429` you received is [concurrency-and-jobs.md](concurrency-and-jobs.md). Alerting on limiter trips is [observability.md](observability.md). Where timeout values come from is [deploy-and-config.md](deploy-and-config.md).
6
+
7
+ ## Operating Posture
8
+
9
+ Three failure modes, ranked. The first is worse because nothing about it looks wrong:
10
+
11
+ 1. **The hole that answers `200`.** Missing authorization, mass assignment, an IDOR. No exception, no error rate, no alert. Every test passes because every test is written by someone holding a valid session for the row they are asking about. This is OWASP Top 10 A01 and it is number one on frequency, not on drama.
12
+ 2. **The hole that hands over the machine.** Injection and SSRF. Rarer, because parameterised queries are the default in every maintained driver — but one of these is a database dump or a set of cloud role credentials, not a leaked row.
13
+ 3. **The control that is theatre.** A blocklist, a WAF rule, an escaping helper, a `X-XSS-Protection` header, an unguessable id. These are worse than nothing, because the review that would have found the real bug stops when it sees them.
14
+
15
+ **Never invent a security control.** Not a "sanitiser", not a regex that strips `../`, not a blocklist of bad words, not a rate limit you picked because it sounded generous. Every number and every rule below is from a published standard, a real default, or this repository's own code, and is checkable. A hand-rolled escaper is the same class of error as a hand-rolled KDF: the app works, the requests succeed, and the property you thought you had was never there.
16
+
17
+ **Producing nothing is often correct.** If the endpoint accepts no URL, spawns no process, renders no template, and deserialises nothing but JSON, do not add a "security layer" to it — add the authorization predicate and stop. Deleting a bespoke `escapeSql()` in favour of parameters is a better diff than any amount of new code. Adding CSRF tokens to a `Authorization: Bearer` API is cargo cult; say so and don't write it.
18
+
19
+ ## Injection
20
+
21
+ One rule underneath all seven rows: **user input is never part of the sentence, only ever a value slotted into it by something that knows the grammar.** Escaping is you trying to know the grammar. You don't.
22
+
23
+ | Class | The vulnerable shape | The fix | The fix that only LOOKS like one |
24
+ | --- | --- | --- | --- |
25
+ | **SQL** | `f"SELECT * FROM users WHERE email = '{email}'"`, or an ORM's `.raw()` / `text()` / `execute()` with an interpolated string | Placeholders (`$1`, `?`, `:name`) — the driver sends the query and the values in separate protocol fields, so no value can ever become syntax | `escapeSql()`, `addslashes`, `mysql_real_escape_string`, `replace("'", "''")`. All fail on a numeric context (`WHERE id = 5 OR 1=1` needs no quote at all) and on multibyte charset tricks. |
26
+ | **Identifier / ORDER BY** | `f"ORDER BY {col} {dir}"` — placeholders cannot bind a table, column, or direction | A literal allowlist map: `{"created": "created_at", "total": "total_cents"}[col]`, defaulting to reject. `dir` is `"ASC"` or `"DESC"`, chosen by an `if` | Quoting the identifier, or a regex like `^[a-zA-Z_]+$` — that is a valid column name from another table, and it is enough to sort-oracle a password hash. |
27
+ | **Command** | `exec(\`git log ${file}\`)` — Node's `exec` runs `/bin/sh -c`, so `;`, `` ` ``, `$( )`, and `&&` are live | `execFile('git', ['log', '--', file])` / `subprocess.run([...], shell=False)` — argv, no shell. `--` ends option parsing, so a file named `--upload-pack=…` stays a filename | Quoting the argument, or `shell: true` "just for the pipe". Also: passing argv but no `--` is still **argument injection** — the string becomes a flag, not a value. |
28
+ | **Path traversal** | `path.join(root, req.params.name)` then `fs.readFile` | `realpath` both sides, then compare with the separator (below). Better: never accept a path — accept an id and look the path up in the database | `replace("../", "")` (defeated by `....//`), decoding once (defeated by `%252e%252e%2f`), and `target.startsWith(root)` — which happily accepts `/srv/uploads-evil`. |
29
+ | **Template (SSTI)** | `render_template_string(f"Hello {name}")`, or any user string compiled as a template | User input is **data passed to** a pre-compiled template, never part of the template source | Escaping the output. SSTI is compile-time; `{{7*7}}` becomes `49` before any output filter runs, and in Jinja/Twig the object graph reaches the interpreter from there. |
30
+ | **Deserialization** | `pickle.loads`, `yaml.load` with `Loader=UnsafeLoader`, Java `ObjectInputStream`, PHP `unserialize`, `node-serialize` on anything from the wire | JSON only, then parse into a typed shape ([validation-and-errors.md](validation-and-errors.md)). `yaml.safe_load` when you must take YAML | A signature or an allowlist of classes bolted onto pickle. The Python docs say it plainly: pickle is not secure — construct the object yourself from parsed fields. |
31
+ | **Log injection** | `logger.info("login failed for " + email)` — a `\n` in `email` forges an entire log line, an ANSI escape rewrites the terminal, and both land in whatever parses your logs | Structured logging: the message is a constant, user values are **fields**. See [observability.md](observability.md) | Stripping `\n`. It misses `\r`, `
`, ANSI CSI, and every ingestion-side format you don't control. |
32
+ | **Header / response splitting** | `res.setHeader('Location', userUrl)`, `Set-Cookie` built from input | Never build a header value from raw input; for `Location`, resolve against an allowlist of paths or hosts | Trusting the runtime. Node *does* throw `ERR_INVALID_CHAR` on CR/LF in a header value — which means CRLF splitting is dead there and **open redirect is not**. `Location: https://evil.com` contains no illegal character and is a complete OAuth code-theft primitive ([auth-and-secrets.md](auth-and-secrets.md)). |
33
+
34
+ ```js
35
+ // Path containment, the only shape that holds. realpath FIRST — otherwise a
36
+ // symlink inside root points anywhere and every string check still passes.
37
+ const root = await fs.realpath('/srv/uploads');
38
+ const target = await fs.realpath(path.resolve(root, name)); // throws if absent: fine, that's a 404
39
+ if (target !== root && !target.startsWith(root + path.sep)) { // + sep, or "/srv/uploads-evil" passes
40
+ throw new ForbiddenPath(name);
41
+ }
42
+ ```
43
+
44
+ **Symptom, not rule:** the tell for SQL injection in review is not a missing placeholder — it is a *string* being built anywhere near a query. Grep for template literals and f-strings adjacent to `SELECT|INSERT|UPDATE|DELETE`, and for `.raw(`, `text(`, `execute(` taking a variable. An ORM does not make a codebase safe; it makes the one unsafe call harder to see.
45
+
46
+ ## SSRF
47
+
48
+ The hub's gate 5 states it: **any URL the user supplies is an SSRF until proven otherwise.** "The user" includes a webhook target, an avatar URL, an RSS feed, an OIDC discovery document, a PDF renderer's `<img src>`, and anything a model chose. Your server is inside the perimeter; the attacker is buying your network position with one string.
49
+
50
+ The prize is almost never an internal admin page. It is `http://169.254.169.254/latest/meta-data/iam/security-credentials/` — the cloud instance metadata service, one unauthenticated GET, role credentials in the response body. Enable IMDSv2 (`PUT /latest/api/token` with `X-aws-ec2-metadata-token-ttl-seconds`, hop limit 1, which alone kills SSRF-through-a-proxy) — and still implement everything below, because IMDSv2 protects one address and this protects your whole network.
51
+
52
+ The rule set, all of it, in order:
53
+
54
+ 1. **Allowlist the host when you can.** A webhook to a customer's domain cannot be allowlisted; an OIDC issuer, a payment provider, an internal service can. Exact host set from config, no regex, no suffix matching.
55
+ 2. **Resolve, then check every returned address, then connect.** Not the first address — *all* of them. The attacker controls the ordering of their own DNS answers.
56
+ 3. **Re-validate on every redirect hop.** This is the one that is always missing. The attacker serves a public, allowlisted page that replies `302 Location: http://127.0.0.1:8080/`, and a client that validated only the initial URL follows it obediently. Disable the HTTP library's own redirect following and drive the chain yourself.
57
+ 4. **`https:` only.** No `http:`, no `file:`, no `ftp:`, no `gopher:`, no `data:`. Every "just for local dev" `http:` exception is the exact hole.
58
+ 5. **Reject loopback, link-local, private, CGNAT, and every reserved range** — the table below, IPv6 included, including IPv4 wrapped inside IPv6.
59
+ 6. **No userinfo in the URL.** `https://api.trusted.example@127.0.0.1/` parses with a host of `127.0.0.1`, and reads to a human as the trusted host. Reject any URL with `username` or `password` set.
60
+ 7. **Cap redirects, cap response bytes, set a timeout.** An unbounded body and an infinite stream are denial of service with a `200`.
61
+
62
+ ```
63
+ Blocked v4: 0.0.0.0/8 10/8 127/8 100.64/10 (CGNAT) 169.254/16 (link-local + metadata)
64
+ 172.16/12 192.0.0/24 192.0.2/24 192.168/16 198.18/15 198.51.100/24
65
+ 203.0.113/24 224/4 and above (multicast, reserved, broadcast)
66
+ Blocked v6: :: ::1 fc00::/7 (ULA) fe80::/10 (link-local) ff00::/8 (multicast)
67
+ plus ::ffff:a.b.c.d and 64:ff9b::/96 (NAT64) unwrapped and re-checked as v4
68
+ Unparseable addresses are BLOCKED, not allowed. A string your parser gave up on is not proof of safety.
69
+ ```
70
+
71
+ Forms that defeat a naive check for the literal string `127.0.0.1`, every one of them routable: `2130706433` (decimal), `0x7f000001` (hex), `0177.0.0.1` (octal), `127.1` (`inet_aton` fills the middle), `[::ffff:127.0.0.1]`, `localhost.`, `anything.localhost`, and a DNS name the attacker points at `127.0.0.1` on a five-second TTL.
72
+
73
+ **This repository already contains the worked implementation: `packages/core/src/tools/web-fetch.ts`.** `validasiTarget()` does scheme, host, literal-IP, and post-DNS address checks, `ipv4Terlarang()` / `ipv6Terlarang()` / `parseIpv4Longgar()` / `expandIpv6()` are the range and parser logic including the octal/hex/short forms and the NAT64 and IPv4-mapped wrappers, and the fetch loop calls `validasiTarget()` again **on every redirect target** with `maxRedirections: 0` on the underlying client so no library can follow a hop behind its back. Its limits are `MAKS_REDIRECT = 5`, `MAKS_BYTE = 5 MiB` (truncate, don't fail), `TIMEOUT_MS = 30_000` covering the whole chain — 30s because that call sits inside an agent turn, not inside an HTTP handler. **Read that file before writing SSRF defence; re-deriving these rules from memory is how one of them goes missing.** It also documents the residual hole honestly: it validates resolved addresses then connects by hostname, leaving a DNS-rebinding window that closes only by connecting to the pinned IP with an explicit `Host` header — which undici cannot do without breaking SNI. Copy the honesty too: name the gap you accepted.
74
+
75
+ Inside a request handler the timeouts are not 30s. **Your outbound timeout must be strictly less than your own handler's, which must be less than your caller's** — the hub forbids "a timeout longer than its caller's". Derive them, don't pick them: a 10s inbound budget with two sequential upstream calls means 4s each, connect timeout 2s, and the arithmetic goes in the config file next to the values ([deploy-and-config.md](deploy-and-config.md)).
76
+
77
+ ## CORS
78
+
79
+ **CORS is not authorization.** It is a browser-enforced relaxation of the same-origin policy that decides whether *JavaScript on another origin may read your response*. It does not stop the request from being sent, it does not stop it from taking effect, and it means nothing at all to `curl`, a script, or a mobile app. Deleting a CORS header does not fix a vulnerability; it hides it from one class of client.
80
+
81
+ The account takeover, concretely: you reflect the incoming `Origin` and set `Access-Control-Allow-Credentials: true`. The victim visits `evil.example`, whose page runs `fetch('https://api.example.com/me', {credentials: 'include'})`. The browser attaches the session cookie, your server echoes `Access-Control-Allow-Origin: https://evil.example`, and the attacker's script reads the authenticated response — profile, email, and the CSRF token it needs to then perform writes. **Reflection is a wildcard with extra steps.** People arrive at it because browsers reject a literal `*` when credentials are true; the correct response to that rejection is an allowlist, not a mirror.
82
+
83
+ ```
84
+ Origin: https://app.example.com
85
+ → look up in an exact string Set from config. Not a regex, not endsWith.
86
+ "endsWith('example.com')" matches https://evilexample.com.
87
+ /app\.example\.com/ unanchored matches https://app.example.com.evil.io.
88
+ Access-Control-Allow-Origin: https://app.example.com # echo the MATCHED entry, never the header
89
+ Access-Control-Allow-Credentials: true # only if the API uses cookies at all
90
+ Vary: Origin # or a shared cache serves tenant A's ACAO to tenant B
91
+ Access-Control-Allow-Methods: GET, POST, PATCH, DELETE
92
+ Access-Control-Allow-Headers: content-type, authorization
93
+ Access-Control-Max-Age: 600 # Chromium caps preflight cache at 7200s, Firefox 86400;
94
+ # 600 keeps a policy change one redeploy away
95
+ ```
96
+
97
+ Never allowlist the literal origin `null` — sandboxed iframes, `data:` URLs, and local files all send it, so it is an origin any attacker can obtain. `Access-Control-Allow-Origin: *` is correct and safe for a genuinely public, read-only, credential-free API, and nowhere else.
98
+
99
+ ## CSRF
100
+
101
+ The question is one line: **does the browser attach the credential by itself?**
102
+
103
+ | Auth mechanism | CSRF applies | Because |
104
+ | --- | --- | --- |
105
+ | Session cookie | **Yes** | The browser sends it on a cross-site form POST without the attacker ever reading it |
106
+ | HTTP Basic, TLS client cert, Windows/NTLM auth | **Yes** | Same — ambient, automatic |
107
+ | `Authorization: Bearer …` from JS memory | **No** | A cross-site page cannot set that header on your origin; a preflight would fail. Adding CSRF tokens here is noise |
108
+ | A cookie holding a JWT | **Yes** | The storage decides, not the token format |
109
+
110
+ Primary defence, and the cookie line is [auth-and-secrets.md](auth-and-secrets.md)'s, restated unchanged:
111
+
112
+ ```
113
+ Set-Cookie: __Host-sid=<256 bits of CSPRNG, base64url>; HttpOnly; Secure; SameSite=Lax; Path=/; Max-Age=1209600
114
+ ```
115
+
116
+ Set `SameSite` explicitly — Chrome defaults an unset cookie to Lax, other browsers do not, and "the default protects us" is a claim about a browser matrix you have not tested. Two gaps `Lax` does not close: it still rides along on **top-level GET navigation**, so a state-changing GET is CSRF-able no matter what (the hub's doctrine 1 already forbids that GET); and *site* means eTLD+1, so a compromised or attacker-registered subdomain is same-site and unaffected.
117
+
118
+ Backstop, when the surface is worth it (payments, admin, account changes):
119
+
120
+ - **Synchroniser token** — random per session, stored server-side, sent in a form field or a custom header, compared with `timingSafeEqual`. The default; it has no subdomain caveat.
121
+ - **Double-submit** — cookie plus matching header. Only in its **signed** form: bind the token to the session with an HMAC, because a subdomain can write a cookie onto your apex and naive double-submit then validates the attacker's own value against itself.
122
+ - **`Origin` check** — reject any state-changing request whose `Origin` is present and not in the CORS allowlist. Cheap, and it costs one `if`.
123
+ - **`Sec-Fetch-Site: cross-site` → reject** on state-changing endpoints. Sent by Chrome, Edge, Firefox, and Safari 16.4+; a browser that omits it falls through to the token.
124
+
125
+ ## Rate Limiting
126
+
127
+ | Algorithm | Shape | Trade-off |
128
+ | --- | --- | --- |
129
+ | **Fixed window** | `INCR key:{id}:{minute}` + `EXPIRE`. One round trip | Cheapest, and wrong at the seam: 100/min allows 200 in the two seconds spanning a boundary |
130
+ | **Sliding window log** | Sorted set of request timestamps, trim then count | Exact, and memory grows with traffic per key — an attacker chooses your memory bill |
131
+ | **Sliding window counter** | Weighted blend of previous and current window counters | Two counters, error under a few percent, no boundary burst. The pragmatic default for per-IP limits |
132
+ | **Token bucket / GCRA** | Rate plus burst; one level and one timestamp per key | Allows a *deliberate* burst then smooths — matches how real clients behave (page load fires eight calls). The default for per-key API quotas |
133
+ | **Leaky bucket (queue)** | Fixed drain rate, requests queue | Shapes rather than rejects; use for outbound calls to a rate-limited third party, never for inbound — a queue under attack is just a slower outage |
134
+
135
+ **Limit on several keys at once; one dimension is never enough.**
136
+
137
+ - **Per API key / user** — the quota, the fairness mechanism, the thing you sell.
138
+ - **Per IP** — the only handle on unauthenticated traffic. For IPv6 bucket by **/64, not /128**: a single host is routinely handed a /64 or larger, so per-address limiting counts an attacker with 18 quintillion free identities.
139
+ - **Per expensive endpoint** — search, export, report generation, anything that fans out or renders. A global 1000/min is meaningless when one of those requests costs 4 CPU-seconds; give the expensive route its own much smaller bucket.
140
+ - **Global concurrency** — a request-rate limiter still permits unbounded *concurrent* slow queries. Cap in-flight work as well ([concurrency-and-jobs.md](concurrency-and-jobs.md)).
141
+
142
+ The response contract, per the hub's status table:
143
+
144
+ ```
145
+ HTTP/1.1 429 Too Many Requests
146
+ Retry-After: 30 # RFC 9110 §10.2.3 — delta-seconds or an HTTP-date. The ONLY standard field here.
147
+ Content-Type: application/problem+json
148
+ ```
149
+
150
+ `X-RateLimit-*` is convention, and the IETF `RateLimit` header draft has changed shape between revisions — so pick one form, document it in the API reference, and never let a client infer it. Do not answer `403`, and never `200` with an error body. Your own clients must honour `Retry-After` and add jitter ([concurrency-and-jobs.md](concurrency-and-jobs.md)); trips get a metric, not a log line per request ([observability.md](observability.md)).
151
+
152
+ **Authentication endpoints get their own, much tighter limits, keyed on the account.** A distributed credential-stuffing run rotates IPs by design, so an IP-only limiter sees a hundred well-behaved clients. NIST SP 800-63B §5.2.2 sets the outer bound you may not exceed: **no more than 100 consecutive failed attempts per account.** Be far stricter, and count *failures*, resetting on success:
153
+
154
+ ```
155
+ POST /login 5 failed attempts / 15 min per account + 20 / 15 min per IP + exponential backoff after
156
+ POST /password/reset 3 / hour per account + 10 / hour per IP # every one sends an email you pay for
157
+ POST /token (refresh) keyed per token family, not per IP # a spike here is theft — see auth-and-secrets.md
158
+ ```
159
+
160
+ Why 5: someone who forgot their password needs two or three tries and is never inconvenienced; a stuffing bot needs thousands and is stopped at the door. And the lockout response must be **identical** to a normal failed login — a distinct "account locked" message is the user-enumeration oracle [auth-and-secrets.md](auth-and-secrets.md) spends a section closing.
161
+
162
+ ## Authorization Bug Classes
163
+
164
+ The fix for all four is the same one sentence — **the actor is an argument to the query, and it comes from the session** ([auth-and-secrets.md](auth-and-secrets.md), which owns it). What differs is how each one hides.
165
+
166
+ **IDOR** — the id in the URL is the only thing selecting the row. Owned in full by [auth-and-secrets.md](auth-and-secrets.md); the detection angle here: it is invisible in tests seeded with one tenant, so the regression test is *tenant B asks for tenant A's id and gets a `404`*.
167
+
168
+ **Mass assignment** — the request body is handed to the model:
169
+
170
+ ```py
171
+ # The bug. Nothing here mentions is_admin; that is the point.
172
+ user.update(**request.json) # {"name": "x", "is_admin": true} → is_admin = true
173
+
174
+ # The fix: an allowlist, and it lives in the parse step, not the update step.
175
+ data = UserPatch(**request.json) # unknown fields REJECTED — validation-and-errors.md
176
+ user.update(name=data.name)
177
+ ```
178
+ The fake fix is a denylist — `{k: v for k, v in body.items() if k != "is_admin"}` — which is correct until someone adds `role_id`, `tenant_id`, `credit_balance_cents`, or `email_verified` next quarter and nobody edits the filter. **Symptom:** the request schema and the database model are the same class.
179
+
180
+ **Missing function-level authorization** — the endpoint nothing links to. `DELETE /admin/users/:id` guarded by the button not being rendered; the v2 route added beside a v1 route that had the decorator; the GraphQL resolver; the CSV export; the cron-triggered internal handler. Make the router default-deny and **enumerate it in a test**: walk every registered route, assert each declares a policy, and fail CI on the first one that doesn't. A new endpoint should be unreachable until someone names its rule — a checklist item is a rule that gets forgotten, a failing test is one that can't be.
181
+
182
+ **Confused deputy** — something with more authority than the caller acts on the caller's behalf. The three real shapes: a service that trusts `X-User-Id` from the gateway while its pod is directly reachable on the cluster network; a background job that runs as a superuser and takes `user_id` from a queue message it never authorises; a webhook whose signature you verify but whose *payload target* you don't, so a valid signature moves someone else's money. The deputy must carry the **caller's** authority — a scoped token verified at the destination — and the edge must strip every internal header unconditionally on the way in, whether or not it looks set. **Symptom:** the phrase "it's only reachable from inside the network" appearing in a code comment.
183
+
184
+ ## Comparing Secrets
185
+
186
+ `==` on bytes returns at the first differing byte. Across a network that difference is nanoseconds buried in noise, and it is still an oracle: average enough samples and the attacker recovers a token one byte at a time in roughly `256 × length` batches of requests. Rate limiting does not fix it, because the signal survives averaging — it only makes the attack take longer. Use `crypto.timingSafeEqual` / `hmac.compare_digest` / `subtle.ConstantTimeCompare`; the exact per-language calls and the full list of what counts as a secret are in [auth-and-secrets.md](auth-and-secrets.md).
187
+
188
+ Two things that survive the switch and still leak: **length**, since `timingSafeEqual` throws on a mismatch — so compare fixed-length SHA-256 digests of both sides, not the raw strings; and **the lookup**, since fetching every API key and comparing in a loop makes the loop's trip count proportional to how early you matched. Look the key up by an indexed hash, then constant-time compare the one row you got.
189
+
190
+ ## Security Headers on a JSON API
191
+
192
+ | Header | Verdict on a JSON API |
193
+ | --- | --- |
194
+ | `X-Content-Type-Options: nosniff` | **Matters.** Stops a browser sniffing your JSON as HTML and executing it — the classic path from a reflected value to XSS on your API origin |
195
+ | `Strict-Transport-Security: max-age=31536000; includeSubDomains` | **Matters** (RFC 6797). Add `preload` only if you accept that every current and future subdomain is HTTPS-only, permanently |
196
+ | `Cache-Control: no-store` on authenticated responses | **Matters.** Otherwise a shared proxy or the browser disk cache keeps another user's data |
197
+ | `Content-Disposition: attachment` on user-uploaded downloads | **Matters.** With `nosniff`, it is what stops an uploaded `.html` from executing on your origin |
198
+ | `Content-Security-Policy: default-src 'none'; frame-ancestors 'none'` | **Cheap belt.** Pure JSON executes nothing, but your framework's error page and any docs route do render HTML. Real CSP work belongs on the HTML surface |
199
+ | `X-Frame-Options` / `frame-ancestors` | **Theatre on JSON** — framing a JSON response reveals nothing. Mandatory on any HTML you serve |
200
+ | `X-XSS-Protection: 1; mode=block` | **Theatre, and historically a vulnerability of its own.** Chrome removed the XSS Auditor in version 78. Omit it, or send `0` |
201
+ | `Referrer-Policy`, `Permissions-Policy` | **Theatre on JSON**, correct on HTML pages |
202
+ | `Server`, `X-Powered-By` | Not a control — remove them anyway; version disclosure is free reconnaissance |
203
+
204
+ ## Triage
205
+
206
+ | Suspicion | The first thing to check |
207
+ | --- | --- |
208
+ | A user saw another tenant's data | The repository/query layer, for a call without the actor argument. The route guard is irrelevant — read the `WHERE` |
209
+ | A field changed that no form exposes | Mass assignment: find the update path, look for a whole-body spread or `**request.json` |
210
+ | Outbound traffic to `169.254.169.254` or a private range | Every handler that accepts a URL, then the redirect follower: is validation re-run per hop, and are redirects disabled in the HTTP client? |
211
+ | Rows from a table the endpoint never queries | String-built SQL. Grep f-strings and template literals near `SELECT\|INSERT\|UPDATE\|DELETE`, and `.raw(` / `text(` / `execute(` with a variable |
212
+ | A file read outside the upload directory | Traversal: is there a `realpath` on **both** sides, and is the prefix check `root + sep`? |
213
+ | A shell command with a user-supplied filename | `shell: true`, and a missing `--` before the operands |
214
+ | Actions users deny performing, cookie session | CSRF: `SameSite` on the session cookie, and whether the endpoint is a state-changing GET |
215
+ | Another site is reading authenticated responses | CORS: does `Access-Control-Allow-Origin` echo the request's `Origin`, and is `Allow-Credentials` true? |
216
+ | Login endpoint under sustained load | Does the limiter key on the **account** as well as the IP, and does it count failures rather than requests? |
217
+ | An internal-only endpoint hit from outside | Does the edge strip `X-User-Id`-style headers unconditionally, or only when it doesn't set them? |
218
+ | Stack traces, SQL, or internal ids in a response | Not this file — [validation-and-errors.md](validation-and-errors.md) |
219
+ | A credential appeared in a repo or a log | Not this file — [auth-and-secrets.md](auth-and-secrets.md), rotate first |
220
+
221
+ ## Never Ship
222
+
223
+ | Never | Instead |
224
+ | --- | --- |
225
+ | A query built by string concatenation or interpolation | Placeholders; an allowlist map for identifiers and sort direction |
226
+ | An escaping helper standing in for parameterisation | Delete it; the driver already knows the grammar |
227
+ | `exec` with an interpolated command, or `shell: true` | `execFile` / argv, with `--` before user operands |
228
+ | `startsWith(root)`, or stripping `../` | `realpath` both sides, compare against `root + sep` |
229
+ | `pickle.loads`, `yaml.load(…, UnsafeLoader)`, `unserialize` on wire data | JSON, parsed into a typed shape |
230
+ | User input concatenated into a log message | Structured fields, constant message |
231
+ | A user-supplied URL fetched without validation | Allowlist, resolve-then-check-all-addresses, https only, no userinfo, revalidate every hop, cap redirects/bytes/time |
232
+ | Redirect following left to the HTTP library | Disable it; drive the chain and re-validate each target |
233
+ | A blocklist of `127.0.0.1` / `localhost` strings | Range checks over parsed addresses, with unparseable treated as blocked |
234
+ | `Access-Control-Allow-Origin` echoing the `Origin` header | Exact-match Set, echo the matched entry, `Vary: Origin` |
235
+ | An origin allowlist by regex, `endsWith`, or `null` | Exact string membership |
236
+ | A cookie-session endpoint with no CSRF defence | `SameSite=Lax` plus a synchroniser token on the sensitive routes |
237
+ | CSRF tokens on a `Bearer`-authenticated API | Nothing; the browser cannot forge that header |
238
+ | A state-changing `GET` | `POST` / `PATCH` / `DELETE` — `SameSite=Lax` cannot protect a GET |
239
+ | Rate limiting on IP alone, or per-IPv6-/128 | Per key **and** per /64 **and** per expensive endpoint |
240
+ | Login limited at the same rate as everything else | 5 failures / 15 min per account, well under NIST's 100-consecutive ceiling |
241
+ | A distinct "account locked" response | One identical failure response |
242
+ | `429` without `Retry-After`, or `403` for a rate limit | `429` + `Retry-After` (RFC 9110 §10.2.3) |
243
+ | A request body spread into a model | A parsed DTO with an explicit field allowlist |
244
+ | A route whose authorization is "no UI links to it" | Default-deny router plus a test that enumerates every route |
245
+ | An internal header trusted because the network is private | Strip at the edge; the deputy carries the caller's token |
246
+ | `==` on a token, signature, or reset code | Constant-time compare of equal-length digests |
247
+ | `X-XSS-Protection: 1; mode=block` as a control | `nosniff`, HSTS, `no-store` — and CSP where HTML is served |
248
+
249
+ ## Output
250
+
251
+ Write the code. Then, in a few lines: **what the untrusted input was and where it stopped being untrusted** (the parameterised call, the allowlist, the validated URL); **which of the boundary gates this touched** and whether any is still open; and, for anything network-facing, **the exact limits you set** — timeout, redirect cap, byte cap, rate limit — with the sentence that justifies each. If you refused part of the request — a blocklist someone asked for, an `Origin` reflection, an `http:` exception for local development — say which and why in one line. If the honest answer is that this endpoint needs no new security code, only an authorization predicate, give that answer and stop.
@@ -0,0 +1,238 @@
1
+ # Go
2
+
3
+ This file owns what is true about Go and false elsewhere: `context.Context` propagation, `net/http` server and client configuration, `database/sql` pool knobs, error wrapping, goroutine lifetimes, `log/slog`, struct tags, and routing. Every *value* it names is restated from a sibling, never re-derived — timeout budgets and the shutdown order are [deploy-and-config.md](deploy-and-config.md), the pool-size formula is [testing-and-perf.md](testing-and-perf.md) §8, the log field set is [observability.md](observability.md), the error taxonomy and retryability rules are [validation-and-errors.md](validation-and-errors.md), and locks, outbox, and job semantics are [concurrency-and-jobs.md](concurrency-and-jobs.md). Status codes and the response envelope stay in [api-contracts.md](api-contracts.md).
4
+
5
+ **This is deliberately the shortest stack file.** Go's culture is stdlib-first: there is no dominant framework whose conventions must be encoded, and inventing one — a house `Repository[T]`, a bespoke middleware chain, "our" error package — would be worse than omitting it. What follows is only the places where the language's *zero values* and the stdlib's *defaults* are wrong for a server.
6
+
7
+ ## Operating Posture
8
+
9
+ Two failure modes, and the first is worse:
10
+
11
+ 1. **The zero value that compiles.** `http.Server{}` has no timeouts, `http.Client{}` has no timeout, `sql.DB` has no connection cap, `SetConnMaxLifetime` is forever. None of these are errors, none fail a test, and all of them work perfectly on the first day. They surface as a server holding ten thousand half-open sockets, or as `FATAL: sorry, too many clients already` at the first traffic spike.
12
+ 2. **The goroutine that outlives its reason to exist.** It does not panic, does not log, and does not appear in any metric you have. Memory climbs across a week and the graph looks like a slow leak in the allocator.
13
+
14
+ **Never invent a Go number.** Not `SetMaxOpenConns(100)` because it is round, not `ReadTimeout: 30 * time.Second` copied from a blog post that did not say `ReadTimeout` also covers the request body, not `runtime.GOMAXPROCS` tuning without a profile. And never write `context.Background()` inside a handler because a function needed a ctx and the linter stopped complaining — that line severs cancellation for everything below it.
15
+
16
+ **Permission to add nothing.** If `net/http` + `database/sql` + `log/slog` already do the job, add no framework, no DI container, no ORM, and no logging wrapper. The best Go diff of the week is frequently the one that deletes a hand-rolled router in favour of Go 1.22's `ServeMux`.
17
+
18
+ ## 1. context.Context — First Parameter, Always
19
+
20
+ The habit that buys the most: `ctx` is the first parameter of every function that does I/O, named `ctx`, and it is **never stored in a struct field** (the `context` package says so explicitly; the one legitimate carrier is `*http.Request`, so read it back with `r.Context()`).
21
+
22
+ | Call | Correct form | What the non-ctx form actually does |
23
+ | --- | --- | --- |
24
+ | SQL | `QueryContext` / `ExecContext` / `QueryRowContext` | `Query` is literally `QueryContext(context.Background(), …)` — uncancellable |
25
+ | Transaction | `db.BeginTx(ctx, nil)` | `db.Begin()` — a transaction that survives the client hanging up |
26
+ | Outbound HTTP | `http.NewRequestWithContext(ctx, …)` | `http.NewRequest` — no deadline, no cancellation, ever |
27
+ | Per-call budget | `context.WithTimeout(ctx, d)` + `defer cancel()` | A missing `cancel()` leaks the context and its timer until the deadline fires; `go vet`'s `lostcancel` catches it |
28
+ | Fire-and-forget from a handler | `context.WithoutCancel(r.Context())` (Go 1.21+), then a fresh timeout | `go work(r.Context())` is cancelled the instant the handler returns |
29
+
30
+ ```go
31
+ func (s *Service) Charge(ctx context.Context, id string) error {
32
+ ctx, cancel := context.WithTimeout(ctx, 5*time.Second) // third-party budget, deploy-and-config.md §5
33
+ defer cancel() // frees the timer immediately on the fast path
34
+ ...
35
+ }
36
+ ```
37
+
38
+ What cancellation buys, concretely: the client gives up at 30s and retries; without a propagated ctx your handler keeps a pool connection and a Postgres backend busy for the full `statement_timeout` on a request nobody is reading. The symptom is diagnostic — `pg_stat_activity` full of `active` queries whose client sockets are gone, and a pool that empties under retry load rather than under real load.
39
+
40
+ **`context.Canceled` is not an error you own.** `errors.Is(err, context.Canceled)` means the caller went away: log at `INFO`, do not increment the error metric, do not page. `context.DeadlineExceeded` is yours. Conflating them fills the error dashboard with users who pressed Stop.
41
+
42
+ `ctx.Value` is for request-scoped values that cross API boundaries — request id, trace id, auth principal — keyed by an **unexported struct type** so no other package can collide or read it. Never for optional arguments.
43
+
44
+ ## 2. net/http Server Timeouts — the Zero Value Has None
45
+
46
+ This is the most common Go production bug. `http.ListenAndServe(":8080", mux)` constructs a `http.Server` whose `ReadTimeout`, `ReadHeaderTimeout`, `WriteTimeout`, and `IdleTimeout` are all zero, and **zero means no limit**. One client that opens a TCP connection and dribbles one header byte per minute holds a goroutine and a file descriptor indefinitely. That is Slowloris, it needs no volume, and it costs the attacker nothing.
47
+
48
+ ```go
49
+ srv := &http.Server{
50
+ Addr: ":8080",
51
+ Handler: mux,
52
+ ReadHeaderTimeout: 5 * time.Second, // Slowloris cutoff: full header set or the conn dies
53
+ ReadTimeout: 10 * time.Second, // headers + BODY; must exceed your largest legitimate upload
54
+ WriteTimeout: 15 * time.Second, // > 10s handler budget (deploy-and-config.md §5) + write time
55
+ IdleTimeout: 120 * time.Second, // keep-alive reuse window; longer than any LB idle timeout
56
+ MaxHeaderBytes: 1 << 20, // 1 MiB — same as the default, set explicitly so it is reviewable
57
+ ErrorLog: slog.NewLogLogger(handler, slog.LevelWarn), // else server errors go to the std logger
58
+ }
59
+ log.Fatal(srv.ListenAndServe())
60
+ ```
61
+
62
+ Behaviour you must know before changing those numbers:
63
+
64
+ - **`IdleTimeout` falls back to `ReadTimeout` when zero**, and if both are zero there is no idle timeout at all. Setting only `ReadTimeout` therefore silently sets your keep-alive window to the same value and kills connection reuse.
65
+ - **`ReadTimeout` covers the body**, so a 10s read timeout caps upload duration, not upload size. For one upload route, do not raise the global value — extend that route with `http.NewResponseController(w).SetReadDeadline(...)` (Go 1.20+).
66
+ - **`WriteTimeout` starts when the request header is read**, so it bounds the handler's entire execution plus the response write. A `WriteTimeout` shorter than the handler budget truncates the response mid-body, and the client sees a broken connection rather than a status code.
67
+ - **Streaming endpoints (SSE, long polls) need `WriteTimeout` handled per-route** with `SetWriteDeadline`, not a global zero. A global zero to make SSE work reopens Slowloris for every other route.
68
+
69
+ The client side is the mirror image:
70
+
71
+ | Default | Reality | Set it to |
72
+ | --- | --- | --- |
73
+ | `http.Client{}.Timeout` | `0` — no limit, no matter what the transport does | A total budget, plus `NewRequestWithContext` |
74
+ | `http.DefaultTransport.MaxIdleConnsPerHost` | `2` (`DefaultMaxIdleConnsPerHost`) | Raise for a hot upstream; at 2 you pay a fresh TCP + TLS handshake per concurrent call and it reads as "the upstream is slow" |
75
+ | `http.DefaultTransport.IdleConnTimeout` | 90s | Fine; keep it under the upstream's idle close |
76
+ | Reusing `http.DefaultClient` for a third party | No timeout, shared transport, shared connection pool | One named `*http.Client` per upstream, constructed once, never per request |
77
+
78
+ **Never construct a `*http.Client` inside a handler.** Each one gets its own transport and its own connection pool, so nothing is ever reused and the fd count tracks request rate.
79
+
80
+ ## 3. database/sql Pool
81
+
82
+ `sql.DB` is a pool, not a connection, and `sql.Open` does not connect — it validates the DSN. Verify with `db.PingContext(ctx)` during boot ([deploy-and-config.md](deploy-and-config.md) §3) so a wrong password fails the rollout rather than the first request.
83
+
84
+ ```go
85
+ db.SetMaxOpenConns(9) // ceiling from testing-and-perf.md §8 — never a round number
86
+ db.SetMaxIdleConns(9) // == MaxOpenConns; the default of 2 churns the other 7
87
+ db.SetConnMaxLifetime(5 * time.Minute) // under PgBouncer/RDS idle reaping and under DNS-failover TTL
88
+ db.SetConnMaxIdleTime(1 * time.Minute) // Go 1.15+; returns idle capacity during quiet periods
89
+ ```
90
+
91
+ | Knob | Default | Symptom of leaving it |
92
+ | --- | --- | --- |
93
+ | `SetMaxOpenConns` | `0` = **unlimited** | A burst opens connections until Postgres answers `FATAL: sorry, too many clients already` (SQLSTATE `53300`) — and every other service sharing that database fails at the same moment |
94
+ | `SetMaxIdleConns` | `2` | With `MaxOpen` 9, seven connections are closed and reopened continuously: a latency floor no query tuning removes, and a connection-open rate that tracks traffic |
95
+ | `SetConnMaxLifetime` | `0` = forever | Connections survive a failover; the app keeps talking to a demoted replica, or a proxy silently drops a conn the pool still believes is live (`unexpected EOF` on a random query) |
96
+
97
+ Two loop rules that produce silent wrong answers rather than errors:
98
+
99
+ - **`defer rows.Close()` on every `QueryContext`.** A leaked `*sql.Rows` holds its connection until the GC finalizer runs. The symptom is pool exhaustion that a restart fixes and no query is slow.
100
+ - **Check `rows.Err()` after the `for rows.Next()` loop.** A mid-stream network failure ends `Next()` exactly like end-of-results, so an unchecked loop serves a truncated page with `200 OK`.
101
+
102
+ Pool sizing itself, the `max_connections` ceiling, and the multi-consumer arithmetic are [testing-and-perf.md](testing-and-perf.md) §8 — do not name a Go pool size without it.
103
+
104
+ ## 4. Errors
105
+
106
+ Wrap with `%w`, once, at each boundary that adds context. `%v` formats the error into a string and **severs the chain**: the caller's `errors.Is(err, sql.ErrNoRows)` returns false, the handler returns `500` for a row that is merely absent, and nothing in the log says why.
107
+
108
+ | Situation | Form |
109
+ | --- | --- |
110
+ | A condition callers must branch on | Sentinel: `var ErrNotFound = errors.New("not found")`, compared with `errors.Is` |
111
+ | A failure carrying data (field name, retry-after, driver code) | Typed: `type ValidationError struct{…}` with `Error() string`, extracted by `errors.As` |
112
+ | A driver condition (unique violation `23505`, serialization failure `40001`) | `errors.As(err, &pgErr)` then switch on `pgErr.Code` — see [concurrency-and-jobs.md](concurrency-and-jobs.md) |
113
+ | Several independent failures from one operation | `errors.Join` (Go 1.20+) |
114
+ | Adding context on the way up | `fmt.Errorf("charge order %s: %w", id, err)` — no `"failed to"` prefix; six of them stacked is noise |
115
+
116
+ **Forbidden by name:** `strings.Contains(err.Error(), "duplicate key")`, `err.Error() == "sql: no rows in result set"`, and any retry decision made on a message string. The message is not an API; the driver changes it in a point release and your retry logic quietly becomes a `500` handler. Classification rules live in [validation-and-errors.md](validation-and-errors.md).
117
+
118
+ `panic` is for programmer error only. Recover in **one** middleware that logs the stack and returns `500` — never per handler, and never to convert an expected failure into control flow. A recovered panic that is not logged is an incident you will never hear about.
119
+
120
+ ## 5. Concurrency
121
+
122
+ **A goroutine started per request needs a lifetime bound to that request.** `go audit(r.Context())` is a bug in both directions: the request context is cancelled the moment the handler returns, so fire-and-forget work dies instantly; and a goroutine given `context.Background()` outlives the deploy. Use `context.WithoutCancel(r.Context())` (Go 1.21+) to keep the trace id while dropping the cancellation, then attach a fresh deadline. If the work must survive the process, it is not a goroutine — it is a job ([concurrency-and-jobs.md](concurrency-and-jobs.md)).
123
+
124
+ ```go
125
+ ch := make(chan result) // BUG: unbuffered
126
+ go func() { ch <- work() }() // blocks forever once the select below takes ctx.Done()
127
+ select {
128
+ case r := <-ch:
129
+ return r, nil
130
+ case <-ctx.Done(): // handler returns; that goroutine, its stack, and everything
131
+ return result{}, ctx.Err() // it references are unreachable and immortal
132
+ }
133
+ ```
134
+
135
+ Fix: `make(chan result, 1)`. The send then completes into the buffer whether or not anyone is left to read it. This exact shape — a timeout on the reader, an unbuffered channel on the writer — is the most common goroutine leak in Go services, and its only symptom is RSS climbing on a graph nobody is watching.
136
+
137
+ - **Bounded fan-out is `errgroup`**, not a hand-rolled semaphore: `g, ctx := errgroup.WithContext(ctx)` then `g.SetLimit(8)`. The first non-nil error cancels the derived ctx, so the remaining calls stop instead of finishing work nobody will read. An unbounded `for … { go … }` over a user-supplied list is a denial-of-service on your own upstream.
138
+ - **WaitGroup misuse**, ranked: `wg.Add(1)` *inside* the goroutine (`Wait` can return before it runs — the whole point of the counter is lost); a path that returns before `wg.Done()` (deadlock at `Wait`, which reads as a hang, not a panic); and reusing a `WaitGroup` across rounds without letting `Wait` return.
139
+ - **Loop-variable capture** is fixed per-iteration only for modules whose `go.mod` declares `go 1.22` or later. In a module still on `go 1.21`, `for _, u := range users { go send(u) }` sends the last user N times — and the code looks identical either way. Check the `go` directive before believing it is safe.
140
+ - `go test -race` reports only races it actually observes at runtime; a clean run on a single-threaded test proves nothing about a fan-out path. Test methodology is `kerja-agentik`'s.
141
+
142
+ ## 6. Shutdown
143
+
144
+ Ordering — fail readiness, keep serving 5s, then stop accepting, then drain — is [deploy-and-config.md](deploy-and-config.md) §4 and is not repeated here. The Go mechanics:
145
+
146
+ ```go
147
+ ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
148
+ defer stop()
149
+ <-ctx.Done()
150
+
151
+ shutdownCtx, cancel := context.WithTimeout(context.Background(), 25*time.Second) // drain_deadline, §4
152
+ defer cancel()
153
+ if err := srv.Shutdown(shutdownCtx); err != nil { /* deadline hit: log, then exit non-zero */ }
154
+ ```
155
+
156
+ - **`ListenAndServe` returns `http.ErrServerClosed` on a clean shutdown.** Treating that return as fatal makes every successful deploy exit non-zero and page someone. Compare with `errors.Is(err, http.ErrServerClosed)` and exit 0.
157
+ - **`Shutdown` does not wait for hijacked connections** — WebSockets and anything that took over the conn. Track those with your own `sync.WaitGroup` and close them yourself, or the drain reports success while sockets are still live.
158
+ - **`Shutdown` knows nothing about your background goroutines.** Queue consumers, tickers, and flushers need their own stop signal; `srv.Shutdown` returning is not "the process is idle".
159
+
160
+ ## 7. log/slog
161
+
162
+ `log/slog` is stdlib since Go 1.21. Use it; do not wrap it in a house logging package. The field names, levels, and redaction rules are [observability.md](observability.md)'s — slog is only the mechanism.
163
+
164
+ ```go
165
+ h := slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelInfo})
166
+ slog.SetDefault(slog.New(h))
167
+ log := slog.Default().With(slog.String("request_id", rid), slog.String("route", pattern))
168
+ log.InfoContext(ctx, "order rejected", slog.String("error_code", "order_currency_mismatch"))
169
+ ```
170
+
171
+ - **Use the `Context` variants** (`InfoContext`, `ErrorContext`) so a handler can pull trace ids out of `ctx` — otherwise the join key has to be threaded by hand, and it will be dropped in the one helper that mattered.
172
+ - **Prefer typed attrs** (`slog.String`, `slog.Int`) over alternating key/value: an odd argument count emits `!BADKEY` **at runtime**, not at compile time, and it survives review because the line looks fine.
173
+ - **`With` once per request**, not per call site. Rebuilding the same attrs on every line is both slower and the reason field names drift.
174
+
175
+ ## 8. Struct Tags and the Zero-Value Ambiguity
176
+
177
+ **Always write the JSON tag explicitly.** Without one, Go marshals the exact Go field name, so renaming `OrderId` to `OrderID` in a refactor renames the wire field — a breaking API change with no compile error anywhere in the repo or the tests.
178
+
179
+ The real trap: **a missing field and a false are indistinguishable.** `json.Unmarshal` leaves an absent field at its zero value, so `{"active": false}`, `{}`, and `{"active": null}` all produce `Active == false`. With `omitempty` on the way out, `false`, `0`, and `""` disappear from the response entirely. On a `PATCH` this means "set this to false" and "do not touch this" are the same bytes.
180
+
181
+ | Need | Answer |
182
+ | --- | --- |
183
+ | Distinguish absent from `false` / `0` / `""` on a PATCH | `*bool` / `*int64` — `nil` is absent, non-nil is an intentional value |
184
+ | Nullable database column | `sql.Null[T]` (Go 1.22+ generic) or `sql.NullString` etc. |
185
+ | A field to be echoed back untouched | `json.RawMessage` |
186
+ | Reject unknown fields | `dec := json.NewDecoder(r.Body); dec.DisallowUnknownFields()` — restated from [validation-and-errors.md](validation-and-errors.md) |
187
+ | Cap the body | `r.Body = http.MaxBytesReader(w, r.Body, 1<<20)` **before** decoding; the hub's Boundary Checklist gate 2 |
188
+ | A large integer id from untyped JSON | Never decode into `any` — JSON numbers become `float64` and ids above 2^53 lose precision silently. Decode into a struct, or use `dec.UseNumber()` |
189
+
190
+ Never rely on `omitempty` to hide a secret or an internal field. Use `json:"-"` so the field cannot leak when its value happens to be non-zero.
191
+
192
+ ## 9. Routing
193
+
194
+ **Go 1.22's `ServeMux` is enough for most services.** It matches methods and wildcards natively, and Go 1.23 exposes the matched pattern for the metrics label:
195
+
196
+ ```go
197
+ mux.HandleFunc("GET /orders/{id}", getOrder) // method-aware; a POST here gets 405, not 404
198
+ id := r.PathValue("id") // no regex, no third-party router
199
+ route := r.Pattern // Go 1.23+: the label observability.md §metrics requires
200
+ ```
201
+
202
+ Reach for `chi` (stdlib-compatible `http.Handler`, so nothing else changes) when you hit one of these, and name which one in the commit:
203
+
204
+ | What you need | Stdlib `ServeMux` | Verdict |
205
+ | --- | --- | --- |
206
+ | Per-group middleware (`/admin/*` behind one auth chain) | Manual wrapping at every registration | `chi` — the missed wrapper is an authorization hole |
207
+ | Sub-router mounted under a prefix, with the prefix stripped | Hand-rolled `StripPrefix` per subtree | `chi` |
208
+ | The matched route pattern for metrics on Go < 1.23 | Not exposed | `chi.RouteContext(ctx).RoutePattern()` — see [observability.md](observability.md) |
209
+ | Constrained params (`{id:[0-9]+}`) | Not supported; validate in the handler instead | Usually *not* a reason — validation belongs at the edge anyway |
210
+ | "It's what I know" / raw benchmark numbers | — | Not a reason. Routing is never the bottleneck; the pool is |
211
+
212
+ `echo` and `gin` bring their own context type, their own binding, and their own error handling, which means every helper you write is coupled to the framework. Take that trade only for something they give you that `chi` does not.
213
+
214
+ ## Never Ship (Go)
215
+
216
+ | Never | Instead |
217
+ | --- | --- |
218
+ | `http.ListenAndServe(addr, mux)` in production | An `http.Server` with all four timeouts set (§2) |
219
+ | `http.Client{}` or `http.DefaultClient` for a third party | A named client with `Timeout`, plus `NewRequestWithContext` |
220
+ | A `*http.Client` built inside a handler | One per upstream, built at startup |
221
+ | `db.Query` / `db.Exec` / `db.Begin` | The `…Context` variants, with the request's ctx |
222
+ | `context.Background()` below the entry point | Thread `ctx`; `WithoutCancel` when you genuinely need to outlive the request |
223
+ | `context.Context` in a struct field | First parameter, named `ctx` |
224
+ | `context.WithTimeout` without `defer cancel()` | `defer cancel()`; `go vet` fails otherwise |
225
+ | `sql.DB` with default pool settings | §3, with the number sourced from [testing-and-perf.md](testing-and-perf.md) §8 |
226
+ | `for rows.Next()` with no `rows.Err()` or no `defer rows.Close()` | Both, every time |
227
+ | `fmt.Errorf("...: %v", err)` on a path a caller must classify | `%w` |
228
+ | A retry decision from `err.Error()` text | `errors.Is` / `errors.As` on a sentinel, type, or SQLSTATE |
229
+ | `go work(r.Context())` fire-and-forget | `context.WithoutCancel` + a fresh deadline, or a real job |
230
+ | An unbuffered channel written by a goroutine the reader may abandon | Buffer of 1, or `select` on `ctx.Done()` in the sender too |
231
+ | Unbounded `go` in a loop over user input | `errgroup` with `SetLimit` |
232
+ | `wg.Add(1)` inside the goroutine | Before the `go` statement |
233
+ | Treating `http.ErrServerClosed` as a fatal error | `errors.Is` and exit 0 |
234
+ | `panic` for an expected failure, or a `recover` that logs nothing | Typed error; one recovery middleware that logs the stack |
235
+ | An untagged struct serialized to JSON | Explicit `json:"..."` tags; `json:"-"` for internals |
236
+ | `omitempty` on a field where `false` or `0` is meaningful | `*bool` / `*int64` |
237
+ | Decoding a request body with no size cap | `http.MaxBytesReader` before the decoder |
238
+ | A framework added for routing alone | Go 1.22 `ServeMux` |