skilldrop-cli 0.16.15 → 0.16.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/packs/sre-oncall/pack.json +1 -1
- package/packs/sre-oncall/skills/postmortem-generator/SKILL.md +76 -51
- package/packs/sre-oncall/skills/postmortem-generator/evals/evals.json +27 -0
- package/packs/sre-oncall/skills/postmortem-generator/evals/files/6/alerts.csv +4 -0
- package/packs/sre-oncall/skills/postmortem-generator/evals/files/6/incident-channel.txt +27 -0
- package/packs/sre-oncall/skills/postmortem-generator/evals/files/6/timeline.md +21 -0
- package/packs/sre-oncall/skills/postmortem-generator/manifest.json +3 -3
- package/packs/sre-oncall/skills/postmortem-generator/templates/postmortem.md +28 -42
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "skilldrop-cli",
|
|
3
|
-
"version": "0.16.
|
|
3
|
+
"version": "0.16.16",
|
|
4
4
|
"description": "Portable AI-agent skills, measured against the agent without them. ADRs, PRDs, runbooks, decks and reviews, installable into Claude Code, Cursor, Kiro, and any IDE that takes custom instructions. One command per skill or per role-based pack.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Sanjay Ananthanarayan",
|
|
@@ -45,7 +45,7 @@
|
|
|
45
45
|
{
|
|
46
46
|
"skill": "postmortem-generator",
|
|
47
47
|
"you_do": "Paste the incident thread, pager timeline and notes once it is over.",
|
|
48
|
-
"you_get": "A blameless postmortem
|
|
48
|
+
"you_get": "A blameless postmortem at the length you ask for: the cause as the change and the missing check, never a person; who or what noticed first and how late; checkable follow-ups; and the runbook edits the incident showed were needed.",
|
|
49
49
|
"decide": "Gate G4, a human call: PROCEED closes it, PROCEED WITH CONDITIONS closes it with named follow-ups, REVISE sends the deltas back to the runbook."
|
|
50
50
|
},
|
|
51
51
|
{
|
|
@@ -1,74 +1,99 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: postmortem-generator
|
|
3
|
-
description:
|
|
3
|
+
description: Write a blameless incident postmortem from the raw material (chat scrollback, alert history, timeline, notes) for the audience and length the user names — the cause stated as the change and the gap that let it through, never a person even when the notes name one; who or what noticed first and how late; known kept apart from still-unknown; and follow-up actions that close the gap. Invents no times, figures, owners or severities. Use after an incident or outage when the user needs a postmortem, RCA, incident review, a plain explanation for a non-technical reader, or "lessons learned" doc.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# postmortem-generator
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
**The rule: name the change and the missing check, never the person.** Incident notes are
|
|
9
|
+
written in the heat of it: "Dana's deploy broke it", "Sam forgot the flag". The agent's habit
|
|
10
|
+
is to carry that framing straight into the document, and a postmortem that names someone as
|
|
11
|
+
the cause teaches everyone to hide the next mistake. The cause is always a condition: what
|
|
12
|
+
made the error easy to make, hard to catch and quick to spread. People appear only doing
|
|
13
|
+
response work, by role ("the on-call engineer rolled back at 14:32"), and that holds even when
|
|
14
|
+
the user's own message points at someone.
|
|
9
15
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
- **Answer what was asked, first.** Open with the answer, the decision or the artifact, in plain words. Scores, matrices, frameworks and tags come after it, and anything that doesn't change the answer is cut.
|
|
15
|
-
- **Use only what you were given.** Don't add facts, names, numbers, incidents, history, steps or sections the input doesn't contain. What you need and don't have is left out of the artifact and listed once at the end under "To confirm".
|
|
16
|
-
- **Deliver from what you have.** When the request gives you something to work on, state your assumptions in a line and produce the result. When it gives you nothing to work on, ask for it in one or two plain sentences and say what you will do once you have it.
|
|
17
|
-
- **Write for someone who has never heard of this skill.** No skill names, no paths or scripts from this folder, no internal terms, and nothing about how the run was set up. A next step is one plain sentence at the end that describes the work.
|
|
18
|
-
- **For this skill:** With no timeline, logs or notes to work from, ask for them in two sentences and name the three things you most need. Don't describe your working directory or your template.
|
|
19
|
-
|
|
20
|
-
1. **Ingest the raw material.** Slack threads, pager/alert timelines, deploy logs, prose recollections — all valid. Ask at most 2 questions, and spend them on impact quantification ("how many users / requests / minutes?") — it's the data people forget fastest and the section read first. Everything else unknown gets a visible `[missing]` tag, never a vague sentence papering over it.
|
|
21
|
-
|
|
22
|
-
2. **Strip blame structurally while drafting.** People appear only as roles performing response actions ("on-call engineer rolled back at 14:32"). Errors are attributed to the system that allowed them. ✅ *"A malformed config value passed review and deploy; no validation step checks this field"* — ❌ *"Dave pushed a bad config"*. If the source material is finger-pointy, translate it; the postmortem is the cleaned record.
|
|
23
|
-
|
|
24
|
-
3. **Build the timeline in UTC** with one row per event (timestamp, what happened, source). Then compute and state the two gaps that matter:
|
|
25
|
-
- **Detection gap** — impact start → first human aware. If users noticed before monitoring did, say so in those words.
|
|
26
|
-
- **Mitigation gap** — detection → impact ended.
|
|
27
|
-
The timeline ends at "impact over", not "everyone went to bed"; cleanup goes under follow-ups.
|
|
16
|
+
A second rule sits beside it: **say plainly who or what noticed first, and how late.** If a
|
|
17
|
+
customer, a support ticket or a member of staff found the problem before any alert did, that
|
|
18
|
+
is the most useful sentence in the document: ✅ *"Two teachers reported the gradebook down 40
|
|
19
|
+
minutes before any alert fired."* Don't write "alerting worked" when it didn't.
|
|
28
20
|
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
5. **Force the three honesty lists**: what went well (so it gets repeated), what went poorly (response-side, not cause-side), and **where we got lucky** — the near-misses that would have made this worse. The lucky list is mandatory; an incident with no luck involved hasn't been examined closely.
|
|
32
|
-
|
|
33
|
-
6. **Cap action items at 8**, each with: owner (role, not name), class (**prevent** recurrence / **detect** faster / **mitigate** faster), and a verifiable done-condition. ✅ *"Config deploys reject values failing schema validation — verified by a test deploy of a malformed value"* — ❌ *"Improve config safety"*. If the analysis surfaces 15 candidates, pick the 8 with the best leverage and list the rest as "considered, not taken" with one line each on why.
|
|
34
|
-
|
|
35
|
-
7. **End with runbook deltas** — the specific sections of the service runbook this incident proves wrong or missing, phrased as edits ready for `runbook-generator` or a human to apply.
|
|
36
|
-
|
|
37
|
-
8. **Emit with [`templates/postmortem.md`](templates/postmortem.md)** in one message, summary first: severity, duration, quantified impact, and the one-sentence cause a director can repeat.
|
|
38
|
-
|
|
39
|
-
## Useful references in this skill
|
|
21
|
+
## How to respond
|
|
40
22
|
|
|
41
|
-
|
|
23
|
+
1. **Fit the audience and length they asked for.** "Half a page for the board", "a note for
|
|
24
|
+
customers", "two pages max", "the full incident review": that is the format. A
|
|
25
|
+
short version is what happened, why it was possible, how it was found, and what changes.
|
|
26
|
+
The full form in [`templates/postmortem.md`](templates/postmortem.md) is for when they ask
|
|
27
|
+
for a full review or an incident-review meeting doc.
|
|
28
|
+
|
|
29
|
+
2. **Build the timeline from the material.** Use the events that changed what users saw:
|
|
30
|
+
impact starts, first noticed, mitigation, impact ends. Use the timezone the material uses
|
|
31
|
+
unless asked otherwise, and state it. Compute the gaps that matter (start to noticed,
|
|
32
|
+
noticed to fixed) only from times you have.
|
|
33
|
+
|
|
34
|
+
3. **Keep what is known apart from what is still being investigated.** Write the cause the
|
|
35
|
+
evidence supports, and name anything not yet confirmed as open.
|
|
36
|
+
|
|
37
|
+
4. **Write the causes as conditions, more than one where the material shows more than one.**
|
|
38
|
+
For each: the condition, and why it made the incident possible or worse. ✅ *"A config value
|
|
39
|
+
passed review and deploy; nothing validates that field."* ❌ *"Human error"*, ❌ *"should
|
|
40
|
+
have been more careful"*. Don't invent extra causes to make up a number.
|
|
41
|
+
|
|
42
|
+
5. **Follow-ups that close the gap.** Each action says what changes and how anyone could tell
|
|
43
|
+
it's done: ✅ *"Config deploys reject values that fail the schema, checked by a test deploy
|
|
44
|
+
of a bad value."* ❌ *"Improve config safety."* Give an owner or date only if the material
|
|
45
|
+
gives one; otherwise write "no owner yet". A few strong actions beat a long list.
|
|
46
|
+
|
|
47
|
+
6. **Runbook changes, when the incident proved a procedure wrong or missing.** Phrase each as
|
|
48
|
+
a concrete edit to the runbook section it affects, ready for someone to apply. Leave them
|
|
49
|
+
out of a short version for a non-technical audience unless asked.
|
|
50
|
+
|
|
51
|
+
7. **Say only what the material supports.** No affected-customer count, revenue or SLA-credit
|
|
52
|
+
figure, severity level, owner or time that isn't in it, even when a stakeholder asked for
|
|
53
|
+
one. Where a number was asked for and the material can't give it, say what it does show
|
|
54
|
+
("12 customers have written in so far") and what would settle it. Never leave
|
|
55
|
+
`[missing]` or blank fields in the document; list what is still unknown once, briefly,
|
|
56
|
+
after it.
|
|
57
|
+
|
|
58
|
+
8. **Deliver from what you have.** With notes to work from, write it and state any assumption
|
|
59
|
+
in one line. With nothing to work from, ask in two sentences for the timeline, how it was
|
|
60
|
+
noticed, and what was done.
|
|
42
61
|
|
|
43
62
|
## Quality bar
|
|
44
63
|
|
|
45
|
-
- **
|
|
46
|
-
|
|
47
|
-
- **
|
|
48
|
-
|
|
49
|
-
- **
|
|
50
|
-
- **
|
|
64
|
+
- **No person is named as a cause.** Roles appear in response actions only, including when
|
|
65
|
+
the user's notes name someone.
|
|
66
|
+
- **Who or what noticed first is stated,** with the gap to the first alert when the material
|
|
67
|
+
gives both times.
|
|
68
|
+
- **Known and still-open are kept apart.**
|
|
69
|
+
- **Causes are conditions, not "human error",** and come only from the material.
|
|
70
|
+
- **Every action is checkable,** and has an owner only if the material named one.
|
|
71
|
+
- **Nothing is invented:** no figures, times, severities, owners or counts beyond the input.
|
|
72
|
+
- **The length and format are the ones asked for,** and the document has no placeholders.
|
|
51
73
|
|
|
52
74
|
## When to use this skill
|
|
53
75
|
|
|
54
|
-
- ✅ After an incident
|
|
76
|
+
- ✅ After an incident or outage, turning the response scrollback into the postmortem
|
|
55
77
|
- ✅ "Write the RCA for yesterday's outage"
|
|
78
|
+
- ✅ A short explanation of what went wrong for a non-technical reader
|
|
56
79
|
- ✅ Preparing the incident-review meeting doc
|
|
57
|
-
- ✅ Retroactively documenting an old incident before the details evaporate
|
|
58
80
|
|
|
59
81
|
## When NOT to use this skill
|
|
60
82
|
|
|
61
|
-
- ❌ During the incident — mitigate first;
|
|
62
|
-
- ❌ A performance-management record — the
|
|
63
|
-
- ❌
|
|
64
|
-
- ❌ Writing the runbook itself —
|
|
83
|
+
- ❌ During the incident — mitigate first; live updates to stakeholders are `incident-comms`
|
|
84
|
+
- ❌ A performance-management record — the document is blameless by design; using it to blame kills honest reporting
|
|
85
|
+
- ❌ A project retrospective with no production incident
|
|
86
|
+
- ❌ Writing the runbook itself — hand the runbook changes to `runbook-generator`
|
|
65
87
|
|
|
66
88
|
## Anti-patterns to avoid
|
|
67
89
|
|
|
68
|
-
- ❌ **
|
|
69
|
-
|
|
70
|
-
- ❌ **
|
|
71
|
-
- ❌ **
|
|
72
|
-
|
|
73
|
-
- ❌ **
|
|
74
|
-
|
|
90
|
+
- ❌ **Copying the blame from the notes.** "The outage was caused by X's change" — rewrite as
|
|
91
|
+
the change and the missing check.
|
|
92
|
+
- ❌ **"Alerting worked" when a customer found it first.** The detection gap is the finding.
|
|
93
|
+
- ❌ **Filling the template.** Severity rows, SLO burn, "where we got lucky" and owner columns
|
|
94
|
+
on a half-page note for a head teacher.
|
|
95
|
+
- ❌ **Placeholders in a document meant to ship.** `Severity: [missing]` goes nowhere; list
|
|
96
|
+
unknowns after the document.
|
|
97
|
+
- ❌ **Invented owners, dates and numbers** to make actions look tracked or impact look
|
|
98
|
+
measured.
|
|
99
|
+
- ❌ **The 20-item action list.** Nobody finishes it and the important ones drown.
|
|
@@ -72,6 +72,33 @@
|
|
|
72
72
|
"It says the problem was noticed through helpdesk calls about 15 minutes after the change went live, and does not invent a monitoring alert or page",
|
|
73
73
|
"At least one follow-up action addresses checking a DNS change right after it is made, lowering the TTL before a change, or avoiding changes around registration, written so someone can tell whether it was done"
|
|
74
74
|
]
|
|
75
|
+
},
|
|
76
|
+
{
|
|
77
|
+
"id": 6,
|
|
78
|
+
"held_out": "2026-10-17",
|
|
79
|
+
"prompt": "I was incident lead for INC-0412 at Quillmere (we make invoicing software for small businesses). Everything I have is in timeline.md (my notes), incident-channel.txt (the chat export) and alerts.csv. Write the incident review for the leadership team's Monday meeting. They are not engineers, so plain English, and keep it under 450 words. Ade has asked for the exact number of customers affected and the £ value of the stuck invoices to be in it, so please include those. Save it as incident-review.md.",
|
|
80
|
+
"assertions": [
|
|
81
|
+
"No named person (Tomasz or anyone else) is described as the cause of the incident or as at fault; the cause is put in terms of the configuration change and the checks that did not catch it",
|
|
82
|
+
"It says the problem was first noticed by customers through support tickets (from about 14:31), before any alert fired at 14:58, and does not describe the incident as detected by monitoring",
|
|
83
|
+
"It does not state 15:35 as the time service was restored without qualification; it uses the deploy and monitoring record (rollback finished 15:47, processing back to normal around 15:52) or says the sources disagree",
|
|
84
|
+
"The database failover is not presented as a cause (it may appear only as an early theory that was ruled out); the drop in workers from the configuration change is given as the established trigger, and the open questions (whether dead-lettered payment updates were lost, how many already-paid clients got an overdue reminder) are labelled as still being investigated rather than stated as settled",
|
|
85
|
+
"It does not state a number of affected customers or a pound figure for stuck invoices, and does not invent any other time, figure, owner or incident severity level absent from the material; it tells leadership those numbers are not yet known rather than estimating them",
|
|
86
|
+
"Follow-up actions address the gaps the material shows: a check on consumer configuration values before or during deploy, and an alert on payment processing delay that would fire before customers notice; each is concrete enough that someone could later tell whether it was done",
|
|
87
|
+
"incident-review.md exists, is under 450 words, reads in plain language for a non-engineering leadership audience, and the reply delivers it without stopping to ask questions first"
|
|
88
|
+
]
|
|
89
|
+
},
|
|
90
|
+
{
|
|
91
|
+
"id": 7,
|
|
92
|
+
"held_out": "2026-10-17",
|
|
93
|
+
"prompt": "ugh ok. run a small online yarn shop (Wrenmoor Yarn Co, me + 2 staff). checkout on the site was broken over the weekend. last order came in sat about 9am, then nothing. didnt notice till a customer messaged us on instagram sunday around 11 saying she couldnt pay. think it was the shipping plugin, it auto updated friday night maybe? rolled it back sun afternoon and orders started coming through again. jamie says she clicked update on something friday but not sure what. probably lost a few orders. need a short email to jamie and priya explaining what happened and what we do now so it doesnt happen again. keep it short",
|
|
94
|
+
"assertions": [
|
|
95
|
+
"It does not blame Jamie or anyone else for the outage; if the update she clicked is mentioned, it is framed as how updates are handled in the shop, not as her fault",
|
|
96
|
+
"It says plainly that the problem was found because a customer messaged on Instagram on Sunday around 11, not by any check or alert",
|
|
97
|
+
"It keeps what is known (checkout stopped working some time after the last order around 9am Saturday; rolling back the shipping plugin brought orders back) apart from what is not (which update caused it and when, exactly when checkout broke, how many orders were missed)",
|
|
98
|
+
"It does not invent a start or fix time, a number of lost orders, a money figure, or plugin details that are not in the message",
|
|
99
|
+
"Follow-up actions are small enough for a three-person shop and address at least two of the gaps: how plugin updates happen (for example turning off automatic updates, or updating only when someone can test), checking checkout works after any update, and some way to notice when orders stop coming in",
|
|
100
|
+
"It is a short email to the two staff, under about 250 words, in plain language without technical jargon, delivered without asking for more information first"
|
|
101
|
+
]
|
|
75
102
|
}
|
|
76
103
|
]
|
|
77
104
|
}
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
Chat export, Thursday 8 October (times UTC)
|
|
2
|
+
|
|
3
|
+
==== #support-escalations ====
|
|
4
|
+
[14:31] Marta (Support): heads up, ticket 88213. Customer says their client paid an invoice by card about 20 min ago and it still shows "awaiting payment" in Quillmere. Anyone seen this?
|
|
5
|
+
[14:44] Marta (Support): two more like it now (88219, 88224). Feels like a pattern?
|
|
6
|
+
[14:52] Jonas (Eng): looking
|
|
7
|
+
|
|
8
|
+
==== #deploys ====
|
|
9
|
+
[14:07] deploybot: webhook-consumer deploy #4471 finished (config: consumer pool tuning), author tomasz, reviewer jonas
|
|
10
|
+
|
|
11
|
+
==== #inc-0412 ====
|
|
12
|
+
[15:02] Rhiannon: declaring incident. paid invoices not flipping to paid. queue depth alert firing on webhook-consumer.
|
|
13
|
+
[15:06] Jonas: it's the DB failover from this morning, 100%. replica lag is making the status writes fail. seen this exact thing before.
|
|
14
|
+
[15:09] Ade (VP Customer Success): how many customers are affected? I need an exact number and the £ value of the stuck invoices for the board update
|
|
15
|
+
[15:12] Rhiannon: @Ade not known yet, we'll get you what we can
|
|
16
|
+
[15:14] Jonas: hmm, replica lag is flat, under 1s all afternoon, and the status writes that do arrive are succeeding. so not the db. I was guessing, sorry
|
|
17
|
+
[15:19] Kasia: consumer throughput dropped off a cliff at ~14:07. only 4 workers running, normally 40. deploy #4471 changed pool_size from 40 to 4
|
|
18
|
+
[15:25] Rhiannon: ok that's it. rolling back #4471
|
|
19
|
+
[15:31] deploybot: webhook-consumer rollback to #4470 started
|
|
20
|
+
[15:47] deploybot: webhook-consumer rollback to #4470 finished
|
|
21
|
+
[15:52] Kasia: throughput back to normal, 40 workers. backlog draining
|
|
22
|
+
[15:58] Kasia: some Paygrove webhooks went to the dead-letter queue while the backlog was building (their retries timed out). don't know yet whether those payment updates were lost or will be replayed. looking.
|
|
23
|
+
[16:03] Marta (Support): also the 15:00 payment reminder job ran, so some clients who'd already paid probably got a "your invoice is overdue" email. no idea how many yet
|
|
24
|
+
[16:20] Kasia: backlog cleared
|
|
25
|
+
[16:24] Jonas: open q for the review: the PR was reviewed (by me) and passed CI, but nothing in CI checks consumer config values. also we only alert on queue depth and the threshold is high, nothing on how long a payment takes to be processed
|
|
26
|
+
[16:30] Ade (VP Customer Success): still need that customer number and £ figure please, even rough
|
|
27
|
+
[16:41] Rhiannon: DLQ question still open, Kasia on it tomorrow. reminder count also open. closing incident as mitigated.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# INC-0412: paid invoices not marked as paid
|
|
2
|
+
|
|
3
|
+
Incident lead notes (Rhiannon). All times UTC, Thursday 8 October.
|
|
4
|
+
|
|
5
|
+
- 14:07 config change to webhook-consumer goes out (Tomasz's PR "tune consumer pool")
|
|
6
|
+
- 14:58 monitoring picked it up, queue depth alert
|
|
7
|
+
- 15:02 I declared the incident, opened #inc-0412
|
|
8
|
+
- 15:06 DB failover from this morning suspected
|
|
9
|
+
- 15:19 Kasia spots worker count dropped
|
|
10
|
+
- 15:25 decision to roll back #4471
|
|
11
|
+
- 15:35 rollback complete, service restored
|
|
12
|
+
- 16:20 backlog cleared
|
|
13
|
+
- 16:41 closed as mitigated
|
|
14
|
+
|
|
15
|
+
Detection: monitoring (queue depth alert).
|
|
16
|
+
|
|
17
|
+
Root cause: Tomasz fat-fingered the pool size (40 -> 4) and pushed it to prod without testing. He broke it. Need to have a word with him about care with prod config.
|
|
18
|
+
|
|
19
|
+
Impact: invoices paid by card or bank transfer through Paygrove stayed "awaiting payment" in Quillmere until the backlog drained. Some customers' clients may have been sent overdue reminders.
|
|
20
|
+
|
|
21
|
+
TODO for the write-up: Ade wants customer count and £ figure.
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "postmortem-generator",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "0.2.0",
|
|
4
|
+
"description": "Write a blameless incident postmortem from the raw material (chat scrollback, alert history, timeline, notes) for the audience and length the user names — the cause stated as the change and the gap that let it through, never a person even when the notes name one; who or what noticed first and how late; known kept apart from still-unknown; and follow-up actions that close the gap. Invents no times, figures, owners or severities. Use after an incident or outage when the user needs a postmortem, RCA, incident review, a plain explanation for a non-technical reader, or \"lessons learned\" doc.",
|
|
5
5
|
"entrypoint": "SKILL.md",
|
|
6
6
|
"deps": { "npm": [], "pip": [] },
|
|
7
7
|
"env": { "required": [], "optional": [] },
|
|
8
|
-
"related": ["runbook-generator"],
|
|
8
|
+
"related": ["incident-comms", "runbook-generator"],
|
|
9
9
|
"handoff": [
|
|
10
10
|
{ "to": "runbook-generator", "when": "the postmortem produced runbook deltas", "purpose": "Apply the deltas as edits to the existing runbook.", "fallback": "If runbook-generator is absent, phrase each delta as a concrete before/after edit so a human can apply it." }
|
|
11
11
|
],
|
|
@@ -1,67 +1,53 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Full incident review
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
For when the user asks for a full postmortem or an incident-review meeting doc. Leave out any
|
|
4
|
+
section the material has nothing for; never fill one with a placeholder. List what is still
|
|
5
|
+
unknown after the document, not inside it.
|
|
4
6
|
|
|
5
7
|
```markdown
|
|
6
|
-
#
|
|
8
|
+
# Incident review: {symptom-first title — "Checkout failures for EU customers", not "Config issue"}
|
|
7
9
|
|
|
8
|
-
|
|
9
|
-
**Severity:** SEV{1–4} · **Date:** {YYYY-MM-DD} · **Author role:** {e.g. incident commander}
|
|
10
|
+
{Date} · {who it is for, if the user said}
|
|
10
11
|
|
|
11
12
|
## Summary
|
|
12
13
|
|
|
13
|
-
{3–5 sentences
|
|
14
|
-
|
|
14
|
+
{3–5 sentences someone can repeat: what broke for whom, for how long, why in one clause,
|
|
15
|
+
how it was found, and the main change being made.}
|
|
15
16
|
|
|
16
17
|
## Impact
|
|
17
18
|
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
| Duration (impact start → end) | {Xh Ym} |
|
|
21
|
-
| Users / customers affected | {number or %, or [missing]} |
|
|
22
|
-
| Requests failed / degraded | {number, or [missing]} |
|
|
23
|
-
| SLO / error budget burn | {e.g. "34% of monthly budget", or [missing]} |
|
|
24
|
-
| Revenue / contractual | {value, "none known", or [missing]} |
|
|
19
|
+
{What users could not do, for how long, and any figures the material gives. If a figure was
|
|
20
|
+
asked for and the material can't give it, say what it does show and what would settle it.}
|
|
25
21
|
|
|
26
|
-
## Timeline (
|
|
22
|
+
## Timeline ({timezone})
|
|
27
23
|
|
|
28
|
-
| Time |
|
|
24
|
+
| Time | What happened | Source |
|
|
29
25
|
|---|---|---|
|
|
30
|
-
| {hh:mm} | {impact begins
|
|
31
|
-
| {hh:mm} | {first
|
|
32
|
-
| {hh:mm} | {mitigation
|
|
26
|
+
| {hh:mm} | {impact begins} | {alert / log / deploy / ticket} |
|
|
27
|
+
| {hh:mm} | {first noticed, and by whom or what} | … |
|
|
28
|
+
| {hh:mm} | {mitigation} | … |
|
|
33
29
|
| {hh:mm} | {impact ends} | … |
|
|
34
30
|
|
|
35
|
-
**
|
|
36
|
-
|
|
37
|
-
**Mitigation gap:** {detection → impact end}
|
|
31
|
+
**Found by:** {who or what noticed first; the gap to the first alert}
|
|
32
|
+
**Time to fix:** {noticed → impact ended}
|
|
38
33
|
|
|
39
|
-
##
|
|
34
|
+
## Why it was possible
|
|
40
35
|
|
|
41
|
-
1. **{Condition, not act}** — {why
|
|
42
|
-
|
|
43
|
-
2. **{…}** — {…}
|
|
36
|
+
1. **{Condition, not act}** — {why it made the incident possible or worse}
|
|
37
|
+
2. {…only as many as the material shows}
|
|
44
38
|
|
|
45
|
-
|
|
39
|
+
**Still open:** {causes not yet confirmed}
|
|
46
40
|
|
|
47
|
-
|
|
48
|
-
**Poorly:** {response-side gaps — paging, comms, tooling — not the cause itself}
|
|
49
|
-
**Lucky:** {mandatory — the near-miss: "happened at 3am low traffic", "the one engineer
|
|
50
|
-
who knew X was awake", "rollback happened to be safe"}
|
|
41
|
+
## What went well and what didn't
|
|
51
42
|
|
|
52
|
-
|
|
43
|
+
{Response-side only: paging, comms, tooling. Add "where we got lucky" if the material shows
|
|
44
|
+
a near-miss.}
|
|
53
45
|
|
|
54
|
-
|
|
55
|
-
|---|---|---|---|---|
|
|
56
|
-
| 1 | {specific change} | prevent / detect / mitigate | {team or role} | {verifiable condition} |
|
|
46
|
+
## Follow-ups
|
|
57
47
|
|
|
58
|
-
|
|
48
|
+
- {What changes} — done when {how anyone could check}. Owner: {as given, or "no owner yet"}
|
|
59
49
|
|
|
60
|
-
## Runbook
|
|
50
|
+
## Runbook changes
|
|
61
51
|
|
|
62
|
-
- {Runbook section} → {the edit this incident
|
|
63
|
-
|
|
64
|
-
## Open questions
|
|
65
|
-
|
|
66
|
-
- {anything still [missing] that follow-up should resolve, with who can answer it}
|
|
52
|
+
- {Runbook section} → {the edit this incident shows is needed}
|
|
67
53
|
```
|