@gigzen/populace 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +661 -0
- package/README.md +258 -0
- package/adapters/buzzbuzz.mjs +247 -0
- package/adapters/contract.md +164 -0
- package/adapters/template-rest.mjs +192 -0
- package/adapters/template.mjs +80 -0
- package/examples/buzzbuzz/populace-report.html +245 -0
- package/examples/buzzbuzz/populace-report.json +280 -0
- package/examples/buzzbuzz/populace.config.mjs +51 -0
- package/examples/buzzbuzz/run-test.ps1 +61 -0
- package/examples/demo/adapters/demo.mjs +90 -0
- package/examples/demo/populace-report.html +230 -0
- package/examples/demo/populace-report.json +219 -0
- package/examples/demo/populace.config.mjs +22 -0
- package/examples/rest-api/README.md +85 -0
- package/examples/rest-api/adapter.mjs +166 -0
- package/examples/rest-api/populace.config.mjs +40 -0
- package/examples/rest-api/server.mjs +247 -0
- package/examples/token-expiry/expiry-demo.mjs +119 -0
- package/package.json +56 -0
- package/populace.config.example.mjs +65 -0
- package/src/cli.mjs +591 -0
- package/src/config.mjs +186 -0
- package/src/contract.mjs +130 -0
- package/src/diagnose.mjs +40 -0
- package/src/engine/agent.mjs +264 -0
- package/src/engine/geo.mjs +59 -0
- package/src/engine/index.mjs +4 -0
- package/src/engine/personas.mjs +115 -0
- package/src/engine/world.mjs +120 -0
- package/src/html-report.mjs +218 -0
- package/src/index.mjs +38 -0
- package/src/instrument.mjs +299 -0
- package/src/net.mjs +175 -0
- package/src/report.mjs +251 -0
- package/src/selftest.mjs +1369 -0
- package/src/smoke.mjs +274 -0
- package/src/version.mjs +24 -0
|
@@ -0,0 +1,230 @@
|
|
|
1
|
+
<!doctype html>
|
|
2
|
+
<html lang="en"><head><meta charset="utf-8">
|
|
3
|
+
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
4
|
+
<title>Populace report — Demo App</title>
|
|
5
|
+
<style>
|
|
6
|
+
:root{
|
|
7
|
+
--paper:#F4F5F7;--card:#fff;--ink:#12181D;--body:#3A454E;--muted:#6B7883;
|
|
8
|
+
--rule:#DDE2E7;--accent:#A85A0B;--ok:#2C6E4C;--bad:#A63127;--warn:#9A6A08;
|
|
9
|
+
--bar:#C9D4DC;--barbad:#E0A99F;
|
|
10
|
+
--mono:ui-monospace,"SF Mono","Cascadia Mono",Menlo,Consolas,monospace;
|
|
11
|
+
--sans:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Arial,sans-serif;
|
|
12
|
+
}
|
|
13
|
+
@media (prefers-color-scheme:dark){:root{
|
|
14
|
+
--paper:#0E1418;--card:#161E24;--ink:#E8EEF3;--body:#B0BDC7;--muted:#7B8994;
|
|
15
|
+
--rule:#26323A;--accent:#E0A45C;--ok:#5FAB80;--bad:#DE7568;--warn:#D2A24E;
|
|
16
|
+
--bar:#2E3A43;--barbad:#5C332C;}}
|
|
17
|
+
:root[data-theme="dark"]{--paper:#0E1418;--card:#161E24;--ink:#E8EEF3;--body:#B0BDC7;
|
|
18
|
+
--muted:#7B8994;--rule:#26323A;--accent:#E0A45C;--ok:#5FAB80;--bad:#DE7568;
|
|
19
|
+
--warn:#D2A24E;--bar:#2E3A43;--barbad:#5C332C;}
|
|
20
|
+
:root[data-theme="light"]{--paper:#F4F5F7;--card:#fff;--ink:#12181D;--body:#3A454E;
|
|
21
|
+
--muted:#6B7883;--rule:#DDE2E7;--accent:#A85A0B;--ok:#2C6E4C;--bad:#A63127;
|
|
22
|
+
--warn:#9A6A08;--bar:#C9D4DC;--barbad:#E0A99F;}
|
|
23
|
+
*{box-sizing:border-box}
|
|
24
|
+
body{margin:0;background:var(--paper);color:var(--body);font-family:var(--sans);
|
|
25
|
+
font-size:15.5px;line-height:1.6;-webkit-font-smoothing:antialiased}
|
|
26
|
+
.wrap{max-width:900px;margin:0 auto;padding:0 22px 80px}
|
|
27
|
+
h1,h2,h3{color:var(--ink);margin:0;text-wrap:balance}
|
|
28
|
+
h1{font-family:var(--mono);font-size:clamp(22px,4vw,30px);letter-spacing:-.02em}
|
|
29
|
+
h2{font-size:17px;font-family:var(--mono);letter-spacing:-.01em}
|
|
30
|
+
.mono{font-family:var(--mono)}
|
|
31
|
+
header{padding:44px 0 24px;border-bottom:2px solid var(--ink);display:flex;
|
|
32
|
+
flex-direction:column;gap:10px}
|
|
33
|
+
.sub{font-family:var(--mono);font-size:12.5px;color:var(--muted)}
|
|
34
|
+
.verdict{margin-top:26px;padding:20px 22px;border-radius:6px;display:flex;
|
|
35
|
+
flex-direction:column;gap:8px;border-left:4px solid}
|
|
36
|
+
.verdict.ok{background:color-mix(in srgb,var(--ok) 8%,transparent);border-color:var(--ok)}
|
|
37
|
+
.verdict.bad{background:color-mix(in srgb,var(--bad) 8%,transparent);border-color:var(--bad)}
|
|
38
|
+
.verdict h2{color:var(--ink)}
|
|
39
|
+
.verdict ul{margin:4px 0 0;padding-left:20px}
|
|
40
|
+
.stats{display:grid;gap:12px;grid-template-columns:repeat(auto-fit,minmax(132px,1fr));margin-top:26px}
|
|
41
|
+
.stat{background:var(--card);border:1px solid var(--rule);border-radius:6px;padding:15px 16px}
|
|
42
|
+
.stat .v{font-family:var(--mono);font-size:23px;color:var(--ink);font-weight:600;
|
|
43
|
+
letter-spacing:-.02em;font-variant-numeric:tabular-nums;display:block}
|
|
44
|
+
.stat .l{font-family:var(--mono);font-size:10px;letter-spacing:.12em;
|
|
45
|
+
text-transform:uppercase;color:var(--muted)}
|
|
46
|
+
section{margin-top:42px;display:flex;flex-direction:column;gap:14px}
|
|
47
|
+
.panel{background:var(--card);border:1px solid var(--rule);border-radius:6px;padding:20px 22px}
|
|
48
|
+
.panel.danger{border-color:var(--bad);border-left-width:4px}
|
|
49
|
+
.panel.danger h2{color:var(--bad)}
|
|
50
|
+
.tablewrap{overflow-x:auto;border:1px solid var(--rule);border-radius:6px;background:var(--card)}
|
|
51
|
+
table{border-collapse:collapse;width:100%;min-width:600px}
|
|
52
|
+
th,td{padding:10px 14px;text-align:left;border-bottom:1px solid var(--rule)}
|
|
53
|
+
th{font-family:var(--mono);font-size:10px;letter-spacing:.12em;text-transform:uppercase;
|
|
54
|
+
color:var(--muted);font-weight:600}
|
|
55
|
+
td.num{text-align:right;font-family:var(--mono);font-variant-numeric:tabular-nums;font-size:13.5px}
|
|
56
|
+
td.name{font-size:13.5px;color:var(--ink);white-space:nowrap}
|
|
57
|
+
td.zero{color:var(--muted)}
|
|
58
|
+
td.fail{color:var(--bad);font-weight:600}
|
|
59
|
+
.x{color:var(--bad);margin-right:6px}
|
|
60
|
+
.barcell{width:120px}
|
|
61
|
+
.bar{height:7px;background:var(--bar);border-radius:4px}
|
|
62
|
+
.row-bad .bar{background:var(--barbad)}
|
|
63
|
+
tr.errrow td{padding-top:0;border-bottom:1px solid var(--rule)}
|
|
64
|
+
.err{font-family:var(--mono);font-size:12px;color:var(--bad);padding:3px 0 3px 26px}
|
|
65
|
+
.errn{color:var(--muted);margin-right:8px}
|
|
66
|
+
.trace{font-family:var(--mono);font-size:11.5px;white-space:pre-wrap;overflow-x:auto;
|
|
67
|
+
background:var(--paper);padding:14px;border-radius:5px;margin:0;color:var(--body)}
|
|
68
|
+
ul.cov{list-style:none;margin:0;padding:0;display:flex;flex-direction:column;gap:9px}
|
|
69
|
+
ul.cov li{display:flex;gap:14px;flex-wrap:wrap;align-items:baseline;font-size:14px}
|
|
70
|
+
ul.cov .mono{color:var(--ink);min-width:170px;font-size:13px}
|
|
71
|
+
.would{color:var(--muted);font-size:13.5px}
|
|
72
|
+
.chip{display:inline-block;font-family:var(--mono);font-size:12px;padding:7px 13px;
|
|
73
|
+
border-radius:5px;border:1px solid}
|
|
74
|
+
.chip.ok{color:var(--ok);border-color:var(--ok);background:color-mix(in srgb,var(--ok) 8%,transparent)}
|
|
75
|
+
.chip.bad{color:var(--bad);border-color:var(--bad);background:color-mix(in srgb,var(--bad) 8%,transparent)}
|
|
76
|
+
.chip.warn{color:var(--warn);border-color:var(--warn);background:color-mix(in srgb,var(--warn) 8%,transparent)}
|
|
77
|
+
footer{margin-top:52px;padding-top:20px;border-top:1px solid var(--rule);
|
|
78
|
+
font-size:13px;color:var(--muted);display:flex;flex-direction:column;gap:6px}
|
|
79
|
+
</style></head><body>
|
|
80
|
+
<div class="wrap">
|
|
81
|
+
|
|
82
|
+
<header>
|
|
83
|
+
<h1>Demo App</h1>
|
|
84
|
+
<span class="sub">test environment · 6 simulated people ·
|
|
85
|
+
1m 19s · Sun, 09 Aug 2026 14:08:41 GMT</span>
|
|
86
|
+
</header>
|
|
87
|
+
|
|
88
|
+
<div class="verdict bad">
|
|
89
|
+
<h2>Problems found</h2>
|
|
90
|
+
<ul><li>6 of 246 API calls failed (2.4%).</li></ul>
|
|
91
|
+
</div>
|
|
92
|
+
|
|
93
|
+
|
|
94
|
+
|
|
95
|
+
<div class="stats">
|
|
96
|
+
<div class="stat"><span class="v">246</span><span class="l">API calls</span></div>
|
|
97
|
+
<div class="stat"><span class="v" style="color:var(--bad)">6</span><span class="l">failed</span></div>
|
|
98
|
+
<div class="stat"><span class="v">2.4%</span><span class="l">failure rate</span></div>
|
|
99
|
+
<div class="stat"><span class="v">6</span><span class="l">concurrent users</span></div>
|
|
100
|
+
<div class="stat"><span class="v">12/13</span><span class="l">coverage</span></div>
|
|
101
|
+
</div>
|
|
102
|
+
|
|
103
|
+
<section>
|
|
104
|
+
<h2>Your API under 6 concurrent users</h2>
|
|
105
|
+
<div class="tablewrap">
|
|
106
|
+
<table>
|
|
107
|
+
<thead><tr><th>Method</th><th style="text-align:right">Calls</th>
|
|
108
|
+
<th style="text-align:right">Fails</th><th style="text-align:right">p50</th>
|
|
109
|
+
<th style="text-align:right">p95</th><th>p95 relative</th></tr></thead>
|
|
110
|
+
<tbody>
|
|
111
|
+
<tr class="row-bad">
|
|
112
|
+
<td class="mono name"><span class="x">✖</span>like</td>
|
|
113
|
+
<td class="num">24</td>
|
|
114
|
+
<td class="num fail">6</td>
|
|
115
|
+
<td class="num">48ms</td>
|
|
116
|
+
<td class="num">65ms</td>
|
|
117
|
+
<td class="barcell"><div class="bar" style="width:13.6%"></div></td>
|
|
118
|
+
</tr>
|
|
119
|
+
<tr class="errrow"><td colspan="6"><div class="err"><span class="errn">6×</span>new row violates row-level security policy "post_likes_insert"</div></td></tr>
|
|
120
|
+
<tr class="">
|
|
121
|
+
<td class="mono name">reportLocation</td>
|
|
122
|
+
<td class="num">114</td>
|
|
123
|
+
<td class="num zero">0</td>
|
|
124
|
+
<td class="num">31ms</td>
|
|
125
|
+
<td class="num">48ms</td>
|
|
126
|
+
<td class="barcell"><div class="bar" style="width:10.0%"></div></td>
|
|
127
|
+
</tr>
|
|
128
|
+
|
|
129
|
+
<tr class="">
|
|
130
|
+
<td class="mono name">recentPostsByOthers</td>
|
|
131
|
+
<td class="num">33</td>
|
|
132
|
+
<td class="num zero">0</td>
|
|
133
|
+
<td class="num">376ms</td>
|
|
134
|
+
<td class="num">479ms</td>
|
|
135
|
+
<td class="barcell"><div class="bar" style="width:100.0%"></div></td>
|
|
136
|
+
</tr>
|
|
137
|
+
|
|
138
|
+
<tr class="">
|
|
139
|
+
<td class="mono name">openConversation</td>
|
|
140
|
+
<td class="num">17</td>
|
|
141
|
+
<td class="num zero">0</td>
|
|
142
|
+
<td class="num">86ms</td>
|
|
143
|
+
<td class="num">129ms</td>
|
|
144
|
+
<td class="barcell"><div class="bar" style="width:26.9%"></div></td>
|
|
145
|
+
</tr>
|
|
146
|
+
|
|
147
|
+
<tr class="">
|
|
148
|
+
<td class="mono name">sendMessage</td>
|
|
149
|
+
<td class="num">17</td>
|
|
150
|
+
<td class="num zero">0</td>
|
|
151
|
+
<td class="num">62ms</td>
|
|
152
|
+
<td class="num">79ms</td>
|
|
153
|
+
<td class="barcell"><div class="bar" style="width:16.5%"></div></td>
|
|
154
|
+
</tr>
|
|
155
|
+
|
|
156
|
+
<tr class="">
|
|
157
|
+
<td class="mono name">post</td>
|
|
158
|
+
<td class="num">15</td>
|
|
159
|
+
<td class="num zero">0</td>
|
|
160
|
+
<td class="num">92ms</td>
|
|
161
|
+
<td class="num">111ms</td>
|
|
162
|
+
<td class="barcell"><div class="bar" style="width:23.2%"></div></td>
|
|
163
|
+
</tr>
|
|
164
|
+
|
|
165
|
+
<tr class="">
|
|
166
|
+
<td class="mono name">comment</td>
|
|
167
|
+
<td class="num">8</td>
|
|
168
|
+
<td class="num zero">0</td>
|
|
169
|
+
<td class="num">69ms</td>
|
|
170
|
+
<td class="num">85ms</td>
|
|
171
|
+
<td class="barcell"><div class="bar" style="width:17.7%"></div></td>
|
|
172
|
+
</tr>
|
|
173
|
+
|
|
174
|
+
<tr class="">
|
|
175
|
+
<td class="mono name">createUser</td>
|
|
176
|
+
<td class="num">6</td>
|
|
177
|
+
<td class="num zero">0</td>
|
|
178
|
+
<td class="num">108ms</td>
|
|
179
|
+
<td class="num">192ms</td>
|
|
180
|
+
<td class="barcell"><div class="bar" style="width:40.1%"></div></td>
|
|
181
|
+
</tr>
|
|
182
|
+
|
|
183
|
+
<tr class="">
|
|
184
|
+
<td class="mono name">setProfile</td>
|
|
185
|
+
<td class="num">6</td>
|
|
186
|
+
<td class="num zero">0</td>
|
|
187
|
+
<td class="num">61ms</td>
|
|
188
|
+
<td class="num">64ms</td>
|
|
189
|
+
<td class="barcell"><div class="bar" style="width:13.4%"></div></td>
|
|
190
|
+
</tr>
|
|
191
|
+
|
|
192
|
+
<tr class="">
|
|
193
|
+
<td class="mono name">deleteUser</td>
|
|
194
|
+
<td class="num">6</td>
|
|
195
|
+
<td class="num zero">0</td>
|
|
196
|
+
<td class="num">117ms</td>
|
|
197
|
+
<td class="num">132ms</td>
|
|
198
|
+
<td class="barcell"><div class="bar" style="width:27.6%"></div></td>
|
|
199
|
+
</tr>
|
|
200
|
+
</tbody>
|
|
201
|
+
</table>
|
|
202
|
+
</div>
|
|
203
|
+
</section>
|
|
204
|
+
|
|
205
|
+
<section>
|
|
206
|
+
<h2>What the population did</h2>
|
|
207
|
+
<div class="panel">
|
|
208
|
+
1.8 km travelled · 15 posts ·
|
|
209
|
+
18 likes · 8 comments ·
|
|
210
|
+
17 messages · 0 group joins
|
|
211
|
+
</div>
|
|
212
|
+
</section>
|
|
213
|
+
|
|
214
|
+
<section>
|
|
215
|
+
<h2>Not tested — adapter implements 12/13</h2>
|
|
216
|
+
<div class="panel"><ul class="cov"><li><span class="mono">refreshSession</span><span class="would">would have tested token refresh — without it, any run longer than your token lifetime collapses and looks like your API failing</span></li></ul></div>
|
|
217
|
+
</section>
|
|
218
|
+
|
|
219
|
+
<section>
|
|
220
|
+
<h2>Cleanup</h2>
|
|
221
|
+
<div><div class="chip ok">Cleanup complete — 6 accounts removed</div></div>
|
|
222
|
+
</section>
|
|
223
|
+
|
|
224
|
+
<footer>
|
|
225
|
+
<span>Generated by Populace 0.1.0 · adapter <span class="mono">./adapters/demo.mjs</span></span>
|
|
226
|
+
<span>Simulated people are generated from patterns. This report shows whether your app
|
|
227
|
+
<strong>works</strong> — not whether anyone wants it.</span>
|
|
228
|
+
</footer>
|
|
229
|
+
|
|
230
|
+
</div></body></html>
|
|
@@ -0,0 +1,219 @@
|
|
|
1
|
+
{
|
|
2
|
+
"populace": {
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"generatedAt": "2026-08-09T14:09:59.998Z"
|
|
5
|
+
},
|
|
6
|
+
"run": {
|
|
7
|
+
"app": "Demo App",
|
|
8
|
+
"adapter": "./adapters/demo.mjs",
|
|
9
|
+
"environment": "test",
|
|
10
|
+
"startedAt": "2026-08-09T14:08:41.139Z",
|
|
11
|
+
"durationMs": 78860,
|
|
12
|
+
"population": {
|
|
13
|
+
"agents": 6,
|
|
14
|
+
"cities": [
|
|
15
|
+
"manila",
|
|
16
|
+
"mumbai"
|
|
17
|
+
],
|
|
18
|
+
"tickSeconds": 2,
|
|
19
|
+
"minutes": 1
|
|
20
|
+
}
|
|
21
|
+
},
|
|
22
|
+
"verdict": {
|
|
23
|
+
"status": "problems-found",
|
|
24
|
+
"problems": [
|
|
25
|
+
"6 of 246 API calls failed (2.4%)."
|
|
26
|
+
],
|
|
27
|
+
"failingMethods": [
|
|
28
|
+
{
|
|
29
|
+
"method": "like",
|
|
30
|
+
"failureRate": 0.25,
|
|
31
|
+
"topError": "new row violates row-level security policy \"post_likes_insert\""
|
|
32
|
+
}
|
|
33
|
+
]
|
|
34
|
+
},
|
|
35
|
+
"population": {
|
|
36
|
+
"requested": 6,
|
|
37
|
+
"signedIn": 6,
|
|
38
|
+
"signupFailures": []
|
|
39
|
+
},
|
|
40
|
+
"engineErrors": [],
|
|
41
|
+
"activity": {
|
|
42
|
+
"distanceKm": 1.8,
|
|
43
|
+
"posts": 15,
|
|
44
|
+
"likes": 18,
|
|
45
|
+
"comments": 8,
|
|
46
|
+
"messages": 17,
|
|
47
|
+
"groupJoins": 0
|
|
48
|
+
},
|
|
49
|
+
"api": {
|
|
50
|
+
"calls": 246,
|
|
51
|
+
"failures": 6,
|
|
52
|
+
"failureRate": 0.024390243902439025,
|
|
53
|
+
"durationMs": 78858,
|
|
54
|
+
"methods": [
|
|
55
|
+
{
|
|
56
|
+
"method": "like",
|
|
57
|
+
"calls": 24,
|
|
58
|
+
"failures": 6,
|
|
59
|
+
"failureRate": 0.25,
|
|
60
|
+
"latencyMs": {
|
|
61
|
+
"p50": 48,
|
|
62
|
+
"p95": 65,
|
|
63
|
+
"p99": 74,
|
|
64
|
+
"max": 74
|
|
65
|
+
},
|
|
66
|
+
"errors": [
|
|
67
|
+
{
|
|
68
|
+
"message": "new row violates row-level security policy \"post_likes_insert\"",
|
|
69
|
+
"count": 6
|
|
70
|
+
}
|
|
71
|
+
]
|
|
72
|
+
},
|
|
73
|
+
{
|
|
74
|
+
"method": "reportLocation",
|
|
75
|
+
"calls": 114,
|
|
76
|
+
"failures": 0,
|
|
77
|
+
"failureRate": 0,
|
|
78
|
+
"latencyMs": {
|
|
79
|
+
"p50": 31,
|
|
80
|
+
"p95": 48,
|
|
81
|
+
"p99": 51,
|
|
82
|
+
"max": 53
|
|
83
|
+
},
|
|
84
|
+
"errors": []
|
|
85
|
+
},
|
|
86
|
+
{
|
|
87
|
+
"method": "recentPostsByOthers",
|
|
88
|
+
"calls": 33,
|
|
89
|
+
"failures": 0,
|
|
90
|
+
"failureRate": 0,
|
|
91
|
+
"latencyMs": {
|
|
92
|
+
"p50": 376,
|
|
93
|
+
"p95": 479,
|
|
94
|
+
"p99": 488,
|
|
95
|
+
"max": 488
|
|
96
|
+
},
|
|
97
|
+
"errors": []
|
|
98
|
+
},
|
|
99
|
+
{
|
|
100
|
+
"method": "openConversation",
|
|
101
|
+
"calls": 17,
|
|
102
|
+
"failures": 0,
|
|
103
|
+
"failureRate": 0,
|
|
104
|
+
"latencyMs": {
|
|
105
|
+
"p50": 86,
|
|
106
|
+
"p95": 129,
|
|
107
|
+
"p99": 129,
|
|
108
|
+
"max": 129
|
|
109
|
+
},
|
|
110
|
+
"errors": []
|
|
111
|
+
},
|
|
112
|
+
{
|
|
113
|
+
"method": "sendMessage",
|
|
114
|
+
"calls": 17,
|
|
115
|
+
"failures": 0,
|
|
116
|
+
"failureRate": 0,
|
|
117
|
+
"latencyMs": {
|
|
118
|
+
"p50": 62,
|
|
119
|
+
"p95": 79,
|
|
120
|
+
"p99": 79,
|
|
121
|
+
"max": 79
|
|
122
|
+
},
|
|
123
|
+
"errors": []
|
|
124
|
+
},
|
|
125
|
+
{
|
|
126
|
+
"method": "post",
|
|
127
|
+
"calls": 15,
|
|
128
|
+
"failures": 0,
|
|
129
|
+
"failureRate": 0,
|
|
130
|
+
"latencyMs": {
|
|
131
|
+
"p50": 92,
|
|
132
|
+
"p95": 111,
|
|
133
|
+
"p99": 111,
|
|
134
|
+
"max": 111
|
|
135
|
+
},
|
|
136
|
+
"errors": []
|
|
137
|
+
},
|
|
138
|
+
{
|
|
139
|
+
"method": "comment",
|
|
140
|
+
"calls": 8,
|
|
141
|
+
"failures": 0,
|
|
142
|
+
"failureRate": 0,
|
|
143
|
+
"latencyMs": {
|
|
144
|
+
"p50": 69,
|
|
145
|
+
"p95": 85,
|
|
146
|
+
"p99": 85,
|
|
147
|
+
"max": 85
|
|
148
|
+
},
|
|
149
|
+
"errors": []
|
|
150
|
+
},
|
|
151
|
+
{
|
|
152
|
+
"method": "createUser",
|
|
153
|
+
"calls": 6,
|
|
154
|
+
"failures": 0,
|
|
155
|
+
"failureRate": 0,
|
|
156
|
+
"latencyMs": {
|
|
157
|
+
"p50": 108,
|
|
158
|
+
"p95": 192,
|
|
159
|
+
"p99": 192,
|
|
160
|
+
"max": 192
|
|
161
|
+
},
|
|
162
|
+
"errors": []
|
|
163
|
+
},
|
|
164
|
+
{
|
|
165
|
+
"method": "setProfile",
|
|
166
|
+
"calls": 6,
|
|
167
|
+
"failures": 0,
|
|
168
|
+
"failureRate": 0,
|
|
169
|
+
"latencyMs": {
|
|
170
|
+
"p50": 61,
|
|
171
|
+
"p95": 64,
|
|
172
|
+
"p99": 64,
|
|
173
|
+
"max": 64
|
|
174
|
+
},
|
|
175
|
+
"errors": []
|
|
176
|
+
},
|
|
177
|
+
{
|
|
178
|
+
"method": "deleteUser",
|
|
179
|
+
"calls": 6,
|
|
180
|
+
"failures": 0,
|
|
181
|
+
"failureRate": 0,
|
|
182
|
+
"latencyMs": {
|
|
183
|
+
"p50": 117,
|
|
184
|
+
"p95": 132,
|
|
185
|
+
"p99": 132,
|
|
186
|
+
"max": 132
|
|
187
|
+
},
|
|
188
|
+
"errors": []
|
|
189
|
+
}
|
|
190
|
+
]
|
|
191
|
+
},
|
|
192
|
+
"coverage": {
|
|
193
|
+
"label": "12/13",
|
|
194
|
+
"implemented": [
|
|
195
|
+
"createUser",
|
|
196
|
+
"setProfile",
|
|
197
|
+
"reportLocation",
|
|
198
|
+
"post",
|
|
199
|
+
"recentPostsByOthers",
|
|
200
|
+
"like",
|
|
201
|
+
"comment",
|
|
202
|
+
"openConversation",
|
|
203
|
+
"sendMessage",
|
|
204
|
+
"listGroups",
|
|
205
|
+
"joinGroup",
|
|
206
|
+
"deleteUser"
|
|
207
|
+
],
|
|
208
|
+
"notTested": [
|
|
209
|
+
{
|
|
210
|
+
"method": "refreshSession",
|
|
211
|
+
"wouldHaveTested": "token refresh — without it, any run longer than your token lifetime collapses and looks like your API failing"
|
|
212
|
+
}
|
|
213
|
+
]
|
|
214
|
+
},
|
|
215
|
+
"cleanup": {
|
|
216
|
+
"removed": 6,
|
|
217
|
+
"failed": []
|
|
218
|
+
}
|
|
219
|
+
}
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
// Try Populace with no backend at all:
|
|
2
|
+
//
|
|
3
|
+
// node src/cli.mjs run --config examples/demo/populace.config.mjs --minutes 1
|
|
4
|
+
//
|
|
5
|
+
// The demo adapter fakes a small app in memory, with one slow endpoint and one
|
|
6
|
+
// failing one, so the report has something honest to show.
|
|
7
|
+
|
|
8
|
+
export default {
|
|
9
|
+
app: "Demo App",
|
|
10
|
+
adapter: "./adapters/demo.mjs",
|
|
11
|
+
environment: "test",
|
|
12
|
+
target: { url: "memory://demo" },
|
|
13
|
+
neverRunAgainst: ["https://api.example.com"],
|
|
14
|
+
population: {
|
|
15
|
+
agents: 6,
|
|
16
|
+
cities: ["manila", "mumbai"],
|
|
17
|
+
minutes: 1,
|
|
18
|
+
tickSeconds: 2,
|
|
19
|
+
},
|
|
20
|
+
identity: { phonePrefix: "0900" },
|
|
21
|
+
report: { path: "populace-report.json" },
|
|
22
|
+
};
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Populace against a plain REST API
|
|
2
|
+
|
|
3
|
+
The reason this directory exists: Populace had only ever been pointed at one
|
|
4
|
+
real backend, and that backend was ours. A testing tool that has only tested its
|
|
5
|
+
author's own app has not been shown to be portable — it has been shown to work
|
|
6
|
+
once.
|
|
7
|
+
|
|
8
|
+
This is a second backend, deliberately unlike the first, with nothing to sign up
|
|
9
|
+
for.
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
node examples/rest-api/server.mjs # terminal 1
|
|
13
|
+
node src/cli.mjs run --config examples/rest-api/populace.config.mjs # terminal 2
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
## Result
|
|
17
|
+
|
|
18
|
+
```
|
|
19
|
+
POPULACE REPORT — REST API demo
|
|
20
|
+
test · 6 people · 124.7s
|
|
21
|
+
────────────────────────────────────────────────────
|
|
22
|
+
✔ No failures across 430 API calls.
|
|
23
|
+
────────────────────────────────────────────────────
|
|
24
|
+
4 km · 21 posts · 64 likes · 25 comments · 26 messages · 2 joins
|
|
25
|
+
✔ Cleanup complete — 6 accounts removed.
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
13/13 contract methods, every one exercised.
|
|
29
|
+
|
|
30
|
+
## Why this proves something
|
|
31
|
+
|
|
32
|
+
The server disagrees with Buzz Buzz on every axis that could hide an assumption:
|
|
33
|
+
|
|
34
|
+
| | Buzz Buzz | this server |
|
|
35
|
+
|---|---|---|
|
|
36
|
+
| transport | `supabase-js` client | plain `fetch` |
|
|
37
|
+
| ids | UUID strings | **integers** |
|
|
38
|
+
| errors | `{data, error}` tuples | HTTP status + `{error}` |
|
|
39
|
+
| auth | session on a client object | **bearer token in a header** |
|
|
40
|
+
| list endpoints | rows | **bare id arrays** |
|
|
41
|
+
| enforcement | row-level security, in the database | application code, in the handler |
|
|
42
|
+
|
|
43
|
+
The engine cannot tell them apart. That is the claim, and this is the test of it.
|
|
44
|
+
|
|
45
|
+
## Two bugs this found in Populace itself
|
|
46
|
+
|
|
47
|
+
Neither was theoretical. Both were found by pointing the tool at something new,
|
|
48
|
+
and both would have hit the first stranger who tried it.
|
|
49
|
+
|
|
50
|
+
**1. The engine could not read a bare id.** It did `target.id` on whatever
|
|
51
|
+
`recentPostsByOthers` returned, so an API answering with `[1, 2, 3]` produced
|
|
52
|
+
`undefined` on every `like`. `smoke` had always been tolerant of both shapes —
|
|
53
|
+
so an adapter like this one **passed the smoke test and then failed on the first
|
|
54
|
+
tick of a real run**, which is exactly the case smoke exists to rule out.
|
|
55
|
+
|
|
56
|
+
**2. `smoke` called `createUser` with a different object than the engine.** The
|
|
57
|
+
contract documents `{name, phone, persona, index}`; smoke passed a flat persona
|
|
58
|
+
that also carried a `password`. An adapter written against either shape passed
|
|
59
|
+
one check and failed the other. This one only became visible because a run and
|
|
60
|
+
a smoke test disagreed about the same new backend.
|
|
61
|
+
|
|
62
|
+
Both are now pinned by self-tests — including one that fails if smoke and the
|
|
63
|
+
engine ever drift apart on that argument again.
|
|
64
|
+
|
|
65
|
+
## Using it for your own API
|
|
66
|
+
|
|
67
|
+
If your backend speaks HTTP and JSON, copy `adapter.mjs` and change the URLs.
|
|
68
|
+
The parts worth keeping:
|
|
69
|
+
|
|
70
|
+
- **One `call()` helper** that turns a non-2xx into a thrown `Error`. Never
|
|
71
|
+
return quietly on failure — a swallowed error makes the report blame a later
|
|
72
|
+
method for an earlier fault.
|
|
73
|
+
- **Error messages that describe the fault, not the request.** `409 on POST
|
|
74
|
+
/likes` groups into one line; including the post id turns one bug into fifty.
|
|
75
|
+
- **`signIn`**, so `populace clean` can check for leftovers without creating
|
|
76
|
+
accounts to find out whether they exist.
|
|
77
|
+
- **`refreshSession`**, if your tokens expire. This server's last 15 minutes on
|
|
78
|
+
purpose: an adapter that skips it collapses on a longer run and the report
|
|
79
|
+
blames the API.
|
|
80
|
+
- **A password constant.** The contract never hands you one, because only you
|
|
81
|
+
know what your API accepts. It must be stable, or re-runs create new accounts
|
|
82
|
+
instead of reusing them.
|
|
83
|
+
|
|
84
|
+
The server binds to `127.0.0.1` and holds everything in memory. It is a fixture,
|
|
85
|
+
not something to deploy.
|