@gigzen/populace 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +661 -0
- package/README.md +258 -0
- package/adapters/buzzbuzz.mjs +247 -0
- package/adapters/contract.md +164 -0
- package/adapters/template-rest.mjs +192 -0
- package/adapters/template.mjs +80 -0
- package/examples/buzzbuzz/populace-report.html +245 -0
- package/examples/buzzbuzz/populace-report.json +280 -0
- package/examples/buzzbuzz/populace.config.mjs +51 -0
- package/examples/buzzbuzz/run-test.ps1 +61 -0
- package/examples/demo/adapters/demo.mjs +90 -0
- package/examples/demo/populace-report.html +230 -0
- package/examples/demo/populace-report.json +219 -0
- package/examples/demo/populace.config.mjs +22 -0
- package/examples/rest-api/README.md +85 -0
- package/examples/rest-api/adapter.mjs +166 -0
- package/examples/rest-api/populace.config.mjs +40 -0
- package/examples/rest-api/server.mjs +247 -0
- package/examples/token-expiry/expiry-demo.mjs +119 -0
- package/package.json +56 -0
- package/populace.config.example.mjs +65 -0
- package/src/cli.mjs +591 -0
- package/src/config.mjs +186 -0
- package/src/contract.mjs +130 -0
- package/src/diagnose.mjs +40 -0
- package/src/engine/agent.mjs +264 -0
- package/src/engine/geo.mjs +59 -0
- package/src/engine/index.mjs +4 -0
- package/src/engine/personas.mjs +115 -0
- package/src/engine/world.mjs +120 -0
- package/src/html-report.mjs +218 -0
- package/src/index.mjs +38 -0
- package/src/instrument.mjs +299 -0
- package/src/net.mjs +175 -0
- package/src/report.mjs +251 -0
- package/src/selftest.mjs +1369 -0
- package/src/smoke.mjs +274 -0
- package/src/version.mjs +24 -0
package/README.md
ADDED
|
@@ -0,0 +1,258 @@
|
|
|
1
|
+
# Populace
|
|
2
|
+
|
|
3
|
+
**A simulated population that uses your app through its real API — so you can
|
|
4
|
+
test what needs more than one person.**
|
|
5
|
+
|
|
6
|
+
**Site:** https://shakhtar-sankur.github.io/populace/ ·
|
|
7
|
+
**Test report:** [the full engineering record](https://shakhtar-sankur.github.io/gigzen/test-report.html)
|
|
8
|
+
· A [Gigzen](https://shakhtar-sankur.github.io/gigzen/) product
|
|
9
|
+
|
|
10
|
+
Some bugs only exist when two people are using your app at the same time.
|
|
11
|
+
Presence, live sync, read receipts, notification fan-out, "does deleting this
|
|
12
|
+
remove it for everyone", and every permission rule you wrote — none of them can
|
|
13
|
+
be tested by one developer with one account, however carefully they tap through
|
|
14
|
+
every screen.
|
|
15
|
+
|
|
16
|
+
Populace gives you a few dozen believable people who sign up, move around a real
|
|
17
|
+
city, post, like, comment, message each other and join groups — as **real
|
|
18
|
+
authenticated users**, through **your own API**, with **your own permission rules
|
|
19
|
+
applying**. Then it hands you a report on what broke.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## It has done this to a real, finished app
|
|
24
|
+
|
|
25
|
+

|
|
26
|
+
|
|
27
|
+
That is the *second* run. The first one is the interesting one.
|
|
28
|
+
|
|
29
|
+
### What happened, in plain English
|
|
30
|
+
|
|
31
|
+
On **9 August 2026** we pointed Populace at **Buzz Buzz** — a gig-worker platform
|
|
32
|
+
on Android with a live Postgres backend, 17 tables and 48 row-level-security
|
|
33
|
+
policies. It was finished. It was signed. It had been through a full manual test
|
|
34
|
+
of every screen by the person who wrote it, and it had passed.
|
|
35
|
+
|
|
36
|
+
We started six simulated drivers: three in **Manila**, three in **Mumbai**. Each
|
|
37
|
+
one signed up for a real account, set a profile, started driving a plausible
|
|
38
|
+
route through real streets, and behaved like a person — posting about traffic,
|
|
39
|
+
reading the feed, liking and commenting on what other drivers posted, opening
|
|
40
|
+
conversations, sending messages, joining a group. Nobody told Populace where the
|
|
41
|
+
bugs were. Nobody told it what to look for. It just used the app.
|
|
42
|
+
|
|
43
|
+
**Three and a half minutes later it had found five bugs.** The app that had
|
|
44
|
+
passed a full manual test could not create a working account.
|
|
45
|
+
|
|
46
|
+
### The five, one at a time
|
|
47
|
+
|
|
48
|
+
**1. Signing up created no profile — every new user was broken.**
|
|
49
|
+
A privacy fix earlier that day had restricted write access on the `profiles`
|
|
50
|
+
table to a named list of columns, so that phone numbers could never be read by
|
|
51
|
+
other users. The signup code used an *upsert*. In Postgres, `INSERT … ON CONFLICT
|
|
52
|
+
DO UPDATE` needs `SELECT` permission on every column it touches — and one of
|
|
53
|
+
those columns was, deliberately, unreadable. So the write was refused. Silently.
|
|
54
|
+
|
|
55
|
+
Every account created after that point existed in the auth system with no
|
|
56
|
+
profile row behind it. Then every post, every comment and every group join died
|
|
57
|
+
on a foreign key pointing back at the row that was never written.
|
|
58
|
+
|
|
59
|
+
*Why one person never sees this:* their own account already exists. You only hit
|
|
60
|
+
it on a **fresh signup**, and you only notice the damage when that new user tries
|
|
61
|
+
to do something. Populace creates six brand-new accounts every run, which is why
|
|
62
|
+
it hit the bug in the first fifteen seconds.
|
|
63
|
+
|
|
64
|
+
**2. Likes bounced back, at random.**
|
|
65
|
+
The `post_likes` table has an insert policy and a delete policy, and no `UPDATE`
|
|
66
|
+
policy — by design. The app used an upsert here too, so the second time anyone
|
|
67
|
+
liked a post, it became `ON CONFLICT DO UPDATE` and row-level security refused
|
|
68
|
+
it. The feed refreshes every 2.5 seconds, so a slightly stale "already liked"
|
|
69
|
+
state was completely ordinary, and one tap in four failed.
|
|
70
|
+
|
|
71
|
+
*Why one person never sees this:* with one account and one slow thumb, you rarely
|
|
72
|
+
double-like anything. Six people liking each other's posts on a 2.5-second poll
|
|
73
|
+
do it constantly.
|
|
74
|
+
|
|
75
|
+
**3. Editing your profile failed the same way.** Same upsert, same unreadable
|
|
76
|
+
column, same silent refusal.
|
|
77
|
+
|
|
78
|
+
**4 and 5. Two faults in Populace's own reference adapter.**
|
|
79
|
+
One passed an RPC argument under the wrong name. The other is the one worth
|
|
80
|
+
dwelling on: it **ignored the error returned by a write**. Because that failure
|
|
81
|
+
was swallowed at signup, the report blamed a *later* method for it — so the run
|
|
82
|
+
said "post failed 14 times" when the truth was "the profile row was never
|
|
83
|
+
created."
|
|
84
|
+
|
|
85
|
+
An unchecked error is the precise fault this tool exists to catch, and it was
|
|
86
|
+
sitting in our own code. Making that one line throw changed the report from three
|
|
87
|
+
confusing symptoms into one sentence naming the real cause: *profile row not
|
|
88
|
+
created: permission denied for table profiles.*
|
|
89
|
+
|
|
90
|
+
### The rule worth taking away
|
|
91
|
+
|
|
92
|
+
> **You cannot upsert a column you cannot select.**
|
|
93
|
+
|
|
94
|
+
Three of the five bugs were the same mistake wearing different clothes. It is
|
|
95
|
+
invisible in code review, invisible in a single-user walkthrough, and obvious
|
|
96
|
+
within seconds to six users signing up at once.
|
|
97
|
+
|
|
98
|
+
### After the fixes
|
|
99
|
+
|
|
100
|
+
All five were fixed and the run repeated: the report at the top of this page.
|
|
101
|
+
**No failures across 400 API calls.** Six accounts created, six accounts deleted,
|
|
102
|
+
nothing left behind.
|
|
103
|
+
|
|
104
|
+
**What this does not claim.** Six users for three minutes is a **correctness run,
|
|
105
|
+
not a load test**, and it ran against that project while it was still empty. The
|
|
106
|
+
latencies above are what six concurrent users saw and nothing more. Populace has
|
|
107
|
+
been pointed at exactly one real backend so far, and that backend was ours — the
|
|
108
|
+
next one should be someone else's.
|
|
109
|
+
|
|
110
|
+
---
|
|
111
|
+
|
|
112
|
+
## Try it in ten seconds
|
|
113
|
+
|
|
114
|
+
No backend, no signup, nothing of yours touched — the demo adapter fakes a small
|
|
115
|
+
app in memory.
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
populace demo
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
The demo app has a real bug in it: a row-level-security policy that rejects
|
|
122
|
+
likes. Watch the report find it, name the policy, and exit non-zero. That exit
|
|
123
|
+
code is the whole point — the run fails your build rather than telling you it
|
|
124
|
+
went fine.
|
|
125
|
+
|
|
126
|
+
Two files land in whatever directory you ran it from: `populace-report.json` for
|
|
127
|
+
CI, and `populace-report.html`, which is one self-contained page you can email
|
|
128
|
+
to someone who was not watching your terminal.
|
|
129
|
+
|
|
130
|
+
---
|
|
131
|
+
|
|
132
|
+
## Point it at your own app
|
|
133
|
+
|
|
134
|
+
```bash
|
|
135
|
+
populace init # scaffolds populace.config.mjs + adapters/my-app.mjs
|
|
136
|
+
populace doctor # checks config, reachability and coverage WITHOUT running
|
|
137
|
+
populace run
|
|
138
|
+
populace clean # removes every account a run created
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
You write **one adapter** — thirteen small methods, each answering "how does
|
|
142
|
+
this happen in my app?". Two are required (`createUser`, `deleteUser`);
|
|
143
|
+
everything else is optional and anything you skip is reported as untested rather
|
|
144
|
+
than quietly passing. Full spec in [adapters/contract.md](adapters/contract.md),
|
|
145
|
+
and [adapters/buzzbuzz.mjs](adapters/buzzbuzz.mjs) is a complete real-world
|
|
146
|
+
example in ~150 lines.
|
|
147
|
+
|
|
148
|
+
`populace doctor` on a fresh scaffold says `2/13` and refuses to run — a method
|
|
149
|
+
that exists but does nothing is not coverage.
|
|
150
|
+
|
|
151
|
+
### Check the adapter before you run
|
|
152
|
+
|
|
153
|
+
`doctor` tells you which methods exist. `smoke` tells you whether they work:
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
populace smoke
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
It creates one user, calls every method you implemented once, checks what came
|
|
160
|
+
back against what the contract promises, and deletes the user again. Seconds,
|
|
161
|
+
not minutes — and it reports on *your adapter*, not your app, which is the
|
|
162
|
+
question you actually have while writing one.
|
|
163
|
+
|
|
164
|
+
A full run against a subtly wrong adapter spends five minutes producing a report
|
|
165
|
+
about nothing. This is the step that stops that.
|
|
166
|
+
|
|
167
|
+
**If your API uses expiring tokens, implement `refreshSession`.** Without it,
|
|
168
|
+
any run longer than your token lifetime collapses at once and the report blames
|
|
169
|
+
your API for what were really expired tokens — and the run cannot even delete
|
|
170
|
+
its own accounts, stranding simulated users in your environment. See it happen:
|
|
171
|
+
`node examples/token-expiry/expiry-demo.mjs`
|
|
172
|
+
|
|
173
|
+
## The safety guard
|
|
174
|
+
|
|
175
|
+
Populace creates real accounts and writes real rows. Pointed at production it
|
|
176
|
+
would put invented people in front of paying customers. So it refuses to start
|
|
177
|
+
in three independent ways:
|
|
178
|
+
|
|
179
|
+
1. **`environment` must declare a non-production value.** Opt in, never assumed.
|
|
180
|
+
2. **`neverRunAgainst` is checked against every string in your target** — however
|
|
181
|
+
deeply nested. One match and the run is refused; no flag overrides it.
|
|
182
|
+
3. **An empty denylist warns loudly**, because "I forgot to fill that in" is the
|
|
183
|
+
likeliest version of this mistake.
|
|
184
|
+
|
|
185
|
+
Refusals exit non-zero, so CI catches them too.
|
|
186
|
+
|
|
187
|
+
The same principle runs through the contract: **go through the front door**. Use
|
|
188
|
+
the API your app actually uses — not admin keys, not service-role credentials,
|
|
189
|
+
not direct database writes. A simulation that bypasses your permission rules
|
|
190
|
+
proves nothing about whether they work, and permission bugs are exactly what a
|
|
191
|
+
multi-user simulation is best at finding.
|
|
192
|
+
|
|
193
|
+
---
|
|
194
|
+
|
|
195
|
+
## What you get
|
|
196
|
+
|
|
197
|
+
- **A populated app** instead of an empty one — the cold-start problem, solved for demos and for judging your own UX
|
|
198
|
+
- **Multi-user paths exercised** without recruiting humans: presence, receipts, realtime fan-out, membership counts
|
|
199
|
+
- **Permission rules tested by users who genuinely have different identities**
|
|
200
|
+
- **Latency per endpoint** (p50/p95/p99/max) under N concurrent users
|
|
201
|
+
- **Failures grouped by shape**, not exact text, so one bug is one line rather than fifty
|
|
202
|
+
- **Account deletion actually tested** — the path almost nobody exercises and the one regulators ask about
|
|
203
|
+
- **`populace-report.json`** for CI; the run exits non-zero when problems are found
|
|
204
|
+
- **`populace-report.html`** beside it — one self-contained file, no scripts and nothing
|
|
205
|
+
fetched from the network, for the people who were not watching the terminal
|
|
206
|
+
|
|
207
|
+
### A run always finishes
|
|
208
|
+
|
|
209
|
+
Every call into your adapter has a deadline (`timeoutMs`, 20s by default). Past
|
|
210
|
+
it, Populace stops waiting, records a timeout against that endpoint, and the
|
|
211
|
+
other agents carry on.
|
|
212
|
+
|
|
213
|
+
This matters more than it sounds. We found it the hard way: a real run against a
|
|
214
|
+
flaky link froze two minutes in and sat there silently until something outside
|
|
215
|
+
killed it nine minutes later. No report, no error — just a progress line that
|
|
216
|
+
stopped moving. The API under test was fine; one dead socket had taken the whole
|
|
217
|
+
run with it.
|
|
218
|
+
|
|
219
|
+
A hang is the worst outcome a testing tool can produce, because it does not look
|
|
220
|
+
like a failure. It looks like nothing, and the natural conclusion is that the
|
|
221
|
+
tool is broken. So now a slow or unresponsive endpoint becomes **a line in the
|
|
222
|
+
report**, which is a finding you can act on — and the run still ends with a
|
|
223
|
+
verdict. Set `timeoutMs: 0` if your adapter does long work on purpose.
|
|
224
|
+
|
|
225
|
+
---
|
|
226
|
+
|
|
227
|
+
## What it will not tell you
|
|
228
|
+
|
|
229
|
+
**Whether people want your product.** These people are generated from patterns.
|
|
230
|
+
They will never surprise you the way a real customer does, they cannot tell you
|
|
231
|
+
your onboarding is confusing or your pricing is wrong, and they are least
|
|
232
|
+
accurate for exactly the users least represented online.
|
|
233
|
+
|
|
234
|
+
Use Populace to prove your app **works**. Use real people to decide what to
|
|
235
|
+
**build**. A report full of green ticks means your API held up — not that anyone
|
|
236
|
+
wants what you made.
|
|
237
|
+
|
|
238
|
+
---
|
|
239
|
+
|
|
240
|
+
## Design notes
|
|
241
|
+
|
|
242
|
+
- **The engine never learns your app.** `src/engine/` knows how to be a person;
|
|
243
|
+
everything app-specific lives in an adapter. If app logic ever leaks into the
|
|
244
|
+
engine, Populace has collapsed back into a test script.
|
|
245
|
+
- **Zero runtime dependencies.** Adapters bring their own.
|
|
246
|
+
- **Deterministic identities.** Agent *n* always gets the same phone number, so
|
|
247
|
+
re-runs reuse accounts instead of piling up new ones — and `populace clean`
|
|
248
|
+
can find them after a run that crashed halfway.
|
|
249
|
+
- **Cheap by default.** Scripted agents cost nothing, so hundreds can run at
|
|
250
|
+
once. If you later want genuinely emergent conversation, put an LLM behind a
|
|
251
|
+
tier — a handful of expensive agents among many cheap ones, the way games vary
|
|
252
|
+
NPC detail.
|
|
253
|
+
- **Concurrent, not sequential.** Everyone acts at once, because sequential
|
|
254
|
+
agents never surface a race condition.
|
|
255
|
+
|
|
256
|
+
```bash
|
|
257
|
+
npm test # runs the whole product against an in-memory app — no backend needed
|
|
258
|
+
```
|
|
@@ -0,0 +1,247 @@
|
|
|
1
|
+
// Adapter: Buzz Buzz — a gig-driver tracking app on Supabase.
|
|
2
|
+
//
|
|
3
|
+
// Populace's first customer, and the reference implementation. Read this
|
|
4
|
+
// alongside contract.md to see what a complete adapter looks like: it is ~140
|
|
5
|
+
// lines, and it is the ONLY file in the system that knows these table names.
|
|
6
|
+
//
|
|
7
|
+
// Every call goes through the normal client with the publishable key, as a real
|
|
8
|
+
// authenticated user, so row-level security and triggers apply exactly as they
|
|
9
|
+
// do for a real driver. No service-role key, no direct database access.
|
|
10
|
+
|
|
11
|
+
import { createClient } from "@supabase/supabase-js";
|
|
12
|
+
|
|
13
|
+
const PASSWORD = "SimDriver!2026";
|
|
14
|
+
// Matches the app's own phone→email scheme (SupabaseService.phoneToEmail).
|
|
15
|
+
const phoneToEmail = (phone) => `${String(phone).replace(/\D/g, "") || "driver"}@masaya.local`;
|
|
16
|
+
|
|
17
|
+
export function createAdapter(target) {
|
|
18
|
+
const url = target.url?.replace(/\/$/, "");
|
|
19
|
+
const key = target.key;
|
|
20
|
+
|
|
21
|
+
if (!url || !key) {
|
|
22
|
+
throw new Error(
|
|
23
|
+
"buzzbuzz adapter needs target.url and target.key.\n" +
|
|
24
|
+
" Set BUZZBUZZ_TEST_URL and BUZZBUZZ_TEST_KEY in your environment.",
|
|
25
|
+
);
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
return {
|
|
29
|
+
name: "buzzbuzz",
|
|
30
|
+
|
|
31
|
+
// supabase-js RETURNS network errors rather than throwing, so without this
|
|
32
|
+
// a wrong URL looks identical to "that account doesn't exist" — and a run
|
|
33
|
+
// would cheerfully report failures that were really a typo.
|
|
34
|
+
async healthCheck() {
|
|
35
|
+
const res = await fetch(`${url}/auth/v1/health`, {
|
|
36
|
+
headers: { apikey: key },
|
|
37
|
+
signal: AbortSignal.timeout(12000),
|
|
38
|
+
});
|
|
39
|
+
if (res.status >= 500) throw new Error(`server returned ${res.status}`);
|
|
40
|
+
},
|
|
41
|
+
|
|
42
|
+
/**
|
|
43
|
+
* Cleanup capability: does this identity exist, without creating it?
|
|
44
|
+
*
|
|
45
|
+
* `clean` used to reach accounts through createUser, which signs UP when
|
|
46
|
+
* the identity is absent — so tidying an already-clean project wrote a row
|
|
47
|
+
* to the customer's auth table for every agent, purely to prove the table
|
|
48
|
+
* was empty. This is the read-only path.
|
|
49
|
+
*
|
|
50
|
+
* Returns the user when present, null when definitively absent. A transport
|
|
51
|
+
* failure throws, because "I could not look" must never be recorded as
|
|
52
|
+
* "there was nothing there".
|
|
53
|
+
*/
|
|
54
|
+
async signIn({ phone }) {
|
|
55
|
+
const client = createClient(url, key, {
|
|
56
|
+
auth: { persistSession: false, autoRefreshToken: false },
|
|
57
|
+
});
|
|
58
|
+
const { data, error } = await client.auth.signInWithPassword({
|
|
59
|
+
email: phoneToEmail(phone),
|
|
60
|
+
password: PASSWORD,
|
|
61
|
+
});
|
|
62
|
+
if (!error) return data.user ? { ...data.user, client } : null;
|
|
63
|
+
|
|
64
|
+
// Supabase answers a non-existent account and a wrong password with the
|
|
65
|
+
// same message. Both mean "no simulated account we can act on"; anything
|
|
66
|
+
// else is a real fault and must surface.
|
|
67
|
+
if (/invalid login credentials|email not confirmed/i.test(error.message)) return null;
|
|
68
|
+
throw new Error(error.message);
|
|
69
|
+
},
|
|
70
|
+
|
|
71
|
+
async createUser({ name, phone, persona }) {
|
|
72
|
+
const client = createClient(url, key, {
|
|
73
|
+
auth: { persistSession: false, autoRefreshToken: false },
|
|
74
|
+
});
|
|
75
|
+
const email = phoneToEmail(phone);
|
|
76
|
+
|
|
77
|
+
let { data, error } = await client.auth.signUp({
|
|
78
|
+
email,
|
|
79
|
+
password: PASSWORD,
|
|
80
|
+
options: { data: { full_name: name, phone } },
|
|
81
|
+
});
|
|
82
|
+
if (error) {
|
|
83
|
+
// Already exists from an earlier run — reuse it rather than piling up.
|
|
84
|
+
const signUpError = error;
|
|
85
|
+
({ data, error } = await client.auth.signInWithPassword({ email, password: PASSWORD }));
|
|
86
|
+
if (error) {
|
|
87
|
+
// Report BOTH, because the fallback's error is usually the misleading
|
|
88
|
+
// one. This threw a bare "Invalid login credentials" for two nights of
|
|
89
|
+
// debugging: the real cause was signUp being refused — a rate limit
|
|
90
|
+
// after many runs in one hour — and the sign-in then failing simply
|
|
91
|
+
// because the account had never been created. The message named the
|
|
92
|
+
// symptom and hid the cause, which sent the diagnosis the wrong way
|
|
93
|
+
// twice. Losing the first error to report the second is exactly the
|
|
94
|
+
// fault this tool exists to catch, in our own reference adapter.
|
|
95
|
+
throw new Error(`${error.message} (signup first failed: ${signUpError.message})`);
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
if (!data.user) throw new Error("no user returned");
|
|
99
|
+
|
|
100
|
+
// Check this. It used to be fire-and-forget, so when the profile row
|
|
101
|
+
// failed to insert the run carried on and every later post and group-join
|
|
102
|
+
// died on a foreign key instead — the report blamed `post` for a fault
|
|
103
|
+
// that happened during signup. An unchecked error is the exact thing this
|
|
104
|
+
// tool exists to catch, and it was in our own reference adapter.
|
|
105
|
+
// INSERT, not upsert — mirroring the app. An upsert touches `phone`, and
|
|
106
|
+
// ON CONFLICT DO UPDATE needs SELECT on the columns it touches, which
|
|
107
|
+
// privacy_lockdown deliberately removes for that column. A duplicate just
|
|
108
|
+
// means this persona signed up on an earlier run.
|
|
109
|
+
const { error: profileError } = await client.from("profiles").insert({
|
|
110
|
+
id: data.user.id,
|
|
111
|
+
full_name: name,
|
|
112
|
+
phone,
|
|
113
|
+
updated_at: new Date().toISOString(),
|
|
114
|
+
});
|
|
115
|
+
if (profileError && profileError.code !== "23505") {
|
|
116
|
+
throw new Error(`profile row not created: ${profileError.message}`);
|
|
117
|
+
}
|
|
118
|
+
|
|
119
|
+
return { id: data.user.id, client, persona };
|
|
120
|
+
},
|
|
121
|
+
|
|
122
|
+
// Supabase access tokens are short-lived (1 hour by default). We refresh
|
|
123
|
+
// EXPLICITLY rather than letting supabase-js do it in the background:
|
|
124
|
+
// autoRefreshToken is a timer we cannot see, whereas an explicit call is
|
|
125
|
+
// timed and counted like everything else, so a refresh that starts failing
|
|
126
|
+
// shows up in the report instead of quietly poisoning the whole run.
|
|
127
|
+
async refreshSession(user) {
|
|
128
|
+
const { data, error } = await user.client.auth.refreshSession();
|
|
129
|
+
if (error) throw new Error(error.message);
|
|
130
|
+
if (!data?.session) throw new Error("refresh returned no session");
|
|
131
|
+
},
|
|
132
|
+
|
|
133
|
+
async setProfile(user, persona) {
|
|
134
|
+
await user.client.from("driver_settings").upsert({
|
|
135
|
+
user_id: user.id,
|
|
136
|
+
active_app: persona.platform,
|
|
137
|
+
base_rate: persona.rate,
|
|
138
|
+
vehicle_type: persona.vehicle ?? "car",
|
|
139
|
+
share_stats: true,
|
|
140
|
+
updated_at: new Date().toISOString(),
|
|
141
|
+
});
|
|
142
|
+
},
|
|
143
|
+
|
|
144
|
+
async reportLocation(user, { lat, lng, distanceKm, earnings, platform }) {
|
|
145
|
+
await user.client.from("worker_locations").upsert({
|
|
146
|
+
user_id: user.id,
|
|
147
|
+
lat,
|
|
148
|
+
lng,
|
|
149
|
+
accuracy: 5 + Math.random() * 8,
|
|
150
|
+
active_app: platform,
|
|
151
|
+
today_distance_km: Number(distanceKm.toFixed(2)),
|
|
152
|
+
today_earnings: Number(earnings.toFixed(2)),
|
|
153
|
+
updated_at: new Date().toISOString(),
|
|
154
|
+
});
|
|
155
|
+
await user.client.from("route_points").insert({
|
|
156
|
+
user_id: user.id,
|
|
157
|
+
lat,
|
|
158
|
+
lng,
|
|
159
|
+
accuracy: 8,
|
|
160
|
+
active_app: platform,
|
|
161
|
+
recorded_at: new Date().toISOString(),
|
|
162
|
+
});
|
|
163
|
+
},
|
|
164
|
+
|
|
165
|
+
async post(user, text) {
|
|
166
|
+
const { data, error } = await user.client
|
|
167
|
+
.from("feed_posts")
|
|
168
|
+
.insert({ user_id: user.id, body: text })
|
|
169
|
+
.select("id")
|
|
170
|
+
.single();
|
|
171
|
+
if (error) throw error;
|
|
172
|
+
return data?.id;
|
|
173
|
+
},
|
|
174
|
+
|
|
175
|
+
async recentPostsByOthers(user, limit = 10) {
|
|
176
|
+
const { data } = await user.client
|
|
177
|
+
.from("feed_posts")
|
|
178
|
+
.select("id,user_id")
|
|
179
|
+
.neq("user_id", user.id)
|
|
180
|
+
.order("created_at", { ascending: false })
|
|
181
|
+
.limit(limit);
|
|
182
|
+
return (data ?? []).map((p) => ({ id: p.id, userId: p.user_id }));
|
|
183
|
+
},
|
|
184
|
+
|
|
185
|
+
async like(user, postId) {
|
|
186
|
+
// Mirrors the app: insert, and treat a duplicate as success. post_likes
|
|
187
|
+
// has no UPDATE policy, so an upsert on a repeat like is refused by RLS.
|
|
188
|
+
const { error } = await user.client
|
|
189
|
+
.from("post_likes")
|
|
190
|
+
.insert({ post_id: postId, user_id: user.id });
|
|
191
|
+
if (error && error.code !== "23505") throw error;
|
|
192
|
+
},
|
|
193
|
+
|
|
194
|
+
async comment(user, postId, text) {
|
|
195
|
+
const { error } = await user.client
|
|
196
|
+
.from("post_comments")
|
|
197
|
+
.insert({ post_id: postId, user_id: user.id, body: text });
|
|
198
|
+
if (error) throw error;
|
|
199
|
+
},
|
|
200
|
+
|
|
201
|
+
async openConversation(user, otherUserId) {
|
|
202
|
+
// The function signature is start_direct_thread(p_other uuid). PostgREST
|
|
203
|
+
// resolves RPCs by argument NAME, so passing other_user_id looked like a
|
|
204
|
+
// missing function rather than a wrong argument.
|
|
205
|
+
const { data, error } = await user.client.rpc("start_direct_thread", {
|
|
206
|
+
p_other: otherUserId,
|
|
207
|
+
});
|
|
208
|
+
if (error) throw error;
|
|
209
|
+
return data;
|
|
210
|
+
},
|
|
211
|
+
|
|
212
|
+
async sendMessage(user, conversationId, text) {
|
|
213
|
+
// chat_messages.id is a text primary key the client supplies — the app
|
|
214
|
+
// generates one per message, and the adapter was sending none at all.
|
|
215
|
+
const { error } = await user.client.from("chat_messages").insert({
|
|
216
|
+
id: `msg_${crypto.randomUUID()}`,
|
|
217
|
+
thread_id: conversationId,
|
|
218
|
+
sender_id: user.id,
|
|
219
|
+
body: text,
|
|
220
|
+
status: "sent",
|
|
221
|
+
});
|
|
222
|
+
if (error) throw error;
|
|
223
|
+
},
|
|
224
|
+
|
|
225
|
+
async listGroups(user) {
|
|
226
|
+
const { data } = await user.client.from("groups").select("id").limit(10);
|
|
227
|
+
return data ?? [];
|
|
228
|
+
},
|
|
229
|
+
|
|
230
|
+
async joinGroup(user, groupId) {
|
|
231
|
+
// Mirrors the app: ignoreDuplicates so this is ON CONFLICT DO NOTHING.
|
|
232
|
+
// A plain upsert takes the UPDATE path on a repeat join, and there is no
|
|
233
|
+
// UPDATE policy on group_members — which is the failure this adapter
|
|
234
|
+
// surfaced on a 644-call run.
|
|
235
|
+
const { error } = await user.client
|
|
236
|
+
.from("group_members")
|
|
237
|
+
.upsert({ group_id: groupId, user_id: user.id }, { ignoreDuplicates: true });
|
|
238
|
+
if (error) throw error;
|
|
239
|
+
},
|
|
240
|
+
|
|
241
|
+
async deleteUser(user) {
|
|
242
|
+
// Deliberately the app's OWN delete path, so the simulation exercises it.
|
|
243
|
+
const { error } = await user.client.rpc("delete_own_account");
|
|
244
|
+
if (error) throw error;
|
|
245
|
+
},
|
|
246
|
+
};
|
|
247
|
+
}
|
|
@@ -0,0 +1,164 @@
|
|
|
1
|
+
# Writing an adapter
|
|
2
|
+
|
|
3
|
+
The engine knows how to be a *person*: where they go, how chatty they are, when
|
|
4
|
+
they take a break, who they talk to. It knows nothing about your app.
|
|
5
|
+
|
|
6
|
+
An **adapter** is the translation layer. For each thing a simulated person can
|
|
7
|
+
do, it answers one question: *"how does that happen in my product?"*
|
|
8
|
+
|
|
9
|
+
```js
|
|
10
|
+
export function createAdapter(target, config) {
|
|
11
|
+
return { name: "your-app", /* methods below */ };
|
|
12
|
+
}
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
`target` is whatever you put in `target` in `populace.config.mjs`.
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## The contract
|
|
20
|
+
|
|
21
|
+
Thirteen methods. Two are required; the rest are optional, and anything you
|
|
22
|
+
leave out is skipped rather than failing. A read-only app can implement three
|
|
23
|
+
and get a useful run.
|
|
24
|
+
|
|
25
|
+
| Method | | What it exercises in your app |
|
|
26
|
+
|---|---|---|
|
|
27
|
+
| `createUser({name, phone, persona, index})` | **required** | sign-up, sign-in, first-contact |
|
|
28
|
+
| `setProfile(user, persona)` | | the settings a new user configures |
|
|
29
|
+
| `refreshSession(user)` | **read this** | token refresh — see below |
|
|
30
|
+
| `reportLocation(user, {lat, lng, distanceKm, earnings, platform})` | | high-frequency writes — the heaviest sustained load most apps take |
|
|
31
|
+
| `post(user, text)` → `postId` | | content creation |
|
|
32
|
+
| `recentPostsByOthers(user, limit)` → `[{id, userId}]` | | feed reads under concurrent writes, and whether permissions leak |
|
|
33
|
+
| `like(user, postId)` | | high-contention writes on shared rows |
|
|
34
|
+
| `comment(user, postId, text)` | | nested content and its notifications |
|
|
35
|
+
| `openConversation(user, otherUserId)` → `conversationId` | | conversation creation between accounts that have never met |
|
|
36
|
+
| `sendMessage(user, conversationId, text)` | | delivery, ordering, receipts, realtime fan-out |
|
|
37
|
+
| `listGroups(user)` → `[{id}]` | | shared-resource reads |
|
|
38
|
+
| `joinGroup(user, groupId)` | | membership writes and counter accuracy |
|
|
39
|
+
| `deleteUser(user)` | **required** | account deletion and cascade |
|
|
40
|
+
| `healthCheck()` | optional | lets `populace doctor` verify the target before a run |
|
|
41
|
+
|
|
42
|
+
`createUser` returns a **handle** — put the session, token or client on it. You
|
|
43
|
+
get that same object back as `user` on every later call.
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## Two rules
|
|
48
|
+
|
|
49
|
+
**1. Never point an adapter at production.**
|
|
50
|
+
|
|
51
|
+
Populace refuses in three independent ways — `environment` must declare a
|
|
52
|
+
non-production value, `neverRunAgainst` is checked against every string in your
|
|
53
|
+
target, and an empty denylist warns loudly. Your adapter should fail loudly too.
|
|
54
|
+
Simulated people appearing to real users is deception, not testing.
|
|
55
|
+
|
|
56
|
+
**2. Go through the front door.**
|
|
57
|
+
|
|
58
|
+
Use the same API your app uses. Not admin keys, not service-role credentials,
|
|
59
|
+
not direct database writes. A simulation that bypasses your permission rules
|
|
60
|
+
proves nothing about whether they work — and permission bugs are exactly what a
|
|
61
|
+
multi-user simulation is best at finding.
|
|
62
|
+
|
|
63
|
+
The same applies to `deleteUser`: prefer your app's own delete-account path over
|
|
64
|
+
a hard delete. It is the route almost nobody tests and the one regulators ask
|
|
65
|
+
about.
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## Sessions expire — implement `refreshSession`
|
|
70
|
+
|
|
71
|
+
If your API uses short-lived access tokens (most do), implement this. It is
|
|
72
|
+
optional only because some APIs don't need it.
|
|
73
|
+
|
|
74
|
+
Populace calls it every 30 minutes by default — tune with
|
|
75
|
+
`session.refreshEveryMinutes`. If it throws, the agent signs in again from
|
|
76
|
+
scratch rather than going quietly dead.
|
|
77
|
+
|
|
78
|
+
Skipping it on an app with a 1-hour token means any run longer than an hour
|
|
79
|
+
collapses at once, and the report blames **your API** for failures that were
|
|
80
|
+
really expired tokens. Worse: `deleteUser` needs a live session too, so the run
|
|
81
|
+
cannot clean up after itself and leaves simulated accounts stranded in your
|
|
82
|
+
environment.
|
|
83
|
+
|
|
84
|
+
Measured on a pretend app with a 3-second token, same population, same duration:
|
|
85
|
+
|
|
86
|
+
| | no `refreshSession` | with it |
|
|
87
|
+
|---|---|---|
|
|
88
|
+
| API calls failed | 45 of 65 (**69%**) | 0 of 90 (**0%**) |
|
|
89
|
+
| location writes accepted | 9 | 48 |
|
|
90
|
+
| accounts left behind | **4** | 0 |
|
|
91
|
+
|
|
92
|
+
Reproduce it yourself: `node examples/token-expiry/expiry-demo.mjs`
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
96
|
+
## Cleanup without writing to your database — `signIn`
|
|
97
|
+
|
|
98
|
+
`signIn` is **not** a fourteenth contract method. It plays no part in a run, and
|
|
99
|
+
it is not counted in your coverage score. It exists for one job: letting
|
|
100
|
+
`populace clean` ask *"does this identity exist?"* without creating it.
|
|
101
|
+
|
|
102
|
+
Without it, cleanup reaches an account through `createUser` — which signs **up**
|
|
103
|
+
when the identity is absent. So cleaning an already-clean environment creates
|
|
104
|
+
every simulated identity and immediately deletes it again, writing to your auth
|
|
105
|
+
table purely to prove the table was empty. It also makes the per-account result
|
|
106
|
+
meaningless: you cannot tell "found an abandoned account and removed it" from
|
|
107
|
+
"there was nothing there".
|
|
108
|
+
|
|
109
|
+
```js
|
|
110
|
+
/**
|
|
111
|
+
* user — the account exists (attach whatever deleteUser will need)
|
|
112
|
+
* null — it definitively does not exist
|
|
113
|
+
* throw — you could not find out
|
|
114
|
+
*/
|
|
115
|
+
async signIn({ name, phone, persona, index }) {
|
|
116
|
+
const { data, error } = await client.auth.signInWithPassword({
|
|
117
|
+
email: emailFor(phone),
|
|
118
|
+
password: PASSWORD,
|
|
119
|
+
});
|
|
120
|
+
if (!error) return data.user ? { ...data.user, client } : null;
|
|
121
|
+
if (/invalid login credentials/i.test(error.message)) return null;
|
|
122
|
+
throw new Error(error.message); // never swallow — see below
|
|
123
|
+
}
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
**Throw rather than return null when you could not look.** A network failure is
|
|
127
|
+
not evidence of absence. `clean` separates the two and exits non-zero when any
|
|
128
|
+
identity could not be verified, so nobody is handed a false all-clear over their
|
|
129
|
+
own database.
|
|
130
|
+
|
|
131
|
+
Implement it and `populace doctor` reports `Cleanup read-only`. Leave it out and
|
|
132
|
+
Populace still cleans up — it just says plainly that it is creating rows to do
|
|
133
|
+
so, and that it cannot tell you what was already there.
|
|
134
|
+
|
|
135
|
+
---
|
|
136
|
+
|
|
137
|
+
## Re-runs
|
|
138
|
+
|
|
139
|
+
Identities are deterministic — agent *n* always gets the same phone number.
|
|
140
|
+
`createUser` should therefore **sign in** if the account already exists rather
|
|
141
|
+
than creating a second one. This is what lets `populace clean` find and remove
|
|
142
|
+
accounts after a run that crashed halfway.
|
|
143
|
+
|
|
144
|
+
---
|
|
145
|
+
|
|
146
|
+
## What you get back
|
|
147
|
+
|
|
148
|
+
A `populace-report.json` and a terminal summary:
|
|
149
|
+
|
|
150
|
+
- every failure, grouped by shape rather than exact text, with counts
|
|
151
|
+
- p50 / p95 / p99 / max latency per method under N concurrent users
|
|
152
|
+
- which contract methods you did **not** implement, and what each would have tested
|
|
153
|
+
- whether cleanup actually succeeded
|
|
154
|
+
|
|
155
|
+
---
|
|
156
|
+
|
|
157
|
+
## What it will not tell you
|
|
158
|
+
|
|
159
|
+
Whether people *want* your product. Simulated users are generated from patterns;
|
|
160
|
+
they will not surprise you the way a real customer does, and they are least
|
|
161
|
+
accurate for exactly the users least represented online.
|
|
162
|
+
|
|
163
|
+
Use Populace to prove your app **works**. Use real people to decide what to
|
|
164
|
+
**build**.
|