@buildbase/sdk 0.0.89 → 0.0.90

package/docs/USER-IMPORT.md

@@ -0,0 +1,320 @@
+# Importing users: `buildbase import`
+
+Move your users from Clerk, Auth0, Firebase or Supabase into BuildBase with
+one command. It reads the vendor's own export, converts each user, and sends
+them to your BuildBase app in batches of 500:
+
+```bash
+BUILDBASE_SERVER_URL=https://api.console.buildbase.app \
+BUILDBASE_API_TOKEN=$BUILDBASE_API_TOKEN \
+npx buildbase import --from clerk users.csv --dry-run
+```
+
+Run it with `--dry-run` first: it converts the whole file, prints the first
+three users as they would be sent (hashes redacted) and every record it would
+leave out, and sends nothing. Then run it again without the flag.
+
+## What carries over
+
+| From the old system                                    | In BuildBase                                                               |
+| ------------------------------------------------------ | -------------------------------------------------------------------------- |
+| Email address                                          | The user's email. A record without one is not imported                     |
+| Whether the address was verified                       | `emailVerified`                                                            |
+| Name                                                   | The user's name                                                            |
+| The user's id                                          | `externalId` on the user, and on `useSaaSAuth().user`                      |
+| Phone number, and whether it was verified              | The user's phone (see [Phone numbers](#phone-numbers))                     |
+| Password hash, when we can check it                    | Their old password keeps working (see [Password hashes](#password-hashes)) |
+| Google, GitHub, Microsoft, Apple and LinkedIn sign-ins | Linked accounts: signing in with the same provider finds the same user     |
+
+Not carried over: two-step sign-in secrets, metadata, and sign-ins with
+providers BuildBase does not have (Facebook, X, Discord and others), each of
+which is reported as a warning. Imported users get no welcome or verification
+email. Each one fires the `user.imported` webhook, not `user.registered`, with
+`source` set to the `--from` value, and counts toward your plan's end-user
+limit; users past the limit come back as failed rows with `limit_reached`. A
+user imported with a phone number also fires `user.phone_added`.
+
+Microsoft, Apple and LinkedIn give each app (Apple: each team) its own account
+ids, so an imported Microsoft, Apple or LinkedIn sign-in matches only while
+your BuildBase app uses the same Microsoft app registration, Apple team or
+LinkedIn app as the old system did. The server's report says so after the
+totals when the export has any, one sentence per provider, and `--dry-run`
+says the same sentences before anything is sent.
+
+## Options
+
+| Option                      | What it does                                                                                                 |
+| --------------------------- | ------------------------------------------------------------------------------------------------------------ |
+| `--from <source>`           | `clerk`, `auth0`, `firebase` or `supabase` (required)                                                        |
+| `<file>`                    | The export: CSV, JSON or NDJSON, gzipped or not                                                              |
+| `--passwords <file>`        | Auth0: the password hash export from Auth0 support. Clerk: the dashboard CSV, beside API JSON                |
+| `--overwrite`               | Update users that already exist instead of skipping them                                                     |
+| `--dry-run`                 | Convert and print the first users; send nothing. Needs no token                                              |
+| `--server-url <url>`        | Overrides `BUILDBASE_SERVER_URL`, as for `content pull`                                                      |
+| `--signer-key <base64>`     | Firebase only: `base64_signer_key`. Prefer `BUILDBASE_FIREBASE_SIGNER_KEY`, which stays out of shell history |
+| `--salt-separator <base64>` | Firebase only: `base64_salt_separator`                                                                       |
+| `--rounds <n>`              | Firebase only: `rounds`                                                                                      |
+| `--mem-cost <n>`            | Firebase only: `mem_cost`                                                                                    |
+
+| Environment            | What it is                                                                   |
+| ---------------------- | ---------------------------------------------------------------------------- |
+| `BUILDBASE_API_TOKEN`  | An org API token (console, Settings, API tokens) whose role can create users |
+| `BUILDBASE_SERVER_URL` | Your BuildBase API, for example `https://api.console.buildbase.app`          |
+
+## What it prints
+
+```
+buildbase: read 1204 records from users.json (clerk): 1201 to import
+buildbase: batch 1 of 3 done
+buildbase: batch 2 of 3 done
+buildbase: batch 3 of 3 done
+
+buildbase: 1180 created, 0 updated, 15 skipped, 6 failed (1201 users sent, 3 not sent)
+note: Microsoft user ids are per app registration: imported Microsoft links match only if ...
+
+Not sent:
+  record 212: invalid_email: no email address, so this user is not imported
+  ...
+
+Failed:
+  record 17 (ada@example.com): limit_reached: Your plan allows 1,000 end users.
+  ...
+
+Warnings:
+  record 5 (bob@example.com): unsupported_hasher: Clerk's md5 hashes cannot be imported, ...
+  record 9 (cy@example.com): The google sign-in belongs to another user and was skipped.
+```
+
+A record number is the user's place in your export, counting from 1: the row
+after the header in a CSV, the line in NDJSON, the item in a JSON array.
+`Not sent` lists the users the command left out before sending (no email, a
+plaintext password, deleted upstream), the "not sent" of the totals, and
+`--dry-run` lists the same users under the same heading. `Failed` lists the
+users the server refused.
+`Warnings` lists users that went in without something, whatever their status:
+a password hash that cannot be checked, a sign-in with a provider BuildBase
+does not have, a sign-in that already belongs to another user, a phone number
+that is not E.164 or already belongs to another user, or a password
+`--overwrite` did not apply. Skipped users (`exists`) were already there and
+are not listed. Notes about the whole run (`note:`) are printed once, after
+the totals.
+
+The command exits 0 when every user went in or was skipped, 1 when any failed
+or a request was refused, and 2 when it could not start (a bad flag, a file it
+cannot read, no token).
+
+**Running it again is safe.** Users that already exist are skipped, matched by
+email and then by `externalId`, so a run that stopped half way (a network
+error, a refused batch) is finished by running the same command again. With
+`--overwrite` they are updated instead. Batches are never retried on their own.
+
+## Phone numbers
+
+A user's phone number is imported as E.164 (`+14155550123`), with
+`phoneVerified` true when the old system had proven it with a code. A verified
+number counts as a way to sign in where your app offers phone sign-in, and can
+receive text message codes for two-step sign-in. Only a value already in E.164
+form is sent: any other (`(415) 555-0123`, a number without its country code)
+is left out with an `invalid_phone` warning, and the user is imported without
+it. One number belongs to one account, so a number another user already has is
+skipped by the server with a warning, and that user is imported without it
+too.
+
+| Source   | Where the number comes from                                                                                                      | Verified when                                                                    |
+| -------- | -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
+| Clerk    | API: the `phone_numbers[]` entry `primary_phone_number_id` names, or the first. CSV: `primary_phone_number`, or the first listed | API: `verification.status` is `verified`. CSV: it is in `verified_phone_numbers` |
+| Auth0    | `phone_number`                                                                                                                   | `phone_verified` is true                                                         |
+| Firebase | `phoneNumber`                                                                                                                    | Always: Firebase sets it through phone sign-in, which proves it                  |
+| Supabase | `phone`, which Supabase keeps without its `+`; the import adds it back                                                           | `phone_confirmed_at` is set                                                      |
+
+## Password hashes
+
+Imported users sign in with their old password. The first time they do,
+BuildBase checks it against the imported hash and, when it matches, stores
+its own bcrypt hash in its place, so the old hash is used once at most.
+
+| Hasher                        | Comes from                                                                                      |
+| ----------------------------- | ----------------------------------------------------------------------------------------------- |
+| bcrypt                        | Clerk `bcrypt`, Auth0's hash export, Supabase `encrypted_password`                              |
+| argon2id, argon2i             | Clerk `argon2id` and `argon2i` (PHC strings, `$argon2id$v=19$m=...`)                            |
+| pbkdf2_sha256 (Django's form) | Clerk `pbkdf2_sha256_django`, and Clerk `pbkdf2_sha256` whose base64 salt decodes to plain text |
+| Firebase scrypt               | Firebase's export with the project's parameters, and Clerk `scrypt_firebase`                    |
+
+Any other hasher (Clerk's `md5`, `sha256`, `phpass`, `bcrypt_sha256_django`,
+`scrypt_werkzeug` and the rest) imports the user without a password, with a
+warning; they set one with "Forgot password?" or sign in another way. A value
+in a hash column that does not have its hasher's shape is treated the same
+way and never sent.
+
+**A plaintext password is never sent.** A record that carries one (a
+`password` field) is left out entirely and listed as failed with
+`plaintext_password`; remove the column from the export and run again.
+
+## Clerk
+
+Clerk splits what you need across two exports: the dashboard CSV has the
+password hashes and no social sign-ins, and the Backend API has the social
+sign-ins and no hashes. Use whichever you need, or both.
+
+**Passwords only.** In the Clerk Dashboard, open your instance's
+**Settings**, then **User exports**, choose **Export users**, and download the
+CSV when it is ready. Then:
+
+```bash
+npx buildbase import --from clerk clerk-export.csv
+```
+
+It reads `id`, `first_name`, `last_name`, `username` (when there is no name),
+`primary_email_address`, `verified_email_addresses` (the primary address is
+verified when it appears there), `primary_phone_number`,
+`verified_phone_numbers` and `unverified_phone_numbers` (with no primary
+number, the first verified one, then the first unverified one),
+`password_digest` and `password_hasher`.
+
+**With social sign-ins.** Save every user from the Backend API as one JSON
+array, then pass the dashboard CSV with `--passwords` to keep the hashes:
+
+```bash
+for offset in 0 500 1000 1500; do   # one page per 500 users
+  curl -s "https://api.clerk.com/v1/users?limit=500&offset=$offset" \
+    -H "Authorization: Bearer $CLERK_SECRET_KEY"
+done | jq -s 'add' > clerk-users.json
+
+npx buildbase import --from clerk clerk-users.json --passwords clerk-export.csv
+```
+
+From the API it reads `id`, `first_name`, `last_name`,
+`primary_email_address_id` and `email_addresses[]` (`email_address`, and
+`verification.status` of `verified`), `primary_phone_number_id` and
+`phone_numbers[]` (`phone_number`, and `verification.status` of `verified`),
+and `external_accounts[]` (`provider` and `provider_user_id`). Hashes from the
+CSV are matched by user id, then by email. Providers: `oauth_google`,
+`oauth_github`, `oauth_microsoft`, `oauth_apple`, and `oauth_linkedin_oidc` or
+`oauth_linkedin`.
+
+Check against your export: Clerk does not document how
+`verified_email_addresses` and `verified_phone_numbers` separate several values
+(any address inside the cell is read, and numbers split on commas, semicolons
+or pipes), or whether a dashboard CSV can carry external accounts. A
+legacy `oauth_linkedin` account id may not be the id LinkedIn's OpenID sign-in
+gives the same person, in which case that link will not match.
+
+## Auth0
+
+**Users.** Auth0's bulk export does not include password hashes. Start a
+users-exports job with the Management API (or the User Import / Export
+extension), asking for the fields below as JSON, and download the file at the
+job's `location` once it completes. It is gzipped; pass it as it is.
+
+```bash
+curl -X POST "https://$AUTH0_DOMAIN/api/v2/jobs/users-exports" \
+  -H "Authorization: Bearer $AUTH0_MANAGEMENT_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"format":"json","fields":[{"name":"user_id"},{"name":"email"},{"name":"email_verified"},{"name":"name"},{"name":"given_name"},{"name":"family_name"},{"name":"phone_number"},{"name":"phone_verified"},{"name":"identities"}]}'
+# then GET /api/v2/jobs/<id> until status is "completed", and download its location
+```
+
+It reads `user_id`, `email`, `email_verified`, `name` (Auth0 copies the email
+into `name` for a database user, which is not imported as a name),
+`given_name`, `family_name`, `phone_number`, `phone_verified`, and
+`identities[]` (`provider` and `user_id`).
+Providers: `google-oauth2`, `github`, `windowslive` (Microsoft), `linkedin` and
+`apple`. `auth0`, `email` and `sms` are ways to sign in rather than accounts to
+link. Without `identities`, each user's main sign-in still comes from
+`user_id` (`google-oauth2|1100...`).
+
+**Password hashes.** Open a ticket with Auth0 support asking for a password
+hash export of your database connections (Auth0 offers it on paid plans and
+sends it PGP-encrypted; decrypt it first). It is NDJSON, one user per line,
+with `_id` (a string or `{ "$oid": "..." }`), `email`, `passwordHash`
+(bcrypt) and, for users from a custom database, `alt_id`. Pass it with
+`--passwords`:
+
+```bash
+npx buildbase import --from auth0 users.json.gz --passwords auth0-hashes.json
+```
+
+A hash is matched to a user by `auth0|<alt_id>` or `auth0|<_id>` against the
+bulk export's `user_id`, then by email. A database user with no hash in the
+file imports without a password and is listed as a warning; without
+`--passwords` at all, the command says how many database users that is.
+
+Check against your export: whether `{ "name": "identities" }` exports the
+whole array in your tenant (if not, a user's second and later linked sign-ins
+are not carried over), and the exact field names in the hash file Auth0
+support sends.
+
+## Firebase
+
+1. Export with the Firebase CLI, as JSON (the CSV leaves out Microsoft and
+   Apple sign-ins, so the import does not read it). Exporting hashes needs the
+   Editor or Owner role on the project.
+
+   ```bash
+   firebase auth:export users.json --format=JSON --project <project-id>
+   ```
+
+2. Copy the project's hash parameters: in the Firebase console, open
+   **Authentication**, then **Users**, the menu above the list of users, and
+   **Password hash parameters**. They read `base64_signer_key`,
+   `base64_salt_separator`, `rounds` and `mem_cost`. Treat the signer key as a
+   secret.
+
+3. Import, with the signer key in the environment:
+
+   ```bash
+   BUILDBASE_FIREBASE_SIGNER_KEY='<base64_signer_key>' \
+   npx buildbase import --from firebase users.json \
+     --salt-separator 'Bw==' --rounds 8 --mem-cost 14
+   ```
+
+It reads `users[]`: `localId`, `email`, `emailVerified`, `displayName`,
+`phoneNumber` (imported as verified, since Firebase sets it through phone
+sign-in), `passwordHash` and `salt`, and `providerUserInfo[]` (`providerId`
+and `rawId`). If your own code set numbers with the Admin SDK without a code,
+those are imported as verified too; check before relying on them. Providers:
+`google.com`, `github.com`, `microsoft.com` and `apple.com`; `password` and
+`phone` are not links. An export that holds password hashes is refused without
+all four parameters, rather than importing everyone without a password.
+
+Firebase exports only hashes it made with its own scrypt. A user imported into
+Firebase with another hasher, who has not signed in since, comes out with an
+empty hash and imports without a password.
+
+## Supabase
+
+Export `auth.users` with each user's sign-ins from `auth.identities`, as CSV
+(the SQL editor's download, or `\copy` in psql) or JSON. For a large table use
+psql, which has no row limit:
+
+```sql
+select u.id, u.email, u.email_confirmed_at, u.phone, u.phone_confirmed_at,
+       u.encrypted_password, u.raw_user_meta_data, u.deleted_at,
+       coalesce((select json_agg(json_build_object(
+                   'provider', i.provider,
+                   'provider_id', i.provider_id,
+                   'identity_data', i.identity_data))
+                 from auth.identities i where i.user_id = u.id), '[]'::json) as identities
+from auth.users u
+```
+
+```bash
+psql "$SUPABASE_DB_URL" -c "\copy (<the query above>) to 'supabase-users.csv' csv header"
+npx buildbase import --from supabase supabase-users.csv
+```
+
+For JSON, wrap the query as `select json_agg(t) from (<the query>) t` and save
+the result; the SQL editor's one-cell answer is read too.
+
+It reads `id`, `email`, `email_confirmed_at` (verified when set), `phone`
+(stored without its `+`, which is added back) and `phone_confirmed_at`
+(verified when set), `encrypted_password` (bcrypt),
+`raw_user_meta_data.full_name` or `.name`, `deleted_at` (a deleted user is not
+imported), and `identities[]` (`provider`, and `provider_id` or
+`identity_data.sub`). Providers: `google`, `github`, `azure` (Microsoft),
+`linkedin_oidc` or `linkedin`, and `apple`; `email` and `phone` are not links.
+JSON columns may be text in a CSV; both are read.
+
+Check against your export: `auth.identities.provider_id` exists on projects
+created or upgraded since late 2023; on an older one, leave it out of the
+query and the id comes from `identity_data.sub`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@buildbase/sdk",
-  "version": "0.0.89",
+  "version": "0.0.90",
   "type": "module",
   "description": "A SDK for Buildbase",
   "main": "dist/index.cjs",
@@ -123,6 +123,7 @@
     "docs/MCP-AND-AGENT-READINESS.md",
     "docs/THEMING.md",
     "docs/UI-CONFIG.md",
+    "docs/USER-IMPORT.md",
     "docs/SERVER-CONTENT.md"
   ],
   "sideEffects": [