@chipmobilesdk/rn-local-db 0.3.0 → 0.4.0

package/CHANGELOG.md

@@ -1,5 +1,27 @@
 # @chipmobilesdk/rn-local-db
 
+## 0.4.0
+
+### Minor Changes
+
+- Opening a database whose declared schema has not changed now runs no DDL: the applied schema is recorded and compared, instead of re-running CREATE TABLE, PRAGMA table_info and CREATE INDEX for every collection on each start. Index changes take effect — adding `unique`, changing the fields of an index, or removing its declaration used to do nothing. A newly declared field is backfilled the way a write projects it, so an object or array value is NULL for old records as for new ones. ([#19](https://github.com/operationd8979/chip-mobile-sdk/pull/19))
+
+  Writes make fewer round-trips. `insert` is one statement and reports `RECORD_CONFLICT` without writing, `upsert` on an existing record is one statement, and a batch reads the state of everything it touches once per collection rather than once per operation. A record is serialized once per write, and its size is measured without allocating a copy.
+
+- `deleteScopeData`, `resetScope` and `deleteDatabase` now close the connection of any handle still open on what they delete and give back its slot in the open-handle budget. The handle keeps reporting `SCOPE_DELETED`; the app no longer has to close a handle it cannot use, and eight sign-outs without a `close()` no longer end in `OPEN_FAILED`. ([#19](https://github.com/operationd8979/chip-mobile-sdk/pull/19))
+
+  A database file laid out by a newer release of the package is refused with `SCHEMA_DOWNGRADE` instead of being opened. `setStorageAdapter` is exported from `./testing`, so a test can point `openDatabase` at the in-memory adapter. `MigrationContext.execute` types its parameters as `SqlParameter[]` (string, number, boolean or null) rather than `unknown[]`.
+
+  Internal: one builder for INSERT and UPDATE statements shared by the collection API and batches, one definition of a valid migration chain, the handle split out of the open path, and unused code removed.
+
+- `attachments().write()` accepts `sourcePath` in place of `content`. The file is copied natively when the database is unencrypted, and read and encrypted 256 KB at a time when it is, so writing a large photo or recording no longer needs the whole file in JS memory as base64. `content` keeps working and is still decoded whole. The in-memory adapter takes a `readSourceFile` option for tests that write by path. The README, the changelog and the contract comment no longer claim that write memory was flat before this. ([#19](https://github.com/operationd8979/chip-mobile-sdk/pull/19))
+
+### Patch Changes
+
+- Internal: no module in these packages is over 500 lines any more. The camera controller's session I/O moved to `sessionRuntime` and its contract to `contract.ts`; the Play adapter's shape translation to `iapShapes`; the purchase flow's failure mapping and its receipt-validation hook to modules of their own; the AI client's quota request to `quotaRequest`; the in-memory adapter's attachments and the fake ads engine's clock likewise. Behaviour is unchanged — the splits are by responsibility, and every package's suite passes as it stood. ([#19](https://github.com/operationd8979/chip-mobile-sdk/pull/19))
+
+  The repository also measures what it promised: `npm run perf` prints the median of nine runs of the deterministic hot paths against written budgets (recurrence expansion, theme construction, translation lookup, chart geometry, filter compilation), the demo app has a Performance screen reporting startup phases, time to interactive and the same benchmarks on Hermes, and `npm run check:file-size` keeps the limit from drifting. The budgets and the measured figures are recorded in `docs/standards.md` §VI.
+
 ## 0.3.0
 
 ### Minor Changes
@@ -154,7 +176,7 @@ First release. New package, no existing consumers, no migration burden.
 - **Migrations.** Ascending, at most once per scope, one transaction each, with a per-scope ledger. A failing chain returns no handle and alters nothing; a downgrade is refused rather than truncating. `resetScope` is the separately named recovery path.
 - **Deletion lifecycle.** `deleteScopeData`, `deleteDatabase`, and `resetScope` as module-level functions that work with or without an open handle. Nothing is ever deleted unasked — no expiry, no eviction, no sweep on close or upgrade. Every deletion reports what it removed and what it could not.
 - **Attachments.** Package-owned artifacts with availability checks that never open the file, owner-scoped listing, opt-in cascade on record deletion, and app-triggered reconciliation for orphaned files and dangling references. Every filesystem operation is confined to the managed storage area.
-- **Encryption at rest.** Opt-in under an app-supplied key. SQLCipher for records; chunked AES-256-GCM with HKDF-derived per-artifact subkeys for files, one 256 KB chunk in flight so memory is flat in artifact size. The package never generates, persists, caches beyond the handle, returns, or logs key material, and depends on no keychain library.
+- **Encryption at rest.** Opt-in under an app-supplied key. SQLCipher for records; chunked AES-256-GCM with HKDF-derived per-artifact subkeys for files, one 256 KB chunk in flight. (Write memory was not in fact flat until `sourcePath` arrived in 0.4.0: a base64 `content` is decoded whole.) The package never generates, persists, caches beyond the handle, returns, or logs key material, and depends on no keychain library.
 - **Backup posture.** Excluded by default. Enforced on iOS via `NSURLIsExcludedFromBackupKey`; verified and reported on Android, where participation is manifest-owned and unreachable from a library that authors no native code.
 - **Typed diagnostics.** Twenty-nine stable `DiagnosticCode` values with `isCode` so no consumer parses a message. Disk-full, filesystem-access, and corruption stay distinct. Redaction happens at construction, so no error carries a payload, query parameter, scope component, or key material in any build.
 - **`./testing` entrypoint.** In-memory adapter, the conformance suite both adapters must pass identically, deterministic reset, seeding through production write paths, and fault injection for every diagnostic code.

package/README.md

@@ -356,7 +356,7 @@ Every one returns a `DeletionOutcome` rather than a bare success — what was re
 
 If a deletion is interrupted — a force-quit, a battery death — the next open reports `SCOPE_DELETED` rather than presenting the scope as healthy. Call `deleteScopeData` again to finish it.
 
-A handle whose scope is deleted beneath it reports `SCOPE_DELETED` on its next use. The deletion does not wait for it, and nothing revives it: open a new handle and rebuild whatever was built on the old one. An app that retries through the dead handle gets the same error every time, which reaches the user as a screen that only relaunching can fix.
+A handle whose scope is deleted beneath it reports `SCOPE_DELETED` on its next use. The deletion does not wait for it: it closes that handle's connection and gives back its slot in the open-handle budget, so the app does not have to close a handle it can no longer use. Nothing revives it: open a new handle and rebuild whatever was built on the old one. An app that retries through the dead handle gets the same error every time, which reaches the user as a screen that only relaunching can fix.
 
 ---
 
@@ -381,10 +381,25 @@ await attachments.listFor({ collection: 'notes', id: 'n1' });
 await attachments.remove('photo-1');
 ```
 
-`AttachmentInput.content` is one complete base64 string. The current public
-write contract therefore requires the app to hold the encoded payload, and the
-native adapter decodes that full string before processing encrypted chunks.
-Use range reads to avoid loading an entire stored artifact when consuming it.
+For anything large, pass the file instead of its content:
+
+```ts
+await attachments.write({
+  id: 'video-1',
+  contentType: 'video/mp4',
+  sourcePath: recording.path, // absolute path; a file:// prefix is accepted
+});
+```
+
+With `sourcePath` the bytes never pass through JS as a whole: an unencrypted
+artifact is one native file copy, and an encrypted one is read and encrypted
+256 KB at a time, so write memory does not grow with the file. The source file
+is left where it is. `content` is one complete base64 string, held by the app
+and decoded in one piece — fine for a thumbnail, several times the file's size
+in memory for a video. Use range reads to avoid loading an entire stored
+artifact when consuming it; a read without a range returns the whole artifact
+as one string. The in-memory adapter has no filesystem: give it
+`createMemoryAdapter({ readSourceFile })` to test a write by path.
 
 **Availability** answers from `stat()` and, for an encrypted artifact, the 22-byte header — never a content byte and never a decryption. A gallery showing fifty thumbnails does not decrypt fifty files to ask whether they are there.
 
@@ -444,10 +459,9 @@ The provider receives the resolved scope so you can key per identity — which i
 
 **Records** are protected by SQLCipher. **Artifacts** use chunked AES-256-GCM:
 256 KB chunks, a per-artifact subkey derived with HKDF, and an authentication
-tag per chunk. Encryption and range decryption operate by chunk, but the
-current write API first decodes the complete base64 input. Peak write memory is
-therefore not flat in artifact size; size large media before passing it to this
-version.
+tag per chunk. Encryption and range decryption operate by chunk. A write from
+`sourcePath` holds one chunk at a time; a write from base64 `content` first
+decodes the complete input, so its memory grows with the artifact.
 
 A key-provider failure while the device is locked is an **expected condition**, not a bug:
 
@@ -601,7 +615,11 @@ private and unsupported.
 ## Testing against the in-memory adapter
 
 ```ts
-import { createMemoryAdapter, resetMemoryAdapter } from '@chipmobilesdk/rn-local-db/testing';
+import {
+  createMemoryAdapter,
+  resetMemoryAdapter,
+  setStorageAdapter,
+} from '@chipmobilesdk/rn-local-db/testing';
 
 const adapter = createMemoryAdapter();
 beforeEach(() => resetMemoryAdapter(adapter));
@@ -639,7 +657,7 @@ runConformanceSuite(() => myAdapter, reset, hooks);
 
 ### Documented divergences
 
-The in-memory adapter is faithful, not identical. These six differences are known, deliberate, and the complete list — anything else that differs is a bug:
+The in-memory adapter is faithful, not identical. These seven differences are known, deliberate, and the complete list — anything else that differs is a bug:
 
 | Divergence | Why |
 |---|---|
@@ -649,6 +667,7 @@ The in-memory adapter is faithful, not identical. These six differences are know
 | **Backup posture**: reports the requested value | There is no filesystem to set an attribute on and no manifest to verify. |
 | **Artifact chunking**: content is stored whole | Chunk boundaries, nonces, and authentication tags are exercised by the cipher's own tests. |
 | **Corruption and disk-full**: not spontaneously reachable | Both are device conditions. Use `injectFault` to reach the codes. |
+| **Attachments by `sourcePath`**: read through `readSourceFile`, an option of `createMemoryAdapter` | There is no filesystem to read the path from. Without the option the write is refused with `STORAGE_ACCESS`. |
 
 ---
 

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@chipmobilesdk/rn-local-db",
-	"version": "0.3.0",
+	"version": "0.4.0",
 	"private": false,
 	"license": "UNLICENSED",
 	"main": "src/index.ts",

package/skills/sdk-rn-local-db/SKILL.md

@@ -41,7 +41,7 @@ section, not by deleting the database file.
 `deleteScopeData` and `deleteDatabase` are module-level and do not wait for open
 handles. **After either, open a NEW handle and rebuild everything derived from the
 old one** — repositories, providers, the screen behind a retry button. The old handle
-raises `SCOPE_DELETED` (or `CLOSED_HANDLE` if the app closed it first) on every call,
+is closed for you (its connection and handle slot are released) and raises `SCOPE_DELETED` (or `CLOSED_HANDLE` if the app closed it first) on every call,
 so a Retry wired to it can never succeed: the app stays broken until the process
 restarts. An interrupted deletion instead reports `SCOPE_DELETED` from the next
 `openDatabase` — call `deleteScopeData` again to finish it, then open.

package/src/adapter/budgets.ts

@@ -13,7 +13,6 @@
 
 import { Budgets } from '../contract/config';
 import { DiagnosticCodes } from '../contract/errors';
-import type { StorableRecordData } from '../contract/record';
 import { raise } from '../diagnostics/errors';
 import {
 	assertArtifactWithinBudget,
@@ -53,15 +52,7 @@ export function releaseHandleSlot(): void {
 	openHandles = Math.max(0, openHandles - 1);
 }
 
-export function openHandleCount(): number {
-	return openHandles;
-}
-
-/** Between tests, and after a `deleteDatabase` that invalidated every handle. */
+/** Between tests. */
 export function resetHandleSlots(): void {
 	openHandles = 0;
 }
-
-export function assertWritable(collection: string, data: StorableRecordData): void {
-	assertRecordWithinBudget(collection, data);
-}

package/src/adapter/collection.ts

@@ -27,18 +27,12 @@ import { decodeCursor } from '../query/cursor';
 import { buildPage, fetchLimitFor, resolvePageSize } from '../query/paginate';
 import { withStableTiebreaker } from '../query/sort';
 import { validateQuery } from '../query/validate';
-import { collectionTable, hasSoftDelete, hasTimestamps } from '../schema/declare';
+import { collectionTable, hasSoftDelete } from '../schema/declare';
 import type { Executor } from '../sqlite/connection';
 import type { SqlValue } from '../sqlite/native';
-import {
-	decodeRow,
-	encodePayload,
-	insertColumns,
-	projectedColumnNames,
-	projectedValues,
-	updateAssignments,
-} from '../sqlite/row-codec';
+import { decodeRow } from '../sqlite/row-codec';
 import { buildFilter, buildKeysetPredicate, buildOrderBy } from '../sqlite/sql-builder';
+import { insertStatement, updateStatement } from '../sqlite/statements';
 import { assertRecordWithinBudget } from './budgets';
 
 export type SqlCollectionContext = {
@@ -83,44 +77,35 @@ export function createSqlCollection<T extends StorableRecordData>(
 		async insert(record: RecordInput<T>): Promise<WriteOutcome> {
 			context.assertUsable();
 			throwIfFaulted('write');
-			assertRecordWithinBudget(schema.name, record.data);
+			const payload = assertRecordWithinBudget(schema.name, record.data);
 
-			// Checked rather than left to the primary-key constraint: the
-			// constraint would work, but inside a transaction the engine's
-			// error can arrive after other statements have run, and reporting
-			// the conflict before touching anything keeps the failure boring.
-			if (await rowById(record.id, true)) {
+			// One statement: a row that is already there is left untouched and
+			// reported, instead of a SELECT first and an INSERT after (LDB-08). The
+			// failure stays boring — nothing was written, and no engine error
+			// surfaces from inside a transaction.
+			const result = await context
+				.executor()
+				.execute(
+					...insertStatement(schema, record.id, record.data, context.now(), payload, 'ignore'),
+				);
+			if (result.rowsAffected === 0) {
 				raise(DiagnosticCodes.RECORD_CONFLICT, {
 					message: 'record already exists',
 					detail: { collection: schema.name, recordId: record.id },
 				});
 			}
-
-			const { columns, values } = insertColumns(schema, record.id, record.data, context.now());
-			await context
-				.executor()
-				.execute(
-					`INSERT INTO ${table} (${columns.map(name => `"${name}"`).join(', ')}) VALUES (${columns
-						.map(() => '?')
-						.join(', ')})`,
-					values,
-				);
 			return { operation: 'created', id: record.id };
 		},
 
 		async update(id, record): Promise<WriteOutcome> {
 			context.assertUsable();
 			throwIfFaulted('write');
-			assertRecordWithinBudget(schema.name, record.data);
+			const payload = assertRecordWithinBudget(schema.name, record.data);
 
-			const { columns, values } = updateAssignments(schema, record.data, context.now());
 			const result = await context
 				.executor()
 				.execute(
-					`UPDATE ${table} SET ${columns
-						.map(name => `"${name}" = ?`)
-						.join(', ')} WHERE id = ? AND ${LIVE}`,
-					[...values, id],
+					...updateStatement(schema, id, record.data, context.now(), payload, { liveOnly: true }),
 				);
 			if (result.rowsAffected === 0) {
 				// Creating the record instead would be a silent write to an id
@@ -133,47 +118,34 @@ export function createSqlCollection<T extends StorableRecordData>(
 		async upsert(record: RecordInput<T>): Promise<WriteOutcome> {
 			context.assertUsable();
 			throwIfFaulted('write');
-			assertRecordWithinBudget(schema.name, record.data);
-
-			const existed = (await rowById(record.id, true)) !== undefined;
+			const payload = assertRecordWithinBudget(schema.name, record.data);
 			const now = context.now();
 
-			if (existed) {
-				const assignments = ['payload = ?'];
-				const values: SqlValue[] = [encodePayload(record.data)];
-				if (hasTimestamps(schema)) {
-					assignments.push('updated_at = ?');
-					values.push(now);
-				}
-				if (hasSoftDelete(schema)) {
-					// An upsert over a soft-deleted record brings it back: the
-					// caller asked for this id to hold this data.
-					assignments.push('deleted_at = NULL');
-				}
-				const projected = projectedValues(schema, record.data);
-				for (const [index, name] of projectedColumnNames(schema).entries()) {
-					assignments.push(`"${name}" = ?`);
-					values.push(projected[index]!);
-				}
-				await context
+			// An upsert over a soft-deleted record brings it back: the caller
+			// asked for this id to hold this data.
+			const update = async (): Promise<boolean> => {
+				const result = await context
 					.executor()
-					.execute(`UPDATE ${table} SET ${assignments.join(', ')} WHERE id = ?`, [
-						...values,
-						record.id,
-					]);
+					.execute(
+						...updateStatement(schema, record.id, record.data, now, payload, { restore: true }),
+					);
+				return result.rowsAffected > 0;
+			};
+
+			// The update is tried first: one round-trip for a record that exists,
+			// with no SELECT before it (LDB-08).
+			if (await update()) {
 				return { operation: 'updated', id: record.id };
 			}
-
-			const { columns, values } = insertColumns(schema, record.id, record.data, now);
-			await context
+			const inserted = await context
 				.executor()
-				.execute(
-					`INSERT INTO ${table} (${columns.map(name => `"${name}"`).join(', ')}) VALUES (${columns
-						.map(() => '?')
-						.join(', ')})`,
-					values,
-				);
-			return { operation: 'created', id: record.id };
+				.execute(...insertStatement(schema, record.id, record.data, now, payload, 'ignore'));
+			if (inserted.rowsAffected > 0) {
+				return { operation: 'created', id: record.id };
+			}
+			// Another write created the row between the two statements.
+			await update();
+			return { operation: 'updated', id: record.id };
 		},
 
 		async get(id: string): Promise<StoredRecord<T>> {

package/src/adapter/deletion.ts

@@ -60,6 +60,51 @@ export function isScopeInvalidated(name: string, scopeKey: string): boolean {
 
 export function clearAllInvalidations(): void {
 	invalidated.clear();
+	openHandles.clear();
+}
+
+/**
+ * Handles open on a scope, so a deletion can take their connection away
+ * (LDB-14). Marking alone left the native connection and its handle slot held
+ * until the app closed a handle it had been told was dead: eight sign-outs
+ * later every open failed, with the unlinked file still pinned by its
+ * descriptor.
+ */
+const openHandles = new Map<string, Set<() => Promise<void>>>();
+
+/** Returns the function that takes the handle out of the registry again. */
+export function registerOpenHandle(
+	name: string,
+	scopeKey: string,
+	revoke: () => Promise<void>,
+): () => void {
+	const key = invalidationKey(name, scopeKey);
+	const handles = openHandles.get(key) ?? new Set();
+	handles.add(revoke);
+	openHandles.set(key, handles);
+	return () => {
+		handles.delete(revoke);
+		if (handles.size === 0) {
+			openHandles.delete(key);
+		}
+	};
+}
+
+/** Every handle on the scope, or on any scope of the database when none is named. */
+async function revokeHandles(name: string, scopeKey?: string): Promise<void> {
+	const prefix = `${name}:`;
+	for (const [key, handles] of [...openHandles]) {
+		const matches =
+			scopeKey === undefined ? key.startsWith(prefix) : key === invalidationKey(name, scopeKey);
+		if (!matches) {
+			continue;
+		}
+		openHandles.delete(key);
+		for (const revoke of handles) {
+			// A connection that will not close does not get to stop a deletion.
+			await revoke().catch(() => undefined);
+		}
+	}
 }
 
 /** Deleting a scope that holds nothing is success, not an error (FR-062). */
@@ -144,6 +189,8 @@ async function removeScope(config: ScopeTargetConfig): Promise<DeletionOutcome>
 			await markDeleting(catalog, scope.key);
 		}
 		markScopeInvalidated(config.name, scope.key);
+		// Before the files go: a connection still open on them pins them.
+		await revokeHandles(config.name, scope.key);
 
 		const removedArtifacts = await artifactIdsOnDisk(layout, scope.key);
 
@@ -241,6 +288,7 @@ export async function deleteDatabase(config: DatabaseTargetConfig): Promise<Dele
 	for (const scopeKey of scopeKeys) {
 		markScopeInvalidated(config.name, scopeKey);
 	}
+	await revokeHandles(config.name);
 
 	const removed = await removeIfPresent(layout.root);
 	return {