scrapely 1.2.39 → 1.2.41

package/CHANGELOG.md

@@ -4,6 +4,20 @@ All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
 
+## [1.2.41] - 2026-11-08
+
+- An atomic multi-event push (`Actor.pushData(rows, { charges })`) whose charge the limit covers only in part no longer logs "Charging 5 instances of 'audio-minute' event would exceed the end of the account balance - only 3 events were charged". A row is saved only when all of its events are charged, so the SDK gives those 3 back and nothing is charged: the line was untrue. It now says what happened: "Saving this item needs 5 'audio-minute' event(s), but the account balance covers only 3: it was not saved and nothing was charged for it." When some rows still fit, it names how many were saved and charged in full, and when giving a charge back is not confirmed it says so instead of "nothing was charged". A plain `Actor.charge()` that comes back short keeps its line: those events do stay charged.
+- `ChargeResult.coveredCounts` (from `pushData(rows, { charges })`): for each event whose charge came back short, how many events the limit covered (given back, not charged), e.g. `{ 'audio-minute': 3 }`. Actors can cut a row to what the limit covers without reading the log. What is charged and saved does not change.
+- A spending limit of $0 (`maxTotalChargeUsd: 0`) is a real limit, the same as on the platform. It used to be read as "no limit". A missing or non-numeric value still means no limit.
+- A plain `Actor.pushData(items)` to the default dataset of a pay-per-event Actor whose dataset item event is priced goes through the charged push: rows past the spending limit are not saved, and it returns a `ChargeResult`. Before, all rows were saved and only the ones within the limit were paid.
+- A dataset push that fails with a `503 push-outcome-unknown` (the server could not tell whether it was stored) keeps its charge instead of voiding it.
+- `utils.enqueueLinks({ limit })`: enqueue at most `limit` links per page; links are added to the queue in batches of 100.
+- Dragonfly storage clients: named datasets, key-value stores and request queues are looked up per owner (`name:{userId}:{name}`, as the server stores them), a storage is created only with a known owner, and keys created later (dataset items, records, the log truncation marker) take their storage's expiry. Actor runs on the platform use the API storage client and are not affected.
+
+## [1.2.40] - 2026-10-30
+
+- Actors on puppeteer 25 (ESM-only) are sent to the remote browser when puppeteer is loaded with `require()` before the SDK on a run without the preload. The SDK only recognised puppeteer's old `lib/cjs` build, so such a `puppeteer.launch()` tried to start a local browser; the named `launch` export (`const { launch } = require('puppeteer')`) is covered too. puppeteer 24 and older behave as before.
+
 ## [1.2.39] - 2026-10-29
 
 - A refused charge names the limit that refused it. When a charge is refused or cut short, the platform now says why (HTTP 402 `error.data.reason`, and `limitedBy` on a partial acceptance): the run's item limit (`maxItems`), its maximum total charge (`maxTotalChargeUsd`) or the account balance. The SDK used to call all of them the spending limit. Now its log lines name the real limit; `ChargingManager.getChargeLimitReason()` returns `'max-items'`, `'max-charge'` or `'insufficient-funds'` and `getChargeLimitMessage()` the stop message ("Stopped: reached the item limit you set (800 items)", "Stopped: reached the maximum charge you set ($3.00)", "Stopped: your account has no credit left for this run"); and once a limit was reached, a status message (`Actor.setStatusMessage`, `Actor.exit`, `Actor.fail`) whose clause blames the maximum charge gets that clause replaced by the real stop message. Platforms that do not send the reason yet: inferred from the 402 message and `ACTOR_MAX_PAID_DATASET_ITEMS`. What is charged and saved does not change.

package/dist/actor.js

@@ -603,6 +603,17 @@ class Actor {
         }
         const { Dataset } = await import('@crawlee/core');
         const dataset = await Dataset.open();
+        const isDefault = dataset.id === this.config.get('defaultDatasetId');
+        // A priced default-dataset item event: rows are trimmed to the budget and
+        // counted like a charged push, so the spending limit is seen locally.
+        if (eventName === undefined
+            && !this.usesPushDataInterception(dataset)
+            && this.initialized
+            && isDefault
+            && this.chargingManager._isPayPerEvent()
+            && this.chargingManager._isPaidPerItem(undefined, true)) {
+            return this.pushDataWithExplicitCharging(dataset, item, undefined);
+        }
         if (eventName === undefined && !this.usesPushDataInterception(dataset)) {
             // Charge-limit safety net (also for local storage): this push is not
             // charged, so past the run's spending limit it is not stored.

package/dist/charging.d.ts

@@ -84,7 +84,24 @@ export interface ChargeResult {
     eventChargeLimitReached: boolean;
     chargedCount: number;
     chargeableWithinLimit: Record<string, number>;
+    /**
+     * `pushData(rows, { charges })` only: for each event whose charge came back
+     * short, how many events the limit (account balance, maxTotalChargeUsd,
+     * maxItems) covered. A row is saved only when all of its events are charged,
+     * so those events were given back, not charged. Empty when no charge came
+     * back short.
+     */
+    coveredCounts?: Record<string, number>;
+}
+/** @internal A charge of an atomic push that the limit accepted only in part (then given back). */
+export interface ShortCharge {
+    eventName: string;
+    requested: number;
+    covered: number;
+    limit: string;
 }
+/** @internal Log what an atomic push whose charge came back short really did. */
+export declare function logAtomicRefusal(shorts: ShortCharge[], rowsPushed: number, rowsSaved: number, givenBack: boolean): void;
 export interface ActorPricingInfo {
     pricingModel?: ActorRunPricingInfo['pricingModel'];
     maxTotalChargeUsd: number;
@@ -110,6 +127,12 @@ export declare class ChargingManager {
     private lastLimitMaxItems;
     /** Default-dataset rows refused after the limit (one warning, then a count). */
     private refusedDefaultDatasetItems;
+    /**
+     * @internal Set by pushDataWithCharges while it runs (it holds the operation
+     * lock): charges that come back short are collected here instead of being
+     * logged as "only N events were charged", because the atomic push gives them back.
+     */
+    _shortCharges: ShortCharge[] | undefined;
     /**
      * Prepaid-row balance: priced events charged through charge() minus rows
      * pushed to the default dataset without a per-row charge. An Actor that

package/dist/charging.js

@@ -10,6 +10,7 @@ exports.refuseDefaultDatasetPush = refuseDefaultDatasetPush;
 exports.isDefaultDatasetItemEvent = isDefaultDatasetItemEvent;
 exports.resolveDefaultDatasetItemEventName = resolveDefaultDatasetItemEventName;
 exports.hasDefaultDatasetItemEventInPricing = hasDefaultDatasetItemEventInPricing;
+exports.logAtomicRefusal = logAtomicRefusal;
 exports.mergeChargeResults = mergeChargeResults;
 exports.assertItemsSerializable = assertItemsSerializable;
 exports.pushDataAndCharge = pushDataAndCharge;
@@ -234,6 +235,32 @@ function hasDefaultDatasetItemEventInPricing(perEventPrices) {
 function isSyntheticEvent(eventName) {
     return isDefaultDatasetItemEvent(eventName);
 }
+function itemsPhrase(count) {
+    return count === 1
+        ? { what: 'this item', were: 'it was', them: 'it' }
+        : { what: `these ${count} items`, were: 'they were', them: 'them' };
+}
+/** @internal Log what an atomic push whose charge came back short really did. */
+function logAtomicRefusal(shorts, rowsPushed, rowsSaved, givenBack) {
+    const first = shorts[0];
+    if (!first)
+        return;
+    const limit = first.limit.replace(/^the end of /, '');
+    const { what, were, them } = itemsPhrase(rowsPushed);
+    const head = `[Scrapely] Saving ${what} needs ${first.requested} '${first.eventName}' event(s), but ${limit} covers only ${first.covered}`;
+    if (rowsSaved <= 0) {
+        console.log(givenBack
+            ? `${head}: ${were} not saved and nothing was charged for ${them}.`
+            : `${head}: ${were} not saved, but the ${first.covered} event(s) charged for ${them} could not be given back yet (see the warning about it).`);
+        return;
+    }
+    const left = rowsPushed - rowsSaved;
+    const rest = itemsPhrase(left);
+    const tail = givenBack
+        ? `nothing was charged for ${rest.them}`
+        : `what was charged for ${rest.them} could not be given back yet (see the warning about it)`;
+    console.log(`${head}: saved the first ${rowsSaved} item(s), charged in full; the other ${left} ${rest.were.split(' ').pop()} not saved and ${tail}.`);
+}
 function mergeChargeResults(a, b) {
     return {
         eventChargeLimitReached: a.eventChargeLimitReached || b.eventChargeLimitReached,
@@ -244,6 +271,17 @@ function mergeChargeResults(a, b) {
         ])),
     };
 }
+/**
+ * The run's spending limit (maxTotalChargeUsd), or Infinity when uncapped.
+ * 0 is a real limit (no fees), the same as on the platform; only a missing
+ * or non-numeric value means uncapped.
+ */
+function spendingLimitUsd(value) {
+    if (value == null || value === '')
+        return Infinity;
+    const usd = Number(value);
+    return Number.isFinite(usd) && usd >= 0 ? usd : Infinity;
+}
 /**
  * Pay-per-event charging for platform runs.
  * Reports charges via POST /v2/actor-runs/:runId/charge (Redis → worker billing).
@@ -262,6 +300,12 @@ class ChargingManager {
     lastLimitMaxItems;
     /** Default-dataset rows refused after the limit (one warning, then a count). */
     refusedDefaultDatasetItems = 0;
+    /**
+     * @internal Set by pushDataWithCharges while it runs (it holds the operation
+     * lock): charges that come back short are collected here instead of being
+     * logged as "only N events were charged", because the atomic push gives them back.
+     */
+    _shortCharges;
     /**
      * Prepaid-row balance: priced events charged through charge() minus rows
      * pushed to the default dataset without a per-row charge. An Actor that
@@ -577,7 +621,7 @@ class ChargingManager {
     constructor(configuration, apifyClient) {
         this.configuration = configuration;
         this.apifyClientSource = apifyClient;
-        this.maxTotalChargeUsd = configuration.get('maxTotalChargeUsd') || Infinity;
+        this.maxTotalChargeUsd = spendingLimitUsd(configuration.get('maxTotalChargeUsd'));
         this.isAtHome = !!configuration.get('isAtHome');
         this.actorRunId = configuration.get('actorRunId') || '';
     }
@@ -619,7 +663,7 @@ class ChargingManager {
             return {
                 pricingInfo: actorPricingInfo ? JSON.parse(actorPricingInfo) : undefined,
                 chargedEventCounts: JSON.parse(chargedEventCounts),
-                maxTotalChargeUsd: this.configuration.get('maxTotalChargeUsd') || Infinity,
+                maxTotalChargeUsd: spendingLimitUsd(this.configuration.get('maxTotalChargeUsd')),
             };
         }
         if (this.isAtHome) {
@@ -633,13 +677,13 @@ class ChargingManager {
             return {
                 pricingInfo: run.pricingInfo,
                 chargedEventCounts: run.chargedEventCounts,
-                maxTotalChargeUsd: run.options?.maxTotalChargeUsd || Infinity,
+                maxTotalChargeUsd: spendingLimitUsd(run.options?.maxTotalChargeUsd),
             };
         }
         return {
             pricingInfo: undefined,
             chargedEventCounts: {},
-            maxTotalChargeUsd: this.configuration.get('maxTotalChargeUsd') || Infinity,
+            maxTotalChargeUsd: spendingLimitUsd(this.configuration.get('maxTotalChargeUsd')),
         };
     }
     /**
@@ -1328,8 +1372,16 @@ class ChargingManager {
             });
         }
         if (chargedCount < count) {
-            const subject = count === 1 ? 'instance' : 'instances';
-            console.log(`[Scrapely] Charging ${count} ${subject} of '${normalizedEventName}' event would exceed ${this.limitName()} - only ${chargedCount} events were charged`);
+            if (this._shortCharges) {
+                // Inside an atomic push: this charge is given back unless the row is
+                // saved, so "only N events were charged" would not be true there.
+                // pushDataWithCharges logs what really happened instead.
+                this._shortCharges.push({ eventName: normalizedEventName, requested: count, covered: chargedCount, limit: this.limitName() });
+            }
+            else {
+                const subject = count === 1 ? 'instance' : 'instances';
+                console.log(`[Scrapely] Charging ${count} ${subject} of '${normalizedEventName}' event would exceed ${this.limitName()} - only ${chargedCount} events were charged`);
+            }
         }
         return {
             eventChargeLimitReached: this.calculateMaxEventChargeCountWithinLimit(normalizedEventName) <= 0,
@@ -1814,6 +1866,10 @@ async function pushDataWithCharges({ chargingManager, items, charges, isDefaultD
         // An item the storage would refuse fails here, before it is charged.
         assertItemsSerializable(rowsIn);
         validateItems?.(rowsIn);
+        // Charges of this push that came back short: given back, and logged once the
+        // outcome is known (logAtomicRefusal), not as "only N events were charged".
+        const shorts = [];
+        let givenBack = true;
         const undo = async (ops) => {
             for (const op of ops) {
                 if (op.charged <= 0)
@@ -1823,61 +1879,75 @@ async function pushDataWithCharges({ chargingManager, items, charges, isDefaultD
                 else if (!op.operationId)
                     chargingManager.applyAcceptedCount(op.name, op.charged, 0);
                 // Cancel / void; a refused void stays billed and becomes credit.
-                else
-                    await voidOrCredit(chargingManager, op.name, op.operationId, op.charged, op.uncertain);
+                else if ((await voidOrCredit(chargingManager, op.name, op.operationId, op.charged, op.uncertain)) !== 'voided')
+                    givenBack = false;
             }
         };
+        // Per event: how many events the limit covered in its last short answer.
+        const coveredCounts = () => Object.fromEntries(shorts.map((sc) => [sc.eventName, sc.covered]));
         const order = Object.entries(perRow).sort(([a], [b]) => Number(isSyntheticEvent(a)) - Number(isSyntheticEvent(b)) || a.localeCompare(b));
         let ops = [];
-        for (let attempt = 0; attempt < 6 && rows > 0; attempt++) {
-            ops = [];
-            let shortTo = null;
-            for (const [name, n] of order) {
-                const want = n * rows;
-                chargingManager._takeLastAcceptedOperationId?.();
-                let got;
-                if (takeVoidCredit(chargingManager, name, want) > 0) {
-                    got = want;
-                    ops.push({ name, charged: want, uncertain: false, credit: true });
-                }
-                else {
-                    try {
-                        const r = await chargingManager._chargeExclusive({ eventName: name, count: want });
-                        got = r.chargedCount;
-                        ops.push({ name, charged: got, operationId: chargingManager._takeLastAcceptedOperationId?.(), uncertain: false });
+        chargingManager._shortCharges = shorts;
+        try {
+            for (let attempt = 0; attempt < 6 && rows > 0; attempt++) {
+                ops = [];
+                let shortTo = null;
+                for (const [name, n] of order) {
+                    const want = n * rows;
+                    chargingManager._takeLastAcceptedOperationId?.();
+                    let got;
+                    if (takeVoidCredit(chargingManager, name, want) > 0) {
+                        got = want;
+                        ops.push({ name, charged: want, uncertain: false, credit: true });
                     }
-                    catch (err) {
-                        if (!(err instanceof ChargeUncertainError)) {
-                            await undo(ops);
-                            throw err;
+                    else {
+                        try {
+                            const r = await chargingManager._chargeExclusive({ eventName: name, count: want });
+                            got = r.chargedCount;
+                            ops.push({ name, charged: got, operationId: chargingManager._takeLastAcceptedOperationId?.(), uncertain: false });
+                        }
+                        catch (err) {
+                            if (!(err instanceof ChargeUncertainError)) {
+                                await undo(ops);
+                                throw err;
+                            }
+                            // Not confirmed: stays pending and is re-posted at exit, so it counts as accepted.
+                            got = err.count;
+                            ops.push({ name, charged: got, operationId: err.operationId, uncertain: true });
                         }
-                        // Not confirmed: stays pending and is re-posted at exit, so it counts as accepted.
-                        got = err.count;
-                        ops.push({ name, charged: got, operationId: err.operationId, uncertain: true });
+                    }
+                    if (got < want) {
+                        shortTo = Math.floor(got / n);
+                        break;
                     }
                 }
-                if (got < want) {
-                    shortTo = Math.floor(got / n);
+                if (shortTo === null)
                     break;
-                }
+                // An event came back short: void this attempt and retry with the whole
+                // rows the budget covers at the FULL per-row cost (sizing from the short
+                // event alone came back short again: appstore 64df0a3608d74af88, 0 saved).
+                // At least one single-row try: the last affordable row may still fit.
+                const attemptOps = ops.map((op) => ({ name: op.name, charged: op.charged }));
+                await undo(ops);
+                ops = [];
+                rows = Math.min(rows - 1, Math.max(chargingManager._rowsAfterShort(attemptOps, perRow), 1));
             }
-            if (shortTo === null)
-                break;
-            // An event came back short: void this attempt and retry with the whole
-            // rows the budget covers at the FULL per-row cost (sizing from the short
-            // event alone came back short again: appstore 64df0a3608d74af88, 0 saved).
-            // At least one single-row try: the last affordable row may still fit.
-            const attemptOps = ops.map((op) => ({ name: op.name, charged: op.charged }));
-            await undo(ops);
-            ops = [];
-            rows = Math.min(rows - 1, Math.max(chargingManager._rowsAfterShort(attemptOps, perRow), 1));
+        }
+        finally {
+            chargingManager._shortCharges = undefined;
         }
         if (rows <= 0 || ops.length === 0) {
             await undo(ops);
+            logAtomicRefusal(shorts, rowsIn.length, 0, givenBack);
             chargingManager._markChargeLimitHit();
             if (isDefaultDataset)
                 chargingManager._noteRefusedDefaultDatasetItems(rowsIn.length);
-            return { eventChargeLimitReached: true, chargedCount: 0, chargeableWithinLimit: chargingManager._chargeableWithinLimit() };
+            return {
+                eventChargeLimitReached: true,
+                chargedCount: 0,
+                chargeableWithinLimit: chargingManager._chargeableWithinLimit(),
+                coveredCounts: coveredCounts(),
+            };
         }
         const kept = batch ? rowsIn : rowsIn.slice(0, rows);
         if (kept.length < rowsIn.length) {
@@ -1894,10 +1964,12 @@ async function pushDataWithCharges({ chargingManager, items, charges, isDefaultD
                 await undo(ops);
             throw pushErr;
         }
+        logAtomicRefusal(shorts, rowsIn.length, kept.length, givenBack);
         return {
             eventChargeLimitReached: kept.length < rowsIn.length || chargingManager.isChargeLimitReached(),
             chargedCount: kept.length,
             chargeableWithinLimit: chargingManager._chargeableWithinLimit(),
+            coveredCounts: coveredCounts(),
         };
     });
 }
@@ -1919,6 +1991,9 @@ function pushCertainlyFailed(err) {
     for (let e = err, depth = 0; e && depth < 4; e = e.cause, depth++) {
         if (e.outcomeUnknown === true)
             return false;
+        // The server could not tell whether the push was stored (a 503 with this type).
+        if (apiErrorTypeOf(e) === 'push-outcome-unknown')
+            return false;
     }
     if ((0, api_storage_client_js_1.isConnectFailure)(err))
         return true;

package/dist/crypto/secrets.js

@@ -20,6 +20,7 @@ const PBKDF2_ITERATIONS = 100_000;
 const PBKDF2_HASH = 'sha256';
 const SALT_LENGTH = 32;
 const KEY_LENGTH = 32;
+const UNSAFE_KEYS = new Set(['__proto__', 'constructor', 'prototype']);
 // ─── Private Key Decryption ───────────────────────────────────────────────────
 /**
  * Read the private key file from disk and decrypt it with the run passphrase.
@@ -101,6 +102,9 @@ function decryptAllSecrets(encryptedSecrets, privateKeyPem) {
 function mergeDecryptedInput(input, decryptedSecrets) {
     const merged = { ...input };
     for (const [fieldName, value] of Object.entries(decryptedSecrets)) {
+        // Never walk into or write onto an object's prototype chain.
+        if (fieldName.split('.').some((part) => UNSAFE_KEYS.has(part)))
+            continue;
         if (fieldName.includes('.')) {
             // Handle nested paths
             const parts = fieldName.split('.');

package/dist/patch-browser-pool.js

@@ -791,8 +791,31 @@ function patchRawPuppeteerModule(mod, moduleName) {
                 // read-only export binding — the default export is still patched
             }
         }
+        // ESM-only puppeteer (v25+) loaded through require(): the named
+        // `launch` is a read-only binding to the original method, which hands
+        // off to a BrowserLauncher. Patch that class so it redirects too.
+        if (mod.launch === origBoundLaunch)
+            patchPuppeteerBrowserLauncherClass(mod.BrowserLauncher, moduleName);
     }
 }
+/**
+ * Patch puppeteer's BrowserLauncher class (base of ChromeLauncher and
+ * FirefoxLauncher), which every PuppeteerNode launch goes through. Calls that
+ * already went through a patched launch decide the same way again (no CDP URL
+ * → local launch). Idempotent per class.
+ */
+function patchPuppeteerBrowserLauncherClass(BrowserLauncher, moduleName) {
+    const proto = BrowserLauncher?.prototype;
+    if (!proto || typeof proto.launch !== 'function')
+        return;
+    if (Object.prototype.hasOwnProperty.call(proto, '_scrapelyRawCdpPatched'))
+        return;
+    const origLaunch = proto.launch;
+    proto.launch = function browserLauncherLaunchPatched(launchOptions) {
+        return puppeteerLaunchPatched(this.puppeteer, (options) => origLaunch.call(this, options), moduleName, launchOptions);
+    };
+    proto._scrapelyRawCdpPatched = true;
+}
 /**
  * Requests that load (a part of) some copy of @crawlee/browser-pool:
  * - the bare package name (require from @crawlee/playwright etc.),
@@ -804,8 +827,11 @@ const BROWSER_POOL_REQUEST_RE = /^@crawlee\/browser-pool$|[/\\]@crawlee[/\\]brow
 const BROWSER_POOL_FILENAME_RE = /[/\\]@crawlee[/\\]browser-pool[/\\](index|playwright[/\\]playwright-plugin|puppeteer[/\\]puppeteer-plugin)\.js$/;
 /** playwright / playwright-core package entry, by filename (ESM imports and require.cache). */
 const PLAYWRIGHT_ENTRY_FILENAME_RE = /[/\\]node_modules[/\\](playwright|playwright-core)[/\\]index\.js$/;
-/** puppeteer / puppeteer-core CJS package entry (default instance), by filename. */
-const PUPPETEER_ENTRY_FILENAME_RE = /[/\\]node_modules[/\\](puppeteer|puppeteer-core)[/\\]lib[/\\]cjs[/\\]puppeteer[/\\](puppeteer|puppeteer-core)\.js$/;
+/**
+ * puppeteer / puppeteer-core package entry (default instance), by filename:
+ * `lib/cjs/puppeteer/…` up to v24, `lib/puppeteer/…` in the ESM-only v25+.
+ */
+const PUPPETEER_ENTRY_FILENAME_RE = /[/\\]node_modules[/\\](puppeteer|puppeteer-core)[/\\]lib[/\\](?:cjs[/\\])?puppeteer[/\\](puppeteer|puppeteer-core)\.js$/;
 /** puppeteer-core's PuppeteerNode class file (CJS), as requested by node/node.js or cached. */
 const PUPPETEER_NODE_REQUEST_RE = /(^|[/\\])PuppeteerNode(\.js)?$/;
 /**

package/dist/storage/dataset-client.js

@@ -4,6 +4,7 @@ exports.DragonflyDatasetClient = void 0;
 const utils_js_1 = require("./utils.js");
 const billing_tracker_js_1 = require("./billing-tracker.js");
 const dataset_push_script_js_1 = require("./dataset-push-script.js");
+const ttl_manager_js_1 = require("./ttl-manager.js");
 function makeAsyncIterableResult(promiseFn) {
     const promise = promiseFn();
     const iterable = {
@@ -60,11 +61,13 @@ class DragonflyDatasetClient {
             const raw = await this.client.redis.hget(indexKey, `id:${this.id}`);
             if (raw) {
                 const data = JSON.parse(raw);
-                if (data.name)
-                    await this.client.redis.hdel(indexKey, `name:${data.name}`);
+                // Name pointers are scoped by owner (name:{userId}:{name}).
+                const ownerId = data.userId ?? this.client.billingContext?.userId;
+                await (0, utils_js_1.releaseOwnerName)(this.client.redis, indexKey, ownerId, data.name, this.id);
                 data.name = newFields.name;
                 await this.client.redis.hset(indexKey, `id:${this.id}`, JSON.stringify(data));
-                await this.client.redis.hset(indexKey, `name:${newFields.name}`, JSON.stringify(data));
+                if (ownerId && newFields.name)
+                    await this.client.redis.hset(indexKey, (0, utils_js_1.ownerNameField)(ownerId, newFields.name), JSON.stringify(data));
             }
         }
         await this.client.redis.hset(this.infoKey, 'modifiedAt', now.toISOString());
@@ -75,8 +78,7 @@ class DragonflyDatasetClient {
         await this.client.redis.del(this.infoKey, this.itemsKey);
         const indexKey = this.client.key('datasets:index');
         await this.client.redis.hdel(indexKey, `id:${this.id}`);
-        if (info?.name)
-            await this.client.redis.hdel(indexKey, `name:${info.name}`);
+        await (0, utils_js_1.releaseOwnerName)(this.client.redis, indexKey, info?.userId || this.client.billingContext?.userId, info?.name, this.id);
     }
     async downloadItems() {
         const result = await this._listItems();
@@ -216,6 +218,10 @@ class DragonflyDatasetClient {
         // separate round trips allowed concurrent pushes to allocate the same
         // slot indices, silently overwriting items.
         await (0, dataset_push_script_js_1.pushItemsAtomic)(this.client.redis, this.infoKey, this.itemsKey, serializedItems, totalBytes);
+        // The items hash is created by the first push: give it the dataset's expiry.
+        const expiry = this.client.redis.pipeline();
+        (0, ttl_manager_js_1.queueInheritExpiry)(expiry, this.infoKey, [this.itemsKey]);
+        await expiry.exec();
         // Track billing using client's billing context
         if (this.client.billingContext) {
             const billingTracker = new billing_tracker_js_1.BillingTracker(this.client);

package/dist/storage/dataset-collection-client.js

@@ -9,8 +9,10 @@ class DragonflyDatasetCollectionClient {
     }
     async getOrCreate(name) {
         const indexKey = this.client.key('datasets:index');
+        // Every storage belongs to the run's user; the name pointer is scoped by that user.
+        const userId = (0, utils_js_1.requireStorageOwner)(this.client.billingContext, 'dataset');
         if (name) {
-            const existing = await this.client.redis.hget(indexKey, `name:${name}`);
+            const existing = await this.client.redis.hget(indexKey, (0, utils_js_1.ownerNameField)(userId, name));
             if (existing)
                 return JSON.parse(existing);
         }
@@ -25,7 +27,7 @@ class DragonflyDatasetCollectionClient {
         };
         await this.client.redis.hset(indexKey, `id:${id}`, JSON.stringify(data));
         if (name)
-            await this.client.redis.hset(indexKey, `name:${name}`, JSON.stringify(data));
+            await this.client.redis.hset(indexKey, (0, utils_js_1.ownerNameField)(userId, name), JSON.stringify(data));
         const infoKey = this.client.key('dataset', id, 'info');
         const itemsKey = this.client.key('dataset', id, 'items');
         await this.client.redis.hset(infoKey, {
@@ -62,8 +64,7 @@ class DragonflyDatasetCollectionClient {
             const exists = await this.client.redis.exists(infoKey);
             if (!exists) {
                 await this.client.redis.hdel(indexKey, `id:${parsed.id}`);
-                if (parsed.name)
-                    await this.client.redis.hdel(indexKey, `name:${parsed.name}`);
+                await (0, utils_js_1.releaseOwnerName)(this.client.redis, indexKey, parsed.userId ?? this.client.billingContext?.userId, parsed.name, parsed.id);
                 continue;
             }
             const itemCount = parseInt(await this.client.redis.hget(infoKey, 'itemCount') || '0', 10);

package/dist/storage/key-value-store-client.js

@@ -4,6 +4,7 @@ exports.DragonflyKeyValueStoreClient = void 0;
 const node_stream_1 = require("node:stream");
 const utils_js_1 = require("./utils.js");
 const billing_tracker_js_1 = require("./billing-tracker.js");
+const ttl_manager_js_1 = require("./ttl-manager.js");
 const configuration_js_1 = require("../configuration.js");
 const kvs_iteration_js_1 = require("./kvs-iteration.js");
 function makeAsyncIterableListKeys(promiseFn) {
@@ -70,11 +71,13 @@ class DragonflyKeyValueStoreClient {
             const raw = await this.client.redis.hget(indexKey, `id:${this.id}`);
             if (raw) {
                 const data = JSON.parse(raw);
-                if (data.name)
-                    await this.client.redis.hdel(indexKey, `name:${data.name}`);
+                // Name pointers are scoped by owner (name:{userId}:{name}).
+                const ownerId = data.userId ?? this.client.billingContext?.userId;
+                await (0, utils_js_1.releaseOwnerName)(this.client.redis, indexKey, ownerId, data.name, this.id);
                 data.name = newFields.name;
                 await this.client.redis.hset(indexKey, `id:${this.id}`, JSON.stringify(data));
-                await this.client.redis.hset(indexKey, `name:${newFields.name}`, JSON.stringify(data));
+                if (ownerId && newFields.name)
+                    await this.client.redis.hset(indexKey, (0, utils_js_1.ownerNameField)(ownerId, newFields.name), JSON.stringify(data));
             }
         }
         await this.client.redis.hset(this.infoKey, 'modifiedAt', (0, utils_js_1.nowISO)().toISOString());
@@ -85,8 +88,7 @@ class DragonflyKeyValueStoreClient {
         await this.client.redis.del(this.infoKey, this.recordsKey, this.metaKey);
         const indexKey = this.client.key('kvs:index');
         await this.client.redis.hdel(indexKey, `id:${this.id}`);
-        if (info?.name)
-            await this.client.redis.hdel(indexKey, `name:${info.name}`);
+        await (0, utils_js_1.releaseOwnerName)(this.client.redis, indexKey, info?.userId || this.client.billingContext?.userId, info?.name, this.id);
     }
     async recordExists(key) {
         return (await this.client.redis.hexists(this.recordsKey, key)) === 1;
@@ -168,6 +170,8 @@ class DragonflyKeyValueStoreClient {
             pipeline.hset(this.metaKey, key, JSON.stringify({ contentType }));
         }
         pipeline.hset(this.infoKey, 'modifiedAt', (0, utils_js_1.nowISO)().toISOString());
+        // Records/meta are created by the first write: give them the store's expiry.
+        (0, ttl_manager_js_1.queueInheritExpiry)(pipeline, this.infoKey, [this.recordsKey, this.metaKey]);
         await pipeline.exec();
         // Track billing using client's billing context
         if (this.client.billingContext) {

package/dist/storage/kv-store-collection-client.js

@@ -9,17 +9,15 @@ class DragonflyKeyValueStoreCollectionClient {
     }
     async getOrCreate(name) {
         const indexKey = this.client.key('kvs:index');
+        // Every storage belongs to the run's user; the name pointer is scoped by that user.
+        const userId = (0, utils_js_1.requireStorageOwner)(this.client.billingContext, 'key-value store');
         if (name) {
-            const existing = await this.client.redis.hget(indexKey, `name:${name}`);
+            const existing = await this.client.redis.hget(indexKey, (0, utils_js_1.ownerNameField)(userId, name));
             if (existing)
                 return JSON.parse(existing);
         }
         const id = (0, utils_js_1.generateId)();
         const now = (0, utils_js_1.nowISO)();
-        // Derive ownership from the active billing context (as the request-queue
-        // client does); the server authorizes against this persisted field.
-        // The legacy '1' placeholder applies only when no context is present.
-        const userId = this.client.billingContext?.userId ?? '1';
         const data = {
             id,
             name,
@@ -30,7 +28,7 @@ class DragonflyKeyValueStoreCollectionClient {
         };
         await this.client.redis.hset(indexKey, `id:${id}`, JSON.stringify(data));
         if (name)
-            await this.client.redis.hset(indexKey, `name:${name}`, JSON.stringify(data));
+            await this.client.redis.hset(indexKey, (0, utils_js_1.ownerNameField)(userId, name), JSON.stringify(data));
         const infoKey = this.client.key('kvs', id, 'info');
         const recordsKey = this.client.key('kvs', id, 'records');
         const metaKey = this.client.key('kvs', id, 'meta');
@@ -68,8 +66,7 @@ class DragonflyKeyValueStoreCollectionClient {
             const exists = await this.client.redis.exists(infoKey);
             if (!exists) {
                 await this.client.redis.hdel(indexKey, `id:${parsed.id}`);
-                if (parsed.name)
-                    await this.client.redis.hdel(indexKey, `name:${parsed.name}`);
+                await (0, utils_js_1.releaseOwnerName)(this.client.redis, indexKey, parsed.userId, parsed.name, parsed.id);
                 continue;
             }
             items.push(parsed);

package/dist/storage/log-manager.d.ts

@@ -1,5 +1,5 @@
 import type { DragonflyClient } from './dragonfly-client.js';
-import type { UserPlan } from './ttl-manager.js';
+import { type UserPlan } from './ttl-manager.js';
 /**
  * Log configuration
  */

package/dist/storage/log-manager.js

@@ -1,6 +1,7 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.LogManager = void 0;
+const ttl_manager_js_1 = require("./ttl-manager.js");
 const run_metadata_js_1 = require("./run-metadata.js");
 const DEFAULT_LOG_CONFIG = {
     maxLogSize: 5 * 1024 * 1024, // 5MB
@@ -89,10 +90,14 @@ class LogManager {
             // a valid code-point boundary and never store a broken sequence.
             const alignedOffset = LogManager.alignUtf8Boundary(trailingBuf, 0);
             const safeTrailing = trailingBuf.subarray(alignedOffset);
-            // Overwrite the key with only the safe trailing portion
-            await this.client.redis.set(key, safeTrailing);
-            // Mark log as truncated (NX — only set once; ignore if already set)
+            // Overwrite the key with only the safe trailing portion (keeping the log's expiry)
+            await this.client.redis.set(key, safeTrailing, 'KEEPTTL');
+            // Mark log as truncated (NX — only set once; ignore if already set). The marker
+            // is created here, after setLogTtl ran: it takes the log's expiry.
             await this.client.redis.set(this.getTruncatedKey(runId), '1', 'NX');
+            const expiry = this.client.redis.pipeline();
+            (0, ttl_manager_js_1.queueInheritExpiry)(expiry, key, [this.getTruncatedKey(runId)]);
+            await expiry.exec();
         }
     }
     /**
@@ -118,7 +123,7 @@ class LogManager {
         const key = this.getLogKey(runId);
         const ttl = this.client.ttlManager.getTtl('logs', plan);
         if (ttl !== null && ttl > 0) {
-            await this.client.ttlManager.setExpiry(this.client.redis, [key], ttl);
+            await this.client.ttlManager.setExpiry(this.client.redis, [key, this.getTruncatedKey(runId)], ttl);
         }
     }
     /**