@kairos-es/feed-algolia 0.0.0-stage → 0.0.0

package/LICENSE

@@ -0,0 +1,28 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, Neverbland
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

package/README.md

@@ -1,3 +1,67 @@
-# Temporary Holding Version
+# @kairos-es/feed-algolia
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+An Algolia sink for the kairos-es change feed (`@kairos-es/feed`): it keeps one Algolia index in step with a read model, sending only the records that changed, each at its newest state, and rebuilding the index into a fresh one that is swapped in whole when the read model is rebuilt.
+
+It talks to Algolia's REST API through `effect`'s own `HttpClient`, so it adds no SDK dependency. You choose the HTTP transport by providing the client.
+
+## Installation
+
+`@kairos-es/feed` and `effect` are peer dependencies:
+
+```sh
+pnpm add @kairos-es/feed-algolia@alpha @kairos-es/feed@alpha effect@^4.0.5
+```
+
+`@kairos-es/feed` peers `@kairos-es/core` and `@kairos-es/read` in turn; its README says what to install alongside it.
+
+## Wiring
+
+```ts
+import { runFeedSync, SinkName } from '@kairos-es/feed'
+import {
+  AlgoliaAppId,
+  AlgoliaIndexName,
+  makeAlgoliaSink,
+} from '@kairos-es/feed-algolia'
+import { Config, Effect } from 'effect'
+import { FetchHttpClient } from 'effect/http'
+
+const program = Effect.gen(function* () {
+  const sink = yield* makeAlgoliaSink({
+    name: SinkName.make('search'),
+    appId: AlgoliaAppId.make(yield* Config.String('ALGOLIA_APP_ID')),
+    adminKey: yield* Config.Redacted('ALGOLIA_ADMIN_KEY'),
+    index: AlgoliaIndexName.make('courses'),
+    doc: CourseDoc, // the Schema your read model writes into the feed
+  }).pipe(Effect.provide(FetchHttpClient.layer))
+  yield* Effect.forkScoped(runFeedSync({ source, sink, store }))
+})
+```
+
+The admin key is held as `Redacted` and never logged. If you use a scoped key instead, give it exactly these ACLs, on the index and on the resync indices named after it (`<index>__kairos_g…`):
+
+| ACL | What the sink uses it for |
+| :-- | :-- |
+| `addObject` | writing records, copying the served index's settings into a resync index, and moving that index over the served one |
+| `deleteObject` | deleting records |
+| `deleteIndex` | deleting the resync indices an abandoned rebuild left behind |
+| `settings` | reading an index's settings, to check a resync index before it is moved |
+| `editSettings` | marking a resync index, and taking the mark off the served index after the move |
+| `browse` | listing the served index's object ids, for reconcile |
+| `listIndexes` | listing indices, to find the resync indices to delete |
+
+`listIndexes` is required. A key without it stops the worker with a `SinkHalted` (a 403 on `GET /1/indexes`) at the start of every rebuild, before anything is created. The sink does not skip the clean-up when the key cannot list: the clean-up is also what stops a worker that has lost its lease from moving its own half-built index over the served one.
+
+## What it does
+
+- **Writes.** Each batch is one Algolia `batch` call of full-record writes: `addObject` with `objectID` set to the feed key, or `deleteObject`. Nothing partial is ever sent, so resending a batch after a crash is harmless.
+- **Confirmation.** The feed's cursor moves only once Algolia has published the batch's task. `taskPolling` sets how the task is polled (default: from 100 ms, doubling to 2 s, giving up after 60 s, after which the batch is resent).
+- **Oversized records.** Algolia refuses a whole batch for one record over the plan's size limit. The sink measures every record first (its JSON in UTF-8 bytes, `objectID` included) and refuses the oversized ones itself, so they are parked and the rest of the batch goes through. The default limit, `maxRecordBytes: 10_000`, is the smallest plan's, so out of the box nothing is parked that Algolia would have taken. On a plan with a larger record limit, raise `maxRecordBytes` to that limit (for example `100_000` for 100 KB records). Do not set it above your plan's limit: such a record then costs a round trip that Algolia refuses before it is parked.
+- **Hosts.** Unless you set `host`, the sink talks to `<appId>.algolia.net` and, when that host does not answer in time, answers with a 5xx, or answers with a 1xx or 3xx, tries the same request on Algolia's fallback hosts `<appId>-1.algolianet.com` to `<appId>-3.algolianet.com`, in random order. A host that failed that way is skipped by the sink for two minutes, so a host that has gone quiet costs one timeout rather than one per call; when every host is down, all of them are tried again. `timeouts` sets how long one request to one host may take, as `connect`, `read` and `write` (defaults 2 s, 5 s and 30 s, the official JavaScript client's); a read (a `GET`, or a `browse`) may take `connect` plus `read`, and every other call `connect` plus `write`. Setting `host` sends every call to that one base URL and turns this failover off.
+- **Failures.** A 429, or a 5xx or no answer from every host tried, is retried after a back-off that honours `Retry-After`. A 429 is not tried on another host, since it is your application's rate limit; a 429 whose message names a quota (your record quota, or your allowance of indexing by file size) stops the worker instead, with Algolia's message, since waiting does not lift it. A 400 that names a record parks that record. Any other 4xx (bad credentials, a missing ACL) stops the worker, with Algolia's message.
+- **Rebuilds.** When the read model is rebuilt, the sink builds a fresh index named `<index>__kairos_g<generation>_<attempt>`, where `<attempt>` is random and new on every rebuild attempt, copying only the served index's settings, synonyms and rules into it. It loads every record there and then moves it over the served index. Search keeps serving the old index until the move. The rebuild marks its fresh index with one key, `kairosResyncAttempt`, in the index's `userData` setting. Just after the move it puts the served index's settings back exactly as they were before the rebuild began, by copying them from a snapshot it took then (`<index>__kairos_g<generation>_<attempt>__settings`, deleted afterwards). So your own `userData` is kept as it was, and an index that had none still has none. Algolia keeps a `userData: null` as a stored value, so a settings write could not do that. The one exception: if a restore fails, the sink logs a warning and leaves the mark on the served index, and the next rebuild takes it off. If the mark was all the `userData` there was, that index then ends with `userData: null`. Your `userData` must therefore be a JSON object, or absent: a rebuild of an index whose `userData` is anything else stops the worker. A rebuild whose fresh index was deleted under it (by a newer rebuild) is refused rather than moved over the served index, whether it was written to again or deleted just before the move; the sink checks that the served index carries the move's mark before it reports the rebuild published. Starting a rebuild also deletes every other `<index>__kairos_g…` index an earlier, abandoned rebuild left behind, so an interrupted rebuild does not keep holding records against your quota.
+- **Reconcile.** The sink can list the served index's object ids (by `browse`, retrieving the ids alone), so the feed can delete records the index holds and the feed no longer has.
+
+## Replicas
+
+Algolia does not document how replicas behave across a scoped copy and a move. The expectation this package is built on is that the resync index is not given the served index's replicas, and that the served index keeps its replicas across the move and across the settings restore that follows it. The package's own tests check both against a real application when credentials are available. Until you have seen that hold for your application, check your replicas after a first rebuild.

package/dist/cjs/classify.js

@@ -0,0 +1,157 @@
+"use strict";
+
+Object.defineProperty(exports, "__esModule", {
+  value: true
+});
+exports.unreadableAnswer = exports.transportFailure = exports.parseRetryAfter = exports.objectIdNamedIn = exports.classifyAnswer = exports.QUOTA_429 = void 0;
+var _feed = require("@kairos-es/feed");
+var _effect = require("effect");
+/**
+ * How an Algolia answer becomes one of the feed's three sink failures.
+ *
+ * The driver's whole behaviour on failure follows from which of the three a sink
+ * names (`@kairos-es/feed`'s `errors.ts` owns that), so this mapping is the most
+ * consequential thing in the package and it is kept pure, in one place, and tested
+ * on its own. The rules:
+ *
+ * - **A 429 is `SinkRetryable`**, carrying `Retry-After` when Algolia sends one,
+ *   so the driver backs off no less than Algolia asked — UNLESS its message names
+ *   a QUOTA rather than a rate: Algolia answers 429 for an application over its
+ *   record quota or its indexing file-size allowance too, and neither lifts by
+ *   waiting, so retrying would only hold the feed until supervision reported it
+ *   stalled, with Algolia's reason lost. Such a 429 is `SinkHalted`, carrying
+ *   Algolia's message. The wordings recognised are {@link QUOTA_429}; a 429 whose
+ *   wording is not among them stays retryable, which is the safe side to err on.
+ * - **Every 5xx is `SinkRetryable`, and so is a 1xx or 3xx**, a status no call
+ *   here expects and one the transport has already tried on every host, as the
+ *   official client does. A transport failure — no answer at all — is retryable
+ *   too: the request may never have arrived, and resending is safe because every
+ *   write this sink makes is a full-record replace or a delete.
+ * - **A 400 that names an `objectID` in the batch is `SinkRejected` for that key.**
+ *   Algolia refuses a whole batch for one bad record, and names the record in its
+ *   message (`… objectID=<id> …`). Parking just that key lets the driver send the
+ *   rest again without it, which is the difference between one record missing from
+ *   search and the whole feed stopping. The key has to be one the batch carried: an
+ *   id the batch did not send is not a record the driver can park, so such an
+ *   answer falls through to the next rule. The message does not delimit the id, and
+ *   a feed key may hold a space, a comma or anything else, so the id is not parsed
+ *   out of the message: the batch's own keys are matched against what follows
+ *   `objectID=` (`objectIdNamedIn`).
+ * - **Every other 4xx is `SinkHalted`.** Bad credentials, a key without the right
+ *   ACL, an index name Algolia will not accept, a request this package built wrong:
+ *   none of them is mended by sending again, so the worker stops and supervision
+ *   reports it.
+ *
+ * Nothing here reads the request, and nothing here sees a header but `Retry-After`,
+ * so no classification can carry the admin key into a reason string.
+ */
+
+const OBJECT_ID_MARKER = 'objectID=';
+/** What may follow an id in Algolia's message: the end, a space or punctuation. */
+const ID_ENDS = /^(?:$|[\s,;.:)])/;
+/**
+ * The key of `batchKeys` an Algolia error message names, when it names one.
+ *
+ * Algolia's record-level refusals all carry `objectID=<id>` (the record-too-big
+ * message is the common one), with nothing marking where the id ends. So each
+ * batch key is tried against the text after each `objectID=`, and it matches when
+ * that text starts with the key and the key is followed by the end of the message,
+ * a space or punctuation; the longest key that matches wins, so of `big` and
+ * `big one` the message `objectID=big one is too big` names the second.
+ */
+const objectIdNamedIn = (message, batchKeys) => {
+  let named;
+  let from = message.indexOf(OBJECT_ID_MARKER);
+  while (from >= 0) {
+    const after = message.slice(from + OBJECT_ID_MARKER.length);
+    for (const key of batchKeys) {
+      if (after.startsWith(key) && ID_ENDS.test(after.slice(key.length)) && (named === undefined || key.length > named.length)) {
+        named = key;
+      }
+    }
+    from = message.indexOf(OBJECT_ID_MARKER, from + 1);
+  }
+  return _effect.Option.fromUndefinedOr(named);
+};
+/**
+ * The 429 wordings that name a quota, not a rate, matched case-insensitively
+ * anywhere in Algolia's message: an application over its record quota, or over
+ * its allowance of indexing by file size. See the module doc for why each halts.
+ */
+exports.objectIdNamedIn = objectIdNamedIn;
+const QUOTA_429 = exports.QUOTA_429 = [/quota/i, /file size/i];
+/**
+ * `Retry-After` as a duration, given the current time.
+ *
+ * The header is either a whole number of seconds or an HTTP date. A date is turned
+ * into a wait against `now`, which the caller reads from `Clock` so a test's
+ * `TestClock` governs it; a date already past is a wait of zero. Anything else is
+ * ignored rather than guessed at, and the driver's own back-off applies.
+ */
+const parseRetryAfter = (header, nowMillis) => {
+  const trimmed = header.trim();
+  if (/^\d+$/.test(trimmed)) {
+    return _effect.Option.some(_effect.Duration.seconds(Number(trimmed)));
+  }
+  // A bare number with a fraction or a sign is neither form; do not let the date
+  // parser read it as a year.
+  if (/^[-+\d.]+$/.test(trimmed)) {
+    return _effect.Option.none();
+  }
+  return _effect.Option.map(_effect.DateTime.make(trimmed), at => _effect.Duration.millis(Math.max(0, _effect.DateTime.toEpochMillis(at) - nowMillis)));
+};
+/**
+ * Classify an Algolia error answer; see the module doc for the rules.
+ *
+ * `batchKeys` is the set of keys the request carried, empty for a call that sends
+ * no records, so only a write can ever produce a `SinkRejected`.
+ */
+exports.parseRetryAfter = parseRetryAfter;
+const classifyAnswer = (call, answer, batchKeys, nowMillis) => {
+  const reason = `Algolia answered ${answer.status} to ${call.method} ${call.path}: ${answer.message}`;
+  if (answer.status === 429 && QUOTA_429.some(wording => wording.test(answer.message))) {
+    return new _feed.SinkHalted({
+      reason
+    });
+  }
+  if (answer.status === 429 || answer.status >= 500 || answer.status < 200 || answer.status >= 300 && answer.status < 400) {
+    const retryAfter = _effect.Option.flatMap(answer.retryAfter, header => parseRetryAfter(header, nowMillis));
+    return new _feed.SinkRetryable({
+      reason,
+      ...(_effect.Option.isSome(retryAfter) ? {
+        retryAfter: retryAfter.value
+      } : {})
+    });
+  }
+  if (answer.status === 400) {
+    const named = objectIdNamedIn(answer.message, batchKeys);
+    if (_effect.Option.isSome(named)) {
+      const key = named.value;
+      return new _feed.SinkRejected({
+        keys: [key],
+        reasons: new Map([[key, answer.message]])
+      });
+    }
+  }
+  return new _feed.SinkHalted({
+    reason
+  });
+};
+/**
+ * A request that got no answer at all. Retryable: see the module doc for why a
+ * resend is safe.
+ */
+exports.classifyAnswer = classifyAnswer;
+const transportFailure = (call, detail) => new _feed.SinkRetryable({
+  reason: `no answer from Algolia to ${call.method} ${call.path}: ${detail}`
+});
+/**
+ * An answer this package cannot read: a success whose body is not the shape
+ * Algolia documents. Halted, because sending again will get the same body back.
+ */
+exports.transportFailure = transportFailure;
+const unreadableAnswer = (call, detail) => new _feed.SinkHalted({
+  reason: `unreadable answer from Algolia to ${call.method} ${call.path}: ${detail}`
+});
+exports.unreadableAnswer = unreadableAnswer;
+//# sourceMappingURL=classify.js.map

package/dist/cjs/classify.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"classify.js","names":["_feed","require","_effect","OBJECT_ID_MARKER","ID_ENDS","objectIdNamedIn","message","batchKeys","named","from","indexOf","after","slice","length","key","startsWith","test","undefined","Option","fromUndefinedOr","exports","QUOTA_429","parseRetryAfter","header","nowMillis","trimmed","trim","some","Duration","seconds","Number","none","map","DateTime","make","at","millis","Math","max","toEpochMillis","classifyAnswer","call","answer","reason","status","method","path","wording","SinkHalted","retryAfter","flatMap","SinkRetryable","isSome","value","SinkRejected","keys","reasons","Map","transportFailure","detail","unreadableAnswer"],"sources":["../../src/classify.ts"],"sourcesContent":[null],"mappings":";;;;;;AAyCA,IAAAA,KAAA,GAAAC,OAAA;AACA,IAAAC,OAAA,GAAAD,OAAA;AA1CA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AA8DA,MAAME,gBAAgB,GAAG,WAAW;AAEpC;AACA,MAAMC,OAAO,GAAG,kBAAkB;AAElC;;;;;;;;;;AAUO,MAAMC,eAAe,GAAGA,CAC7BC,OAAe,EACfC,SAA+B,KACL;EAC1B,IAAIC,KAA0B;EAC9B,IAAIC,IAAI,GAAGH,OAAO,CAACI,OAAO,CAACP,gBAAgB,CAAC;EAC5C,OAAOM,IAAI,IAAI,CAAC,EAAE;IAChB,MAAME,KAAK,GAAGL,OAAO,CAACM,KAAK,CAACH,IAAI,GAAGN,gBAAgB,CAACU,MAAM,CAAC;IAC3D,KAAK,MAAMC,GAAG,IAAIP,SAAS,EAAE;MAC3B,IACEI,KAAK,CAACI,UAAU,CAACD,GAAG,CAAC,IACrBV,OAAO,CAACY,IAAI,CAACL,KAAK,CAACC,KAAK,CAACE,GAAG,CAACD,MAAM,CAAC,CAAC,KACpCL,KAAK,KAAKS,SAAS,IAAIH,GAAG,CAACD,MAAM,GAAGL,KAAK,CAACK,MAAM,CAAC,EAClD;QACAL,KAAK,GAAGM,GAAG;MACb;IACF;IACAL,IAAI,GAAGH,OAAO,CAACI,OAAO,CAACP,gBAAgB,EAAEM,IAAI,GAAG,CAAC,CAAC;EACpD;EACA,OAAOS,cAAM,CAACC,eAAe,CAACX,KAAK,CAAC;AACtC,CAAC;AAED;;;;;AAAAY,OAAA,CAAAf,eAAA,GAAAA,eAAA;AAKO,MAAMgB,SAAS,GAAAD,OAAA,CAAAC,SAAA,GAA0B,CAAC,QAAQ,EAAE,YAAY,CAAC;AAExE;;;;;;;;AAQO,MAAMC,eAAe,GAAGA,CAC7BC,MAAc,EACdC,SAAiB,KACmB;EACpC,MAAMC,OAAO,GAAGF,MAAM,CAACG,IAAI,EAAE;EAC7B,IAAI,OAAO,CAACV,IAAI,CAACS,OAAO,CAAC,EAAE;IACzB,OAAOP,cAAM,CAACS,IAAI,CAACC,gBAAQ,CAACC,OAAO,CAACC,MAAM,CAACL,OAAO,CAAC,CAAC,CAAC;EACvD;EACA;EACA;EACA,IAAI,YAAY,CAACT,IAAI,CAACS,OAAO,CAAC,EAAE;IAC9B,OAAOP,cAAM,CAACa,IAAI,EAAE;EACtB;EACA,OAAOb,cAAM,CAACc,GAAG,CAACC,gBAAQ,CAACC,IAAI,CAACT,OAAO,CAAC,EAAGU,EAAE,IAC3CP,gBAAQ,CAACQ,MAAM,CAACC,IAAI,CAACC,GAAG,CAAC,CAAC,EAAEL,gBAAQ,CAACM,aAAa,CAACJ,EAAE,CAAC,GAAGX,SAAS,CAAC,CAAC,CACrE;AACH,CAAC;AAED;;;;;;AAAAJ,OAAA,CAAAE,eAAA,GAAAA,eAAA;AAMO,MAAMkB,cAAc,GAAGA,CAC5BC,IAAsB,EACtBC,MAA0B,EAC1BnC,SAA+B,EAC/BiB,SAAiB,KACC;EAClB,MAAMmB,MAAM,GAAG,oBAAoBD,MAAM,CAACE,MAAM,OAAOH,IAAI,CAACI,MAAM,IAAIJ,IAAI,CAACK,IAAI,KAAKJ,MAAM,CAACpC,OAAO,EAAE;EACpG,IACEoC,MAAM,CAACE,MAAM,KAAK,GAAG,IACrBvB,SAAS,CAACM,IAAI,CAAEoB,OAAO,IAAKA,OAAO,CAAC/B,IAAI,CAAC0B,MAAM,CAACpC,OAAO,CAAC,CAAC,EACzD;IACA,OAAO,IAAI0C,gBAAU,CAAC;MAAEL;IAAM,CAAE,CAAC;EACnC;EACA,IACED,MAAM,CAACE,MAAM,KAAK,GAAG,IACrBF,MAAM,CAACE,MAAM,IAAI,GAAG,IACpBF,MAAM,CAACE,MAAM,GAAG,GAAG,IAClBF,MAAM,CAACE,MAAM,IAAI,GAAG,IAAIF,MAAM,CAACE,MAAM,GAAG,GAAI,EAC7C;IACA,MAAMK,UAAU,GAAG/B,cAAM,CAACgC,OAAO,CAACR,MAAM,CAACO,UAAU,EAAG1B,MAAM,IAC1DD,eAAe,CAACC,MAAM,EAAEC,SAAS,CAAC,CACnC;IACD,OAAO,IAAI2B,mBAAa,CAAC;MACvBR,MAAM;MACN,IAAIzB,cAAM,CAACkC,MAAM,CAACH,UAAU,CAAC,GAAG;QAAEA,UAAU,EAAEA,UAAU,CAACI;MAAK,CAAE,GAAG,EAAE;KACtE,CAAC;EACJ;EACA,IAAIX,MAAM,CAACE,MAAM,KAAK,GAAG,EAAE;IACzB,MAAMpC,KAAK,GAAGH,eAAe,CAACqC,MAAM,CAACpC,OAAO,EAAEC,SAAS,CAAC;IACxD,IAAIW,cAAM,CAACkC,MAAM,CAAC5C,KAAK,CAAC,EAAE;MACxB,MAAMM,GAAG,GAAGN,KAAK,CAAC6C,KAAK;MACvB,OAAO,IAAIC,kBAAY,CAAC;QACtBC,IAAI,EAAE,CAACzC,GAAG,CAAC;QACX0C,OAAO,EAAE,IAAIC,GAAG,CAAC,CAAC,CAAC3C,GAAG,EAAE4B,MAAM,CAACpC,OAAO,CAAC,CAAC;OACzC,CAAC;IACJ;EACF;EACA,OAAO,IAAI0C,gBAAU,CAAC;IAAEL;EAAM,CAAE,CAAC;AACnC,CAAC;AAED;;;;AAAAvB,OAAA,CAAAoB,cAAA,GAAAA,cAAA;AAIO,MAAMkB,gBAAgB,GAAGA,CAC9BjB,IAAsB,EACtBkB,MAAc,KAEd,IAAIR,mBAAa,CAAC;EAChBR,MAAM,EAAE,6BAA6BF,IAAI,CAACI,MAAM,IAAIJ,IAAI,CAACK,IAAI,KAAKa,MAAM;CACzE,CAAC;AAEJ;;;;AAAAvC,OAAA,CAAAsC,gBAAA,GAAAA,gBAAA;AAIO,MAAME,gBAAgB,GAAGA,CAC9BnB,IAAsB,EACtBkB,MAAc,KAEd,IAAIX,gBAAU,CAAC;EACbL,MAAM,EAAE,qCAAqCF,IAAI,CAACI,MAAM,IAAIJ,IAAI,CAACK,IAAI,KAAKa,MAAM;CACjF,CAAC;AAAAvC,OAAA,CAAAwC,gBAAA,GAAAA,gBAAA","ignoreList":[]}

package/dist/cjs/identifiers.js

@@ -0,0 +1,38 @@
+"use strict";
+
+Object.defineProperty(exports, "__esModule", {
+  value: true
+});
+exports.AlgoliaTaskId = exports.AlgoliaIndexName = exports.AlgoliaAppId = void 0;
+var _effect = require("effect");
+/**
+ * The Algolia sink's branded identifiers.
+ *
+ * Three values cross this package's boundary that would otherwise be bare
+ * strings and numbers, and each is easy to pass in the place of another: an
+ * application id where an index name belongs, a feed key where an index name
+ * belongs, a task id from one index polled against another. Branding them is the
+ * repository's standing rule for a domain identifier, and here it is also what
+ * keeps a `Receipt` — an index and a task id — from being assembled by hand out of
+ * values that never belonged together.
+ */
+
+/**
+ * The Algolia application id: the first half of the credentials, and the name of
+ * the application's hosts. Not a secret — it is sent beside every search a browser
+ * makes — so it is a plain branded string, unlike the admin key.
+ */
+const AlgoliaAppId = exports.AlgoliaAppId = /*#__PURE__*/_effect.Schema.String.pipe(/*#__PURE__*/_effect.Schema.check(/*#__PURE__*/_effect.Schema.isNonEmpty()), /*#__PURE__*/_effect.Schema.brand('AlgoliaAppId'));
+/**
+ * The name of an Algolia index. The sink's SERVED index is one; a resync builds into
+ * another, named after the served one, the generation it rebuilds and the attempt.
+ */
+const AlgoliaIndexName = exports.AlgoliaIndexName = /*#__PURE__*/_effect.Schema.String.pipe(/*#__PURE__*/_effect.Schema.check(/*#__PURE__*/_effect.Schema.isNonEmpty()), /*#__PURE__*/_effect.Schema.brand('AlgoliaIndexName'));
+/**
+ * An Algolia task id: what every write returns, and what `getTask` is polled with
+ * until the write is published. A task belongs to the index it was issued on, and
+ * is polled there and nowhere else — which is why a receipt carries the index
+ * beside it.
+ */
+const AlgoliaTaskId = exports.AlgoliaTaskId = /*#__PURE__*/_effect.Schema.Number.pipe(/*#__PURE__*/_effect.Schema.check(/*#__PURE__*/_effect.Schema.isInt(), /*#__PURE__*/_effect.Schema.isGreaterThanOrEqualTo(0)), /*#__PURE__*/_effect.Schema.brand('AlgoliaTaskId'));
+//# sourceMappingURL=identifiers.js.map

package/dist/cjs/identifiers.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"identifiers.js","names":["_effect","require","AlgoliaAppId","exports","Schema","String","pipe","check","isNonEmpty","brand","AlgoliaIndexName","AlgoliaTaskId","Number","isInt","isGreaterThanOrEqualTo"],"sources":["../../src/identifiers.ts"],"sourcesContent":[null],"mappings":";;;;;;AAWA,IAAAA,OAAA,GAAAC,OAAA;AAXA;;;;;;;;;;;;AAaA;;;;;AAKO,MAAMC,YAAY,GAAAC,OAAA,CAAAD,YAAA,gBAAGE,cAAM,CAACC,MAAM,CAACC,IAAI,cAC5CF,cAAM,CAACG,KAAK,cAACH,cAAM,CAACI,UAAU,EAAE,CAAC,eACjCJ,cAAM,CAACK,KAAK,CAAC,cAAc,CAAC,CAC7B;AAGD;;;;AAIO,MAAMC,gBAAgB,GAAAP,OAAA,CAAAO,gBAAA,gBAAGN,cAAM,CAACC,MAAM,CAACC,IAAI,cAChDF,cAAM,CAACG,KAAK,cAACH,cAAM,CAACI,UAAU,EAAE,CAAC,eACjCJ,cAAM,CAACK,KAAK,CAAC,kBAAkB,CAAC,CACjC;AAGD;;;;;;AAMO,MAAME,aAAa,GAAAR,OAAA,CAAAQ,aAAA,gBAAGP,cAAM,CAACQ,MAAM,CAACN,IAAI,cAC7CF,cAAM,CAACG,KAAK,cAACH,cAAM,CAACS,KAAK,EAAE,eAAET,cAAM,CAACU,sBAAsB,CAAC,CAAC,CAAC,CAAC,eAC9DV,cAAM,CAACK,KAAK,CAAC,eAAe,CAAC,CAC9B","ignoreList":[]}

package/dist/cjs/index.js

@@ -0,0 +1,39 @@
+"use strict";
+
+Object.defineProperty(exports, "__esModule", {
+  value: true
+});
+Object.defineProperty(exports, "AlgoliaAppId", {
+  enumerable: true,
+  get: function () {
+    return _identifiers.AlgoliaAppId;
+  }
+});
+Object.defineProperty(exports, "AlgoliaIndexName", {
+  enumerable: true,
+  get: function () {
+    return _identifiers.AlgoliaIndexName;
+  }
+});
+Object.defineProperty(exports, "AlgoliaTaskId", {
+  enumerable: true,
+  get: function () {
+    return _identifiers.AlgoliaTaskId;
+  }
+});
+Object.defineProperty(exports, "DEFAULT_MAX_RECORD_BYTES", {
+  enumerable: true,
+  get: function () {
+    return _records.DEFAULT_MAX_RECORD_BYTES;
+  }
+});
+Object.defineProperty(exports, "makeAlgoliaSink", {
+  enumerable: true,
+  get: function () {
+    return _sink.makeAlgoliaSink;
+  }
+});
+var _identifiers = require("./identifiers.js");
+var _records = require("./records.js");
+var _sink = require("./sink.js");
+//# sourceMappingURL=index.js.map

package/dist/cjs/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","names":["_identifiers","require","_records","_sink"],"sources":["../../src/index.ts"],"sourcesContent":[null],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAmCA,IAAAA,YAAA,GAAAC,OAAA;AAKA,IAAAC,QAAA,GAAAD,OAAA;AAEA,IAAAE,KAAA,GAAAF,OAAA","ignoreList":[]}

package/dist/cjs/records.js

@@ -0,0 +1,105 @@
+"use strict";
+
+Object.defineProperty(exports, "__esModule", {
+  value: true
+});
+exports.recordBytes = exports.encodeBatch = exports.DEFAULT_MAX_RECORD_BYTES = void 0;
+var _feed = require("@kairos-es/feed");
+var _effect = require("effect");
+/**
+ * Turning a batch of feed changes into Algolia batch operations, and refusing the
+ * records Algolia would refuse BEFORE they are sent.
+ *
+ * ## Full-record replace, and nothing partial
+ *
+ * An upsert is `addObject` with the record's `objectID` set to the feed key, which
+ * replaces whatever Algolia held under that id; a delete is `deleteObject`. Both
+ * leave the index the same however many times they are sent, which is the
+ * idempotence the driver relies on when it resends a batch it could not confirm.
+ * Algolia's partial operations (`partialUpdateObject`, the built-in `Increment`
+ * and its kin) do not: applied twice they apply twice. They are never used.
+ *
+ * The key is written LAST, over the encoded document, so a document that carries an
+ * `objectID` field of its own cannot point a record at another key.
+ *
+ * ## Why oversized records are refused here
+ *
+ * Algolia refuses a whole `batch` when one record in it is over the plan's record
+ * size limit. Its answer does name the record, and `classify.ts` turns that into a
+ * rejection of that one key — but only after a round trip that wrote nothing, and
+ * only for the first oversized record the batch held. Checking every record's size
+ * up front rejects all of them at once, so the driver's resend of the rest goes
+ * through first time. The 400 path stays as the backstop for a limit configured
+ * higher than the plan's.
+ *
+ * The size is the record's JSON in UTF-8 bytes, `objectID` included, which is how
+ * Algolia measures it. The default limit is the smallest Algolia plan's (10 KB);
+ * an application on a larger plan raises it.
+ *
+ * ## A document that does not encode
+ *
+ * The document schema is the read model's, and a document it cannot encode — or one
+ * that encodes to something other than a JSON object, which is all an Algolia
+ * record can be — is a defect in the read model, not a record the sink refuses. It
+ * dies, loudly, rather than being parked where nobody looks.
+ */
+
+/** The smallest Algolia plan's record size limit, in bytes. */
+const DEFAULT_MAX_RECORD_BYTES = exports.DEFAULT_MAX_RECORD_BYTES = 10_000;
+const utf8 = /*#__PURE__*/new TextEncoder();
+/** A record's size as Algolia measures it: its JSON, in UTF-8 bytes. */
+const recordBytes = record => utf8.encode(JSON.stringify(record)).length;
+exports.recordBytes = recordBytes;
+const isJsonObject = u => typeof u === 'object' && u !== null && !Array.isArray(u);
+/**
+ * Encode a batch, failing `SinkRejected` with EVERY oversized key when there is
+ * one; see the module doc.
+ */
+const encodeBatch = (doc, maxRecordBytes, batch) => _effect.Effect.gen(function* () {
+  const encode = _effect.Schema.encodeEffect(doc);
+  const operations = [];
+  const keys = new Set();
+  const reasons = new Map();
+  for (const change of batch) {
+    keys.add(change.key);
+    if (change._tag === 'Delete') {
+      operations.push({
+        action: 'deleteObject',
+        body: {
+          objectID: change.key
+        }
+      });
+      continue;
+    }
+    const encoded = yield* _effect.Effect.orDie(encode(change.doc));
+    if (!isJsonObject(encoded)) {
+      return yield* _effect.Effect.die(new Error(`feed-algolia: the document for key '${change.key}' encodes to ${encoded === null ? 'null' : typeof encoded}, and an Algolia record must be a JSON object`));
+    }
+    const record = {
+      ...encoded,
+      objectID: change.key
+    };
+    const size = recordBytes(record);
+    if (size > maxRecordBytes) {
+      reasons.set(change.key, `the record is ${size} bytes, over the configured maxRecordBytes of ${maxRecordBytes}`);
+      continue;
+    }
+    operations.push({
+      action: 'addObject',
+      body: record
+    });
+  }
+  const rejected = Array.from(reasons.keys());
+  if (_effect.Array.isReadonlyArrayNonEmpty(rejected)) {
+    return yield* _effect.Effect.fail(new _feed.SinkRejected({
+      keys: rejected,
+      reasons
+    }));
+  }
+  return {
+    operations,
+    keys
+  };
+});
+exports.encodeBatch = encodeBatch;
+//# sourceMappingURL=records.js.map

package/dist/cjs/records.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"records.js","names":["_feed","require","_effect","DEFAULT_MAX_RECORD_BYTES","exports","utf8","TextEncoder","recordBytes","record","encode","JSON","stringify","length","isJsonObject","u","Array","isArray","encodeBatch","doc","maxRecordBytes","batch","Effect","gen","Schema","encodeEffect","operations","keys","Set","reasons","Map","change","add","key","_tag","push","action","body","objectID","encoded","orDie","die","Error","size","set","rejected","from","Arr","isReadonlyArrayNonEmpty","fail","SinkRejected"],"sources":["../../src/records.ts"],"sourcesContent":[null],"mappings":";;;;;;AAqCA,IAAAA,KAAA,GAAAC,OAAA;AACA,IAAAC,OAAA,GAAAD,OAAA;AAtCA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AA0CA;AACO,MAAME,wBAAwB,GAAAC,OAAA,CAAAD,wBAAA,GAAG,MAAM;AAE9C,MAAME,IAAI,gBAAG,IAAIC,WAAW,EAAE;AAE9B;AACO,MAAMC,WAAW,GACtBC,MAAyC,IAC9BH,IAAI,CAACI,MAAM,CAACC,IAAI,CAACC,SAAS,CAACH,MAAM,CAAC,CAAC,CAACI,MAAM;AAAAR,OAAA,CAAAG,WAAA,GAAAA,WAAA;AAQvD,MAAMM,YAAY,GAAIC,CAAU,IAC9B,OAAOA,CAAC,KAAK,QAAQ,IAAIA,CAAC,KAAK,IAAI,IAAI,CAACC,KAAK,CAACC,OAAO,CAACF,CAAC,CAAC;AAE1D;;;;AAIO,MAAMG,WAAW,GAAGA,CACzBC,GAAuB,EACvBC,cAAsB,EACtBC,KAA2C,KAE3CC,cAAM,CAACC,GAAG,CAAC,aAAS;EAClB,MAAMb,MAAM,GAAGc,cAAM,CAACC,YAAY,CAACN,GAAG,CAAC;EACvC,MAAMO,UAAU,GAA0B,EAAE;EAC5C,MAAMC,IAAI,GAAG,IAAIC,GAAG,EAAW;EAC/B,MAAMC,OAAO,GAAG,IAAIC,GAAG,EAAmB;EAC1C,KAAK,MAAMC,MAAM,IAAIV,KAAK,EAAE;IAC1BM,IAAI,CAACK,GAAG,CAACD,MAAM,CAACE,GAAG,CAAC;IACpB,IAAIF,MAAM,CAACG,IAAI,KAAK,QAAQ,EAAE;MAC5BR,UAAU,CAACS,IAAI,CAAC;QACdC,MAAM,EAAE,cAAc;QACtBC,IAAI,EAAE;UAAEC,QAAQ,EAAEP,MAAM,CAACE;QAAG;OAC7B,CAAC;MACF;IACF;IACA,MAAMM,OAAO,GAAY,OAAOjB,cAAM,CAACkB,KAAK,CAAC9B,MAAM,CAACqB,MAAM,CAACZ,GAAG,CAAC,CAAC;IAChE,IAAI,CAACL,YAAY,CAACyB,OAAO,CAAC,EAAE;MAC1B,OAAO,OAAOjB,cAAM,CAACmB,GAAG,CACtB,IAAIC,KAAK,CACP,uCAAuCX,MAAM,CAACE,GAAG,gBAAgBM,OAAO,KAAK,IAAI,GAAG,MAAM,GAAG,OAAOA,OAAO,+CAA+C,CAC3J,CACF;IACH;IACA,MAAM9B,MAAM,GAAG;MAAE,GAAG8B,OAAO;MAAED,QAAQ,EAAEP,MAAM,CAACE;IAAG,CAAE;IACnD,MAAMU,IAAI,GAAGnC,WAAW,CAACC,MAAM,CAAC;IAChC,IAAIkC,IAAI,GAAGvB,cAAc,EAAE;MACzBS,OAAO,CAACe,GAAG,CACTb,MAAM,CAACE,GAAG,EACV,iBAAiBU,IAAI,iDAAiDvB,cAAc,EAAE,CACvF;MACD;IACF;IACAM,UAAU,CAACS,IAAI,CAAC;MAAEC,MAAM,EAAE,WAAW;MAAEC,IAAI,EAAE5B;IAAM,CAAE,CAAC;EACxD;EACA,MAAMoC,QAAQ,GAAG7B,KAAK,CAAC8B,IAAI,CAACjB,OAAO,CAACF,IAAI,EAAE,CAAC;EAC3C,IAAIoB,aAAG,CAACC,uBAAuB,CAACH,QAAQ,CAAC,EAAE;IACzC,OAAO,OAAOvB,cAAM,CAAC2B,IAAI,CAAC,IAAIC,kBAAY,CAAC;MAAEvB,IAAI,EAAEkB,QAAQ;MAAEhB;IAAO,CAAE,CAAC,CAAC;EAC1E;EACA,OAAO;IAAEH,UAAU;IAAEC;EAAI,CAAE;AAC7B,CAAC,CAAC;AAAAtB,OAAA,CAAAa,WAAA,GAAAA,WAAA","ignoreList":[]}