@atproto/lex 0.0.22 → 0.0.24

package/CHANGELOG.md

@@ -1,5 +1,31 @@
 # @atproto/lex
 
+## 0.0.24
+
+### Patch Changes
+
+- [#4828](https://github.com/bluesky-social/atproto/pull/4828) [`c62651d`](https://github.com/bluesky-social/atproto/commit/c62651dd69f1e18bd854b66e499b91fee9eaa856) Thanks [@matthieusieben](https://github.com/matthieusieben)! - Accept legacy blob references in non-strict mode. Legacy blob references (objects with `cid` and `mimeType` properties) are now accepted when `strict: false`, which is the default behavior when `strictResponseProcessing` is disabled on the Client.
+
+  BREAKING: The `allowLegacy` option has been removed from the blob schema builder, and legacy blobs are now handled automatically based on the strictness mode: in strict mode they are rejected, and in non-strict mode they are accepted. Consumers should stop passing `allowLegacy` and rely on strictness configuration instead. Likewise, CLI consumers should stop using the removed `--allowLegacyBlobs` flag and use the default strict/non-strict behavior.
+
+- Updated dependencies [[`c62651d`](https://github.com/bluesky-social/atproto/commit/c62651dd69f1e18bd854b66e499b91fee9eaa856), [`f6f100c`](https://github.com/bluesky-social/atproto/commit/f6f100c33700a7ff58a1458109cc7420131feed0), [`b6b231f`](https://github.com/bluesky-social/atproto/commit/b6b231f9c05cf90239d4a29aa0ae2592ea5ce928), [`c62651d`](https://github.com/bluesky-social/atproto/commit/c62651dd69f1e18bd854b66e499b91fee9eaa856), [`c62651d`](https://github.com/bluesky-social/atproto/commit/c62651dd69f1e18bd854b66e499b91fee9eaa856), [`f6f100c`](https://github.com/bluesky-social/atproto/commit/f6f100c33700a7ff58a1458109cc7420131feed0), [`f6f100c`](https://github.com/bluesky-social/atproto/commit/f6f100c33700a7ff58a1458109cc7420131feed0)]:
+  - @atproto/lex-data@0.0.15
+  - @atproto/lex-client@0.0.19
+  - @atproto/lex-schema@0.0.18
+  - @atproto/lex-json@0.0.15
+  - @atproto/lex-installer@0.0.24
+  - @atproto/lex-builder@0.0.21
+
+## 0.0.23
+
+### Patch Changes
+
+- Updated dependencies [[`527f5d4`](https://github.com/bluesky-social/atproto/commit/527f5d4c5d0c9264c2ff6f23ad06a41163fc6809), [`527f5d4`](https://github.com/bluesky-social/atproto/commit/527f5d4c5d0c9264c2ff6f23ad06a41163fc6809), [`c4df84c`](https://github.com/bluesky-social/atproto/commit/c4df84cd78df68ee8cb7289e7b61b3a032ad484e), [`527f5d4`](https://github.com/bluesky-social/atproto/commit/527f5d4c5d0c9264c2ff6f23ad06a41163fc6809), [`a99dd58`](https://github.com/bluesky-social/atproto/commit/a99dd58b5fe1995e571cf5e7b0105355583efa93), [`e5e5bcf`](https://github.com/bluesky-social/atproto/commit/e5e5bcf85fbc0d418f05724d684e7265be6a0be9), [`ac6bd18`](https://github.com/bluesky-social/atproto/commit/ac6bd18f1dc3397dd29008eff2a1e40702a4e138), [`c5c6c7d`](https://github.com/bluesky-social/atproto/commit/c5c6c7dac3b08e5f63cc918f57705573028ad797)]:
+  - @atproto/lex-schema@0.0.17
+  - @atproto/lex-client@0.0.18
+  - @atproto/lex-builder@0.0.20
+  - @atproto/lex-installer@0.0.23
+
 ## 0.0.22
 
 ### Patch Changes

package/README.md

@@ -10,7 +10,7 @@ Type-safe Lexicon tooling for AT Protocol data.
 - Tree-shaking and composition friendly
 
 ```typescript
-// Build data with generated builders and validators
+// Build and validate data with generated utilities
 
 const newPost = app.bsky.feed.post.$build({
   text: 'Hello, world!',
@@ -194,7 +194,6 @@ Options:
 - `--exclude <patterns...>` - List of strings or regex patterns to exclude lexicon documents by their IDs
 - `--include <patterns...>` - List of strings or regex patterns to include lexicon documents by their IDs
 - `--lib <package>` - Package name of the library to import the lex schema utility "l" from (default: `@atproto/lex`)
-- `--allowLegacyBlobs` - Allow generating schemas that accept legacy blob references (disabled by default; enable this if you encounter issues while processing records created a long time ago)
 - `--importExt <ext>` - File extension to use for import statements in generated files (default: `.js`). Use `--importExt ""` to generate extension-less imports
 - `--fileExt <ext>` - File extension to use for generated files (default: `.ts`)
 - `--indexFile` - Generate an "index" file that re-exports all root-level namespaces (disabled by default)
@@ -222,7 +221,7 @@ function renderPost(p: app.bsky.feed.post.Main) {
 
 ### Building data
 
-It is recommended to use the generated builders to create data that conforms to the schema. This ensures that all required fields are present.
+It is recommended to use the generated builders to create data that conforms to the schema. TypeScript ensures that all required fields are present at compile time.
 
 ```typescript
 import { l } from '@atproto/lex'
@@ -234,6 +233,10 @@ const post = app.bsky.feed.post.$build({
   text: 'Hello, world!',
   createdAt: l.toDatetimeString(new Date()),
 })
+
+// For runtime validation, use $parse()/$validate() instead
+const postWithDefaults = app.bsky.feed.post.$parse(post)
+app.bsky.feed.post.$validate(post)
 ```
 
 ### Validation Helpers
@@ -347,7 +350,7 @@ if (result.success) {
 }
 ```
 
-All schema methods that perform validation (`$parse`, `$safeParse`, `$validate`, `$safeValidate`) accept an optional `{ strict }` option. When `strict` is `false`, validation becomes more lenient: datetime string format checks are relaxed (e.g. datetimes without timezones are accepted; other string formats remain strict), blob MIME type and size constraints are not enforced, and non-raw CIDs are allowed in blob references. This is primarily used internally by the XRPC client when `strictResponseProcessing` is disabled, but can also be used directly:
+All schema methods that perform validation (`$parse`, `$safeParse`, `$validate`, `$safeValidate`) accept an optional `{ strict }` option. When `strict` is `false`, validation becomes more lenient: datetime string format checks are relaxed (e.g. datetimes without timezones are accepted; other string formats remain strict), blob MIME type and size constraints are not enforced, non-raw CIDs are allowed in blob references, and legacy blob reference format (objects with `cid` and `mimeType` properties) is accepted. This is primarily used internally by the XRPC client when `strictResponseProcessing` is disabled, but can also be used directly:
 
 ```typescript
 // Strict mode (default) - rejects datetime without timezone
@@ -359,7 +362,7 @@ app.bsky.feed.post.$safeParse(data, { strict: false })
 
 #### `$build(data)` - Build with Defaults
 
-Builds data without needing to specify the `$type` property, and properly types the result:
+Builds data by adding the `$type` property and properly types the result. Note that `$build()` does not perform validation - use `$parse()` if you need validation:
 
 ```typescript
 import { l } from '@atproto/lex'
@@ -607,7 +610,7 @@ const client = new Client(session, {
 
 - **`validateRequest`** — When `true`, outgoing request bodies are validated against the Lexicon input schema before sending. Useful in development to catch errors early. Default: `false`.
 - **`validateResponse`** — When `true`, incoming response bodies are validated against the Lexicon output schema. Disabling this can improve performance when you trust the upstream service. Default: `true`.
-- **`strictResponseProcessing`** — When `true` (default), the client will strictly process responses according to Lex encoding rules, rejecting responses containing invalid Lex data (e.g. floating-point numbers, malformed `$bytes` or `$link` objects). When `false`, the client accepts such responses in a lenient mode: invalid values are returned as-is rather than being rejected or converted, `datetime` string format checks become more lenient (e.g. datetimes without timezones are accepted) while other string formats remain strict, blob MIME type and size constraints are not enforced, and legacy blob references are coerced into standard `BlobRef` objects. Default: `true`.
+- **`strictResponseProcessing`** — When `true` (default), the client will strictly process responses according to Lex encoding rules, rejecting responses containing invalid Lex data (e.g. floating-point numbers, malformed `$bytes` or `$link` objects). When `false`, the client accepts such responses in a lenient mode: invalid values are returned as-is rather than being rejected or converted, `datetime` string format checks become more lenient (e.g. datetimes without timezones are accepted) while other string formats remain strict, blob MIME type and size constraints are not enforced, and legacy blob reference format (objects with `cid` and `mimeType` properties) is accepted. Default: `true`.
 
 ### Core Methods
 
@@ -662,7 +665,8 @@ console.log(result.cid)
 Options:
 
 - `rkey` - Custom record key (auto-generated if not provided)
-- `validate` - Validate record against schema before creating
+- `validate` - Asks the PDS to validate the record against schema when processing the request
+- `validateRequest` - Validate the record locally against schema before submitting the request
 - `swapCommit` - CID for optimistic concurrency control
 
 #### `client.get()`
@@ -703,6 +707,8 @@ await client.put(app.bsky.actor.profile, {
 Options:
 
 - `rkey` - Record key (required for non-literal keys)
+- `validate` - Validate record against schema before updating (falls back to `validateRequest` option if not specified)
+- `validateRequest` - Alternative way to enable validation (used if `validate` is not specified)
 - `swapCommit` - Expected repo commit CID
 - `swapRecord` - Expected record CID
 
@@ -759,7 +765,7 @@ The `xrpcSafe()` method returns a union type that includes the success case (`Xr
 import {
   Client,
   XrpcResponseError,
-  XrpcUpstreamError,
+  XrpcInvalidResponseError,
   XrpcInternalError,
 } from '@atproto/lex'
 import * as com from './lexicons/com.js'
@@ -777,20 +783,26 @@ if (result.success) {
 } else {
   // Handle failure - result is an XrpcFailure
   if (result instanceof XrpcResponseError) {
-    // The server returned a valid XRPC error response
-    result.error // string (e.g. "HandleNotFound", "AuthenticationRequired", etc.)
+    // The server responded with an error status code (4xx or 5xx).
+    // This is used for all error responses, whether or not they have a valid XRPC error payload.
+
+    result.error // string (e.g. "HandleNotFound", "AuthenticationRequired", "UpstreamFailure", etc.)
     result.message // string
     result.response.status // number
     result.response.headers // Headers
-    result.payload // { body: { error: string, message?: string }; encoding: string }
-  } else if (result instanceof XrpcUpstreamError) {
-    // The response was not a valid XRPC response (e.g. malformed JSON,
-    // data does not match schema, connection dropped)
+    result.payload // undefined | { body: unknown; encoding: string }
+
+    // Coerce to a valid XRPC error payload using toJSON():
+    result.toJSON() // { error: string, message?: string }
+  } else if (result instanceof XrpcInvalidResponseError) {
+    // The response was truly invalid (3xx redirect, malformed JSON, schema mismatch, etc.).
+    // This is a more specific error for responses that are not processable.
+
     result.error // "UpstreamFailure"
     result.message // string
     result.response.status // number
     result.response.headers // Headers
-    result.payload // null | { body: unknown; encoding: string }
+    result.payload // undefined | { body: unknown; encoding: string }
   } else if (result instanceof XrpcInternalError) {
     // Something went wrong on the client side (network error, etc.)
     result.error // "InternalServerError"
@@ -810,9 +822,9 @@ if (result.success) {
 
 The `XrpcFailure<M>` type is a union of three error classes:
 
-1. **`XrpcResponseError`** - The server returned a valid XRPC error response (non-2xx with proper error payload)
+1. **`XrpcResponseError`** - The server responded with a 4xx/5xx error status code. This is used for all error responses from the upstream server.
 
-2. **`XrpcUpstreamError`** - The response was invalid or unprocessable (malformed JSON, schema mismatch, incomplete response)
+2. **`XrpcInvalidResponseError`** - The upstream server returned a 2xx/3xx that does not comply with XRPC specifications for successful responses. A sub-class, `XrpcResponseValidationError`, is used for payload schema validation failures specifically.
 
 3. **`XrpcInternalError`** - Client-side errors (network failures, timeouts, etc.)
 
@@ -919,8 +931,16 @@ import {
   ifDatetimeString, // Returns DatetimeString or undefined
 
   // Blob references
-  BlobRef, // { $type: 'blob', ref: Cid, mimeType: string, size: number }
-  isBlobRef, // Type guard for BlobRef objects
+  BlobRef, // TypedBlobRef | LegacyBlobRef
+  LegacyBlobRef, // { cid: string, mimeType: string }
+  TypedBlobRef, // { $type: 'blob', ref: Cid, mimeType: string, size: number }
+  isBlobRef, // Type guard for BlobRef (accepts both TypedBlobRef and LegacyBlobRef)
+  isLegacyBlobRef, // Type guard for LegacyBlobRef objects
+  isTypedBlobRef, // Type guard for TypedBlobRef objects
+  getBlobCid, // Extract Cid from BlobRef or LegacyBlobRef
+  getBlobCidString, // Extract CID string from BlobRef or LegacyBlobRef
+  getBlobMime, // Extract MIME type from BlobRef or LegacyBlobRef
+  getBlobSize, // Extract size from BlobRef (returns undefined for LegacyBlobRef)
 
   // Equality
   lexEquals, // Deep equality (handles CIDs and bytes)
@@ -947,7 +967,7 @@ isLanguageString('en-US') // true
 
 ### Datetime Strings
 
-Many AT Protocol records (such as posts, likes, and follows) include a `createdAt` field that expects a valid `DatetimeString`. While `new Date().toISOString()` produces a string that looks like a valid datetime, it is not guaranteed to always conform to the AT Protocol's [datetime format requirements](https://atproto.com/specs/lexicon#datetime) (for example, `Date` objects representing dates before year 10 or after year 9999 will produce non-conforming strings). To ensure correctness and type safety, use the `DatetimeString` utilities exported from `@atproto/lex`:
+Many AT Protocol records (such as posts, likes, and follows) include a `createdAt` field that expects a valid `DatetimeString`. While `new Date().toISOString()` produces a string that looks like a valid datetime, it is not guaranteed to always conform to the AT Protocol's [datetime format requirements](https://atproto.com/specs/lexicon#datetime) (for example, `Date` objects representing dates before year 0 or after year 9999 will produce non-conforming strings). To ensure correctness and type safety, use the `DatetimeString` utilities exported from `@atproto/lex`:
 
 - **`toDatetimeString(date: Date)`** - Converts a `Date` object into a valid `DatetimeString`, throwing an `InvalidDatetimeError` if the date cannot be represented as a valid AT Protocol datetime.
 - **`asDatetimeString(input: string)`** - Validates and casts an arbitrary string to `DatetimeString`, throwing an `InvalidDatetimeError` if the string does not conform.
@@ -1022,37 +1042,96 @@ This will make the generated code more easily tree-shakeable from places that im
 
 ### Blob references
 
-In AT Protocol, binary data (blobs) are referenced using `BlobRef`, which include metadata like MIME type and size. These references are what allow PDSs to determine which binary data ("files") is referenced by records.
+In AT Protocol, binary data (blobs) are referenced using blob references, which include metadata like MIME type and size. These references allow PDSs to determine which binary data ("files") is referenced by records.
+
+#### TypedBlobRef: The Current Standard
+
+The current standard format for blob references is `TypedBlobRef`:
 
 ```typescript
-import { BlobRef, isBlobRef } from '@atproto/lex'
+import { TypedBlobRef } from '@atproto/lex'
 
-const blobRef: BlobRef = {
+const blobRef: TypedBlobRef = {
   $type: 'blob',
   ref: parseCid('bafybeihdwdcefgh4dqkjv67uzcmw7ojee6xedzdetojuzjevtenxquvyku'),
   mimeType: 'image/png',
   size: 12345,
 }
+```
+
+**When creating new blobs**, always use the `TypedBlobRef` format. This is the format returned by `client.uploadBlob()` and expected by PDS endpoints.
+
+#### LegacyBlobRef: Historical Format
+
+Historically, blob references used a simpler format without the `$type` property:
 
-if (isBlobRef(blobRef)) {
-  console.log('Valid BlobRef:', blobRef.mimeType, blobRef.size)
+```typescript
+type LegacyBlobRef = {
+  cid: string // CID as a string (not a Cid object)
+  mimeType: string // No size property
 }
 ```
 
-> [!NOTE]
+**Legacy blob references still exist in the AT Protocol network** in older records created before the format migration. While new blobs should always be created as `TypedBlobRef`, your code must be prepared to handle both formats when reading existing data.
+
+#### Working with Both Formats
+
+The `BlobRef` type is a union that accepts both formats:
+
+```typescript
+import {
+  BlobRef,
+  isBlobRef,
+  isTypedBlobRef,
+  isLegacyBlobRef,
+} from '@atproto/lex'
+
+// When reading data, always use BlobRef to handle both formats
+function processBlobRef(blob: BlobRef) {
+  if (isTypedBlobRef(blob)) {
+    console.log('Modern blob:', blob.ref, blob.mimeType, blob.size)
+  } else if (isLegacyBlobRef(blob)) {
+    console.log('Legacy blob:', blob.cid, blob.mimeType)
+  }
+}
+
+// Or use the isBlobRef type guard which accepts both
+if (isBlobRef(value)) {
+  // value is BlobRef (either TypedBlobRef or LegacyBlobRef)
+}
+```
+
+Helper functions work with both formats:
+
+```typescript
+import {
+  getBlobCid,
+  getBlobCidString,
+  getBlobMime,
+  getBlobSize,
+} from '@atproto/lex'
+
+// These utilities work with both TypedBlobRef and LegacyBlobRef
+const cid = getBlobCid(blobRef) // Returns Cid object
+const cidStr = getBlobCidString(blobRef) // Returns string (optimized)
+const mime = getBlobMime(blobRef) // Returns mimeType
+const size = getBlobSize(blobRef) // Returns number | undefined (legacy refs lack size)
+```
+
+> [!IMPORTANT]
 >
-> Historically, references to blobs were represented as simple objects with the following structure:
+> **Validation behavior with legacy blobs:**
 >
-> ```typescript
-> type LegacyBlobRef = {
->   cid: string
->   mimeType: string
-> }
-> ```
+> - In **strict mode** (`strict: true`, the default): Legacy blob references are rejected during validation. Use this mode when you control the data source and expect only modern blobs.
+> - In **non-strict mode** (`strict: false`): Legacy blob references are accepted. This mode is used automatically when `strictResponseProcessing: false` is set on the Client, allowing your application to handle older records from the network gracefully.
 >
-> These should no longer be used for new records, but existing records using this format might still be encountered. To handle legacy blob references when validating data, enable the `--allowLegacyBlobs` flag when generating TypeScript schemas with `lex build`. You can use `isLegacyBlobRef()` from `@atproto/lex` to discriminate legacy blob references.
+> ```typescript
+> // Strict mode (default) - rejects legacy blobs
+> schema.$safeParse(data) // { strict: true }
 >
-> When using non-strict validation (e.g. `$safeParse(data, { strict: false })`), legacy blob references are automatically coerced into standard `BlobRef` objects with `size: -1`, even without `--allowLegacyBlobs`.
+> // Non-strict mode - accepts legacy blobs
+> schema.$safeParse(data, { strict: false })
+> ```
 
 ### Actions
 

package/bin/lex

@@ -76,12 +76,6 @@ yargs(hideBin(process.argv))
           describe:
             'package name of the library to import the lex schema utility "l" from',
         },
-        allowLegacyBlobs: {
-          type: 'boolean',
-          default: false,
-          describe:
-            'generate schemas that accept legacy blob references (disabled by default; enable this if you encounter issues while processing records created a long time ago)',
-        },
         importExt: {
           type: 'string',
           default: '.js',

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@atproto/lex",
-  "version": "0.0.22",
+  "version": "0.0.24",
   "license": "MIT",
   "description": "Lexicon tooling for AT",
   "keywords": [
@@ -38,19 +38,20 @@
   "dependencies": {
     "tslib": "^2.8.1",
     "yargs": "^17.0.0",
-    "@atproto/lex-builder": "^0.0.19",
-    "@atproto/lex-client": "^0.0.17",
-    "@atproto/lex-data": "^0.0.14",
-    "@atproto/lex-json": "^0.0.14",
-    "@atproto/lex-installer": "^0.0.22",
-    "@atproto/lex-schema": "^0.0.16"
+    "@atproto/lex-builder": "^0.0.21",
+    "@atproto/lex-client": "^0.0.19",
+    "@atproto/lex-data": "^0.0.15",
+    "@atproto/lex-json": "^0.0.15",
+    "@atproto/lex-installer": "^0.0.24",
+    "@atproto/lex-schema": "^0.0.18"
   },
   "devDependencies": {
     "@types/yargs": "^17.0.33",
     "vitest": "^4.0.16"
   },
   "scripts": {
-    "prebuild": "./bin/lex build --clear --lexicons ./lexicons --out ./tests/lexicons --lib @atproto/lex-schema -- ignore additional npm args",
+    "codegen": "./bin/lex build --override --indexFile --lexicons ./lexicons --out ./tests/lexicons --lib @atproto/lex-schema",
+    "prebuild": "pnpm run codegen",
     "build": "tsc --build tsconfig.build.json",
     "test": "vitest run"
   }