cry-search 1.0.0

package/CLAUDE.md

@@ -0,0 +1,285 @@
+# cry-search library
+
+cry-search is a library for fast searching in large datasets.
+
+## Tech stack
+1. Bun bundler for development, testing, and bundling
+2. ESM jscript in dist for distribution
+
+## Architecture
+
+The library has two implementations:
+1. **Numeric (default)** - Memory-optimized using Uint32Array for token storage
+   - Location: `src/common/numeric/`
+   - ~45% less memory than string implementation
+   - Global token registry with array-based lookups
+   - Tokens stored as numeric IDs in Uint32Array
+2. **String (legacy)** - Original string-based implementation
+   - Location: `src/common/string/`
+   - Uses string arrays for token storage
+   - Available with `*String` suffix (e.g., `createSearchArrayMetadataString`)
+
+### Token Registry (Numeric Implementation)
+
+The numeric implementation uses a global token registry (`src/utils/tokenRegistry.ts`):
+- `tokens: string[]` - Global array of all tokens (preallocated 10k)
+- Token ID = index in array
+- During metadata build: temporary `Map<string, number>` for fast lookups, cleared after use
+- During search: `findTokenId()` linear search (no Map in memory)
+- Match modes encoded in high 3 bits of Uint32 token ID
+
+## Requirements
+1. Metadata
+   1. searching uses metadata for object arrays to be searched
+   2. metadata is prepared in advance
+   3. the purpose of having metadata is to make searches as fast as possible
+2. Search requirements
+   1. input preparation
+      1. date normalization (first step)
+         1. ISO 8601: "2025-12-27T11:35:25.814Z" or "2025-12-27" → "20251227"
+         2. ISO year-month: "2025-12" → "202512"
+         3. d.m.yyyy: "1.3.2025" or "01.03.2025" → "20250301"
+         4. d.m.yy: "27.12.25" → "20251227" (2-digit year assumes 2000s)
+         5. m.yyyy: "3.2025" or "03.2025" → "202503"
+         6. month name + year: "mar 2025", "march 2025", "marec 2025" → "202503"
+         7. month names supported in English and Slovenian (full and abbreviated)
+      2. preprocessing
+         1. before splitting the string, we find possible numbers and replace decimal commas with decimal periods and drop leading zeros
+            1. eg "abc 12,3x" -> "abc 12.3x"
+            2. eg "abc ,3x" -> "abc .3x"
+            3. eg "abc 0,3x" -> "abc .3x"
+            4. eg "abc 0.3x" -> "abc .3x"
+            5. eg "007" -> "7"
+            6. multiple periods stay as-is: "1.2.3" -> "1.2.3"
+         2. we remove spaces in pattern digits - digits
+            1. eg. "12 - 25" => "12-25"
+         3. we insert spaces between starting letters and digits (only at letter-to-digit boundaries)
+            1. eg "SI123" -> "SI 123"
+            2. eg "123SI" -> "123 SI"
+            3. eg "A1B2" -> "A1B2"
+      3. sanitation
+         1. case-insensitive
+         2. diacritics-insensitive (č,ć,c -> same c)
+         3. we only keep letters (a-z), digits (0-9), periods (.), minuses (-), spaces (" ") in search
+         4. other characters are replaced with space (" ")
+   2. tokenization
+      1. we split strings on spaces
+      2. then we trim all substrings
+   3. matching
+      1. we match words in any order
+      2. we match numbers from start or from end
+         1. eg. 123 matches 12345 and 45123 but not 7812345
+   4. nested data extraction
+      1. nested objects and arrays are automatically extracted recursively
+      2. Date objects are converted to ISO strings before tokenization
+      3. example: { customer: { firstName: 'John' }, items: [{ opis: 'A' }] } extracts "john", "a"
+   5. dot notation in searchInFields and fieldMatchModes
+      1. use dot notation to target specific nested fields: 'customer.firstName', 'items.opis'
+      2. arrays of objects: 'items.opis' extracts from all array elements
+      3. deeply nested: 'data.level1.level2.value'
+      4. fieldMatchModes supports dot notation: { 'items.barcode': 'startEnd' }
+   6. building metadata
+      1. function prepareStringForSearch -> tokens[]
+      2. function createSearchArrayMetadata (numeric, default)
+         1. all elements in array must have unique _id:string field
+         2. prepares input as metadata for each array item
+         3. stores tokens as Uint32Array of token IDs
+         4. creates temporary tokensMap, clears after use
+         5. supports per-field match modes (encoded in token ID high bits)
+   7. searching
+      1. searchInData / searchInDataReturnIds
+         1. prepares query using prepareStringForSearch
+         2. matches query tokens against numeric metadata
+         3. returns matching IDs
+      2. searchInDataReturnObjects
+         1. calls searchInDataReturnIds and looks up objects
+   8. linked arrays searching
+      1. purpose: search across two related arrays (e.g., stranke + pacienti) where secondary items belong to primary items via foreign key
+      2. building linked metadata
+         1. function createSearchLinkedMetadata
+            1. takes primary items + their metadata
+            2. takes secondary items + their metadata
+            3. takes foreignKeyGetter function to extract primary ID from secondary item
+            4. builds reverse index: primary._id -> secondary._id[]
+            5. no data duplication - stores references and indexes only
+      3. searching linked arrays
+         1. function findInLinkedArrays
+            1. for each primary item, collects all tokens from primary + all its secondaries
+            2. matches query tokens against combined token pool
+            3. determines matchedIn: 'primary' | 'secondary' | 'both'
+            4. returns only secondaries that contributed to the match (not all secondaries)
+            5. tokens that match in primary are excluded from secondary matching to avoid foreign key redundancy
+         2. function findInLinkedArraysSimple
+            1. calls findInLinkedArrays and returns just the results array
+      4. example use case
+         1. search "krajnik angie" matches stranka "Krajnik" with pacient "Angie"
+         2. returns: { primary: Stranka, secondaries: [matching Pacient], matchedIn: 'both' }
+
+## Types
+
+1. Id: string
+2. NumericToken: number (index into global tokens array)
+3. NumericTokenSortedList: Uint32Array (sorted numeric token IDs)
+4. NumericSearchMetadata: Map<Id, NumericTokenSortedList> (default SearchMetadata)
+5. StringSearchMetadata: Map<Id, string[]> (legacy, string-based)
+6. LinkedSearchMetadata: { primaryMeta, linkedMeta, primaryToLinked: Map<Id, Id[]> }
+7. SearchableObject: { _id: Id, _deleted?: Date, _blocked?: Date }
+8. SearchableData<T>: Map<Id, T> where T extends SearchableObject
+9. SearchableObjectSpec<T extends SearchableObject, C = any>:
+   1. searchInFields?: (keyof T | string)[] - supports dot notation like 'customer.firstName'
+   2. dontSearchInFields?: (keyof T)[]
+   3. searchContext?: C
+   4. extractSearchableStringFn?: (obj: SearchableObject) => string | undefined
+   5. enrichSearchObjectFn?: (obj: SearchableObject, searchContext: C) => void
+   6. fieldMatchModes?: Partial<Record<keyof T | string, MatchMode>> - per-field match modes
+10. MatchMode: 'start' | 'end' | 'startEnd' | 'anywhere' | 'whole'
+    1. Field token prefixes: '<' (start), '>' (end), '+' (startEnd), '*' (anywhere), '!' (whole)
+    2. Numeric: encoded in high 3 bits of Uint32 token ID
+11. SearchableLinkedObjectsSpec: primary SearchableObjectSpec, linked SearchableObjectSpec, foreignKeyGetter
+
+## Search Query Prefixes
+
+Override match behavior per token at search time. Priority: query prefix > field mode > global SearchOpts.
+
+### User-friendly prefixes
+| Prefix | Mode | Description | Example |
+|--------|------|-------------|---------|
+| `word..` | start | Match tokens starting with word | `APL..` matches "APL-IP15P" |
+| `..word` | end | Match tokens ending with word | `..Pro` matches "MacBookPro" |
+| `=word` | whole | Exact match only | `=jan` matches "jan" not "jana" |
+| `?word` | anywhere | Match word anywhere in token | `?book` matches "MacBook" |
+| `--word` | negation | Exclude results containing word | `apple --iphone` excludes iPhones |
+| `~word` | negation | Same as `--word` | `apple ~iphone` |
+
+### Programmatic prefixes
+| Prefix | Mode |
+|--------|------|
+| `<word` | start |
+| `>word` | end |
+| `+word` | startEnd |
+| `*word` | anywhere |
+| `!word` | whole |
+
+### Examples
+1. `"jana useni"` → finds "jana usenik" (default startEnd matching)
+2. `"jana =useni"` → no match (whole match required, "useni" ≠ "usenik")
+3. `"jana --macka"` → finds "jana" but excludes any result containing "macka"
+4. `"APL.."` → finds all tokens starting with "APL" (APL-IP15P, APL-MBP16)
+5. `"..nik"` → finds tokens ending with "nik" (usenik, krajnik)
+
+
+## Main Functions (Default = Numeric)
+
+1. prepareStringForSearch(input: string): string[]
+   1. pipeline: normalizeDates → preprocessString → sanitiseString → tokenize → sort
+2. createSearchArrayMetadata(data: SearchableData, spec?): NumericSearchMetadata
+   1. creates temporary tokensMap for fast string→number lookups
+   2. stores tokens as Uint32Array per item
+   3. clears tokensMap after use (only global tokens[] remains)
+   4. for _deleted or _blocked nothing is added
+3. prepareObjectForSearch(obj, spec?, tokensMap): NumericTokenSortedList | undefined
+   1. returns undefined if _deleted or _blocked
+   2. if custom extractSearchableStringFn provided, uses simple processing
+   3. otherwise processes fields individually to support per-field match modes
+   4. recursively extracts strings from nested objects/arrays
+4. searchInData / searchInDataReturnIds(query: string, metadata): Id[]
+5. searchInDataReturnObjects<T>(query, metadata, data): Map<Id, T>
+6. searchInDataWithLimit(query, metadata, limit): Id[]
+7. updateSearchMetadata(metadata, objectId, obj, spec)
+   1. updates or deletes metadata for this object
+   2. taking into account _blocked and _deleted
+8. updateSearchMetadataBatch(metadata, batch, spec)
+   1. creates temporary tokensMap, clears after use
+   2. updates or deletes metadata for batch of objects
+
+## Token Registry Functions
+
+1. tokens: string[] - global array (read-only)
+2. getTokenCount(): number
+3. getTokenById(id: number): string | undefined
+4. findTokenId(token: string): number (-1 if not found)
+5. registerToken(token: string): number
+6. getOrCreateTokenId(token: string, tokensMap): number
+7. createTokensMap(): Map<string, number>
+8. clearTokensMap(tokensMap): void
+9. resetTokenRegistry(): void (for testing only)
+
+## String Implementation (Legacy)
+
+Available with `*String` suffix:
+- createSearchArrayMetadataString
+- createSearchLinkedMetadataString
+- updateSearchMetadataString
+- updateSearchMetadataBatchString
+- findInArray (uses string metadata)
+- findInArrayReturnDataAndMeta
+- findInLinkedArrays
+
+## File Structure
+
+```
+src/
+├── common/
+│   ├── numeric/           # Default numeric implementation
+│   │   ├── index.ts
+│   │   ├── createSearchArrayMetadata.ts
+│   │   ├── createSearchLinkedMetadata.ts
+│   │   ├── updateSearchMetadata.ts
+│   │   └── searchInData.ts
+│   ├── string/            # Legacy string implementation
+│   │   ├── index.ts
+│   │   ├── createSearchArrayMetadata.ts
+│   │   ├── createSearchLinkedMetadata.ts
+│   │   └── updateSearchMetadata.ts
+│   ├── findInArray.ts           # String-based search
+│   ├── findInLinkedArrays.ts    # String-based linked search
+│   ├── syncSearchArrayMetadata.ts
+│   ├── updateSearchLinkedMetadata.ts
+│   └── SearchUniverse.ts        # Multi-collection manager (uses numeric)
+├── utils/
+│   ├── tokenRegistry.ts         # Global tokens array
+│   ├── prepareStringForSearch.ts
+│   ├── prepareStringForSearchNumeric.ts
+│   ├── matchToken.ts            # String-based matching
+│   ├── matchTokenNumeric.ts     # Numeric matching
+│   └── ...
+├── types/
+│   └── index.ts
+└── index.ts                     # Main exports (numeric as default)
+```
+
+## Coding rules
+
+1. each function is in separate file
+   1. preprocessString
+   2. sanitiseString
+   3. tokenize
+   4. matchToken / matchTokenNumeric
+   5. searchInData
+
+## Testing
+
+1. Large real-life datasets are in ./test-data (arikli, stranke, pacienti).
+2. tests are in ./test
+3. tests per implementation are in ./test/(IMPLEMENTATION)
+4. We have interactive.ts that
+   1. loads all three datasets and creates SearchMetadata for each
+   2. measures memory consumption for each loaded dataset and for created SearchMetadata
+   3. lets user enter search string in a loop and presents
+      1. 3 results from each dataset (where found)
+         1. no fieldnames, just values
+      2. prints time spent searching in each dataset
+   4. Supports command lines for changing data
+      1. 621771cadf380d819d0b0970: Angie -> Ančka
+         1. find object with _id===621771cadf380d819d0b0970
+         2. find key with value "Angie"
+         3. change value to Ančka
+      2. 621771cadf380d819d0b0970: x siamska -> siamska
+      3. 621771cadf380d819d0b0970: _blocked -> true/false
+         1. sets _blocked to new Date() or deletes the field
+      4. 621771cadf380d819d0b0970: _deleted -> true/false
+         1. like blocked
+5. benchmark-numeric-vs-string.ts compares implementations
+   1. Memory: numeric ~45% less
+   2. Build time: ~same
+   3. Search time: varies by match mode

package/LICENSE.md

@@ -0,0 +1,67 @@
+# Proprietary License
+
+Copyright (c) 2025. All Rights Reserved.
+
+## Terms and Conditions
+
+### 1. Definitions
+
+- "Software" refers to the cry-search library, including all source code, documentation, and associated files.
+- "Licensor" refers to the copyright holder(s) of this Software.
+- "You" refers to any individual or entity that accesses, uses, or attempts to use this Software.
+
+### 2. Grant of License
+
+Subject to the terms of this License, the Licensor grants You a limited, non-exclusive, non-transferable, revocable license to use this Software **solely for personal, non-commercial, evaluation purposes**.
+
+### 3. Restrictions
+
+You are expressly **PROHIBITED** from:
+
+1. **Commercial Use**: Using the Software, in whole or in part, for any commercial purpose, including but not limited to:
+   - Incorporating the Software into commercial products or services
+   - Using the Software to provide services to third parties for compensation
+   - Using the Software in any revenue-generating activity
+   - Using the Software in a business or organizational context
+
+2. **Modification**: Modifying, adapting, translating, reverse engineering, decompiling, disassembling, or creating derivative works based on the Software.
+
+3. **Distribution**: Distributing, sublicensing, leasing, renting, lending, selling, reselling, or otherwise transferring the Software or any rights therein to any third party.
+
+4. **Sharing**: Forwarding, sharing, publishing, uploading, or making the Software available to any third party through any means, including but not limited to:
+   - Public or private repositories
+   - File sharing services
+   - Email or messaging platforms
+   - Any form of electronic or physical distribution
+
+5. **Copying**: Copying or reproducing the Software except as strictly necessary for personal evaluation use on a single machine.
+
+6. **Removal of Notices**: Removing, altering, or obscuring any copyright, trademark, or other proprietary notices contained in the Software.
+
+### 4. Intellectual Property
+
+The Software and all copies thereof are proprietary to the Licensor and title thereto remains exclusively with the Licensor. All rights in the Software not specifically granted in this License are reserved to the Licensor.
+
+### 5. No Warranty
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. IN NO EVENT SHALL THE LICENSOR BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT, OR OTHERWISE, ARISING FROM, OUT OF, OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+### 6. Termination
+
+This License is effective until terminated. Your rights under this License will terminate automatically without notice if You fail to comply with any of its terms. Upon termination, You must immediately cease all use of the Software and destroy all copies in Your possession or control.
+
+### 7. Governing Law
+
+This License shall be governed by and construed in accordance with applicable laws, without regard to conflicts of law principles.
+
+### 8. Entire Agreement
+
+This License constitutes the entire agreement between You and the Licensor concerning the Software and supersedes all prior or contemporaneous oral or written communications, proposals, and representations with respect to the Software.
+
+### 9. Contact
+
+For licensing inquiries, including requests for commercial licenses, please contact the copyright holder.
+
+---
+
+**BY USING THIS SOFTWARE, YOU ACKNOWLEDGE THAT YOU HAVE READ THIS LICENSE, UNDERSTAND IT, AND AGREE TO BE BOUND BY ITS TERMS AND CONDITIONS.**

package/README.md

@@ -0,0 +1,283 @@
+# cry-search
+
+A fast, memory-efficient search library for large datasets with support for tokenized matching, linked collections, and per-field match modes.
+
+## Features
+
+- **Fast search** - Pre-built metadata enables instant queries on large datasets
+- **Memory efficient** - Numeric token storage uses ~45% less memory than string-based approaches
+- **Updatable** - Add, update, or remove items without rebuilding metadata
+- **Flexible matching** - Per-field match modes and query prefixes for prefix, suffix, anywhere, exact, and negation
+- **Search prefixed** - use `--word` to exclude it, `...word` for suffix, and more
+- **Automatic normalization** - Handles diacritics, case, dates, and number formatting
+- **[Universal search](./UNIVERSE.md)** - Search and update across linked collections (e.g., customers with their pets)
+
+## Table of Contents
+
+- [cry-search](#cry-search)
+  - [Features](#features)
+  - [Table of Contents](#table-of-contents)
+  - [Installation](#installation)
+  - [Examples](#examples)
+    - [Single Table Search](#single-table-search)
+    - [SearchUniverse with Linked Collections](#searchuniverse-with-linked-collections)
+    - [Updating Existing Data](#updating-existing-data)
+  - [Architecture](#architecture)
+    - [Text Processing Pipeline](#text-processing-pipeline)
+    - [Metadata Building](#metadata-building)
+    - [Searching](#searching)
+    - [Memory Optimization](#memory-optimization)
+  - [Query Prefixes](#query-prefixes)
+  - [Legacy String Implementation](#legacy-string-implementation)
+  - [Specification](#specification)
+  - [License](#license)
+
+## Installation
+
+```bash
+npm install cry-search
+```
+
+## Examples
+
+### Single Table Search
+
+```typescript
+import {
+  createSearchArrayMetadata,
+  searchInDataReturnObjects,
+  arrayToSearchableData,
+  type SearchableObject,
+} from 'cry-search';
+
+interface Product extends SearchableObject {
+  _id: string;
+  name: string;
+  description: string;
+  sku: string;
+}
+
+const products: Product[] = [
+  { _id: '1', name: 'iPhone 15 Pro', description: 'Latest Apple smartphone', sku: 'APL-IP15P' },
+  { _id: '2', name: 'Samsung Galaxy S24', description: 'Android flagship phone', sku: 'SAM-GS24' },
+  { _id: '3', name: 'MacBook Pro 16"', description: 'Apple laptop for professionals', sku: 'APL-MBP16' },
+];
+
+// Convert array to searchable data (Map<id, item>)
+const data = arrayToSearchableData(products);
+
+// Build search metadata (do this once, reuse for all searches)
+const metadata = createSearchArrayMetadata(data, {
+  searchInFields: ['name', 'description', 'sku'],
+});
+
+// Search
+const results = searchInDataReturnObjects('apple', metadata, data);
+// Returns: Map with products 1 and 3
+
+const results2 = searchInDataReturnObjects('samsung galaxy', metadata, data);
+// Returns: Map with product 2
+
+// Search is diacritics and case insensitive
+const results3 = searchInDataReturnObjects('IPHONE', metadata, data);
+// Returns: Map with product 1
+```
+
+### SearchUniverse with Linked Collections
+
+For related data (like customers and their pets), use `SearchUniverse` to search across collections:
+
+```typescript
+import { SearchUniverse, type SearchableObject } from 'cry-search';
+
+interface Stranka extends SearchableObject {
+  _id: string;
+  name: string;
+  address: string;
+}
+
+interface Pacient extends SearchableObject {
+  _id: string;
+  stranka_id: string;  // Foreign key to Stranka
+  name: string;
+  species: string;
+}
+
+// Create universe and register collections
+const universe = new SearchUniverse();
+
+universe.registerCollection<Stranka>('stranke', {
+  spec: { searchInFields: ['name', 'address'] },
+});
+
+universe.registerCollection<Pacient>('pacienti', {
+  spec: { dontSearchInFields: ['stranka_id'] },
+  linkedTo: {
+    collectionName: 'stranke',
+    foreignKeyGetter: (p) => p.stranka_id,
+  },
+});
+
+// Load data
+universe.loadCollection('stranke', [
+  { _id: 's1', name: 'Krajnik', address: 'Ljubljana' },
+  { _id: 's2', name: 'Novak', address: 'Maribor' },
+]);
+
+universe.loadCollection('pacienti', [
+  { _id: 'p1', stranka_id: 's1', name: 'Angie', species: 'cat' },
+  { _id: 'p2', stranka_id: 's1', name: 'Rex', species: 'dog' },
+  { _id: 'p3', stranka_id: 's2', name: 'Bella', species: 'cat' },
+]);
+
+// Search across all collections
+const results = universe.searchUniversally('krajnik angie');
+// Returns:
+// {
+//   stranke: [{ _id: 's1', name: 'Krajnik', ... }],
+//   pacienti: [{
+//     primary: { _id: 's1', name: 'Krajnik', ... },
+//     linked: [{ _id: 'p1', name: 'Angie', ... }],
+//     matchedIn: 'both'
+//   }]
+// }
+
+// Search only in linked items
+const results2 = universe.searchUniversally('rex');
+// Returns pacienti results with matchedIn: 'linked'
+
+// Search only in primary - returns all linked items
+const results3 = universe.searchUniversally('novak');
+// Returns Novak stranka with Bella in linked array
+```
+
+### Updating Existing Data
+
+When data changes, update the metadata to keep search in sync:
+
+```typescript
+import {
+  createSearchArrayMetadata,
+  updateSearchMetadata,
+  updateSearchMetadataBatch,
+  arrayToSearchableData,
+} from 'cry-search';
+
+// Initial setup
+const data = arrayToSearchableData(products);
+const metadata = createSearchArrayMetadata(data);
+
+// Update a single item
+const updatedProduct = { ...products[0], name: 'iPhone 16 Pro' };
+data.set(updatedProduct._id, updatedProduct);
+updateSearchMetadata(metadata, updatedProduct._id, updatedProduct);
+
+// Update multiple items at once
+const updates = [
+  { _id: '1', name: 'iPhone 16 Pro Max', description: 'Newest Apple phone', sku: 'APL-IP16PM' },
+  { _id: '4', name: 'iPad Pro', description: 'Apple tablet', sku: 'APL-IPAD' },
+];
+for (const item of updates) {
+  data.set(item._id, item);
+}
+updateSearchMetadataBatch(metadata, updates);
+
+// Mark item as deleted (removes from search but keeps in data)
+const deletedProduct = { ...products[1], _deleted: new Date() };
+data.set(deletedProduct._id, deletedProduct);
+updateSearchMetadata(metadata, deletedProduct._id, deletedProduct);
+
+// With SearchUniverse - handles linked indexes automatically
+universe.updateSearchMetadata('pacienti', 'p1', {
+  _id: 'p1',
+  stranka_id: 's2',  // Changed owner from s1 to s2
+  name: 'Angie',
+  species: 'cat',
+});
+```
+
+## Architecture
+
+cry-search uses a two-phase approach: **build** metadata once, then **search** instantly.
+
+### Text Processing Pipeline
+
+Both metadata building and search queries go through the same pipeline:
+
+```
+Input: "Čokolada 27.12.2025 SI123"
+   ↓
+1. Normalize dates     → "Čokolada 20251227 SI123"
+2. Preprocess          → "Čokolada 20251227 SI 123"  (split letter-digit)
+3. Sanitize            → "cokolada 20251227 si 123"  (lowercase, remove diacritics)
+4. Tokenize            → ["cokolada", "20251227", "si", "123"]
+5. Sort                → ["123", "20251227", "cokolada", "si"]
+6. Numeric IDs         → Uint32Array [42, 891, 156, 7]  (indices into global registry)
+```
+
+### Metadata Building
+
+Tokens are stored as numeric IDs in a global registry. Each item's tokens are stored in a sorted `Uint32Array` for memory efficiency and fast binary search.
+
+### Searching
+
+Query tokens are matched against metadata using binary search. All query tokens must match for an item to be returned. Match modes (start, end, startEnd, anywhere, whole) control how tokens are compared.
+
+### Memory Optimization
+
+The numeric implementation stores tokens as `Uint32Array` indices instead of string arrays, reducing memory usage by ~45% compared to the string-based implementation.
+
+## Query Prefixes
+
+Override match behavior per token at search time using prefixes:
+
+| Prefix | Mode | Description |
+|--------|------|-------------|
+| `word..` | start | Match tokens starting with "word" |
+| `..word` | end | Match tokens ending with "word" |
+| `=word` | whole | Exact match only |
+| `?word` | anywhere | Match "word" anywhere in token |
+| `--word` or `~word` | negation | Exclude results containing "word" |
+
+Alternative prefixes (for programmatic use): `<word` (start), `>word` (end), `+word` (startEnd), `*word` (anywhere), `!word` (whole)
+
+```typescript
+// Find "jana" but only exact match on "usenik"
+searchInDataReturnObjects('jana =usenik', metadata, data);
+// Matches "Jana Usenik" but not "Jana Useniker"
+
+// Find "apple" but exclude results containing "iphone"
+searchInDataReturnObjects('apple --iphone', metadata, data);
+// Matches MacBook Pro but not iPhone
+
+// Find products with SKU starting with "APL"
+searchInDataReturnObjects('APL..', metadata, data);
+// Matches APL-IP15P, APL-MBP16, etc.
+```
+
+Priority: query prefix > field match mode > global SearchOpts default.
+
+## Legacy String Implementation
+
+A string-based implementation is available for backwards compatibility. Import with the `String` suffix:
+
+```typescript
+import {
+  createSearchArrayMetadataString,
+  updateSearchMetadataString,
+  findInArray,  // String-based search function
+} from 'cry-search';
+```
+
+## Specification
+
+See [CLAUDE.md](./CLAUDE.md) for the complete technical specification including:
+- Input processing pipeline (date normalization, preprocessing, sanitization)
+- Tokenization rules
+- Matching behavior
+- Linked search semantics
+- Type definitions
+
+## License
+
+See [LICENSE.md](./LICENSE.md) for license terms.