shamela 1.3.1 → 1.3.3

package/README.md

@@ -15,28 +15,50 @@
 
 A universal TypeScript library for accessing and downloading Maktabah Shamela v4 APIs. The package runs in both Node.js and modern browsers, providing ergonomic helpers to interact with the Shamela API, download master and book databases, and retrieve book data programmatically.
 
+## Features
+
+- 🚀 **Full data lifecycle** – fetch metadata, download master and book databases, and query the results entirely in-memory.
+- 🔐 **Runtime configuration** – configure API credentials, WASM paths, and custom fetch/logging implementations at runtime.
+- 🧠 **Content tooling** – parse, sanitise, and post-process Arabic book content with utilities tailored for Shamela formatting.
+- 🌐 **Environment aware** – automatically selects optimal sql.js WASM bundles for Node.js, browsers, and bundled runtimes.
+- 🧪 **Well-tested** – comprehensive unit and end-to-end coverage to ensure reliable integrations.
+
 ## Table of Contents
 
+- [Features](#features)
 - [Installation](#installation)
-- [Environment Variables](#environment-variables)
-- [Usage](#usage)
-    - [Getting Started](#getting-started)
-    - [Next.js / Bundled Environments](#nextjs--bundled-environments)
-    - [API Functions](#api-functions)
-        - [getMasterMetadata](#getmastermetadata)
-        - [downloadMasterDatabase](#downloadmasterdatabase)
-        - [getBookMetadata](#getbookmetadata)
-        - [downloadBook](#downloadbook)
-        - [getBook](#getbook)
-        - [getMaster](#getmaster)
-        - [getCoverUrl](#getcoverurl)
+- [Quick Start](#quick-start)
+  - [Standard Node.js](#standard-nodejs)
+  - [Next.js / Bundled Environments](#nextjs--bundled-environments)
+  - [Browser](#browser)
+- [API Reference](#api-reference)
+  - [Configuration](#configuration)
+    - [configure](#configure)
+    - [resetConfig](#resetconfig)
+    - [getConfig](#getconfig)
+    - [getConfigValue](#getconfigvalue)
+    - [requireConfigValue](#requireconfigvalue)
+  - [Metadata & Downloads](#metadata--downloads)
+    - [getMasterMetadata](#getmastermetadata)
+    - [downloadMasterDatabase](#downloadmasterdatabase)
+    - [getBookMetadata](#getbookmetadata)
+    - [downloadBook](#downloadbook)
+    - [getCoverUrl](#getcoverurl)
+  - [Data Access](#data-access)
+    - [getBook](#getbook)
+    - [getMaster](#getmaster)
+  - [Content Utilities](#content-utilities)
+    - [parseContentRobust](#parsecontentrobust)
+    - [sanitizePageContent](#sanitizepagecontent)
+    - [splitPageBodyFromFooter](#splitpagebodyfromfooter)
+    - [removeArabicNumericPageMarkers](#removearabicnumericpagemarkers)
+    - [removeTagsExceptSpan](#removetagsexceptspan)
+  - [Supporting Utilities](#supporting-utilities)
+    - [buildUrl](#buildurl)
+    - [httpsGet](#httpsget)
 - [Examples](#examples)
-    - [Downloading the Master Database](#downloading-the-master-database)
-    - [Downloading a Book](#downloading-a-book)
-    - [Retrieving Book Data](#retrieving-book-data)
-    - [Retrieving Master Data in memory](#retrieving-master-data-in-memory)
 - [Data Structures](#data-structures)
-- [Next.js demo](#nextjs-demo)
+- [Next.js Demo](#nextjs-demo)
 - [Troubleshooting](#troubleshooting)
 - [Testing](#testing)
 - [License](#license)
@@ -44,103 +66,53 @@ A universal TypeScript library for accessing and downloading Maktabah Shamela v4
 ## Installation
 
 ```bash
-bun add shamela
+npm install shamela
 ```
 
-or
-
 ```bash
-npm install shamela
+bun add shamela
 ```
 
-or
-
 ```bash
 yarn add shamela
 ```
 
-or
-
 ```bash
 pnpm install shamela
 ```
 
-## Environment Variables
-
-Before using the library, you need to set up some environment variables for API keys and endpoints:
+## Quick Start
 
-- `SHAMELA_API_KEY`: Your API key for accessing the Shamela API.
-- `SHAMELA_API_MASTER_PATCH_ENDPOINT`: The endpoint URL for the master database patches.
-- `SHAMELA_API_BOOKS_ENDPOINT`: The base endpoint URL for book-related API calls.
-- `SHAMELA_SQLJS_WASM_URL` (optional): Override the default CDN URL used to load the `sql.js` WebAssembly binary when running in the browser.
+### Standard Node.js
 
-You can set these variables in a `.env` file at the root of your project:
-
-```dotenv
-SHAMELA_API_KEY=your_api_key_here
-SHAMELA_API_MASTER_PATCH_ENDPOINT=https://shamela.ws/api/master_patch
-SHAMELA_API_BOOKS_ENDPOINT=https://shamela.ws/api/books
-# Optional when you host sql-wasm.wasm yourself
-# SHAMELA_SQLJS_WASM_URL=https://example.com/sql-wasm.wasm
-```
+For simple Node.js scripts (non-bundled environments), the library auto-detects the WASM file:
 
-### Runtime configuration (browsers and serverless)
-
-When you cannot rely on environment variables—such as when running inside a browser, an edge worker, or a serverless function—use the `configure` helper to provide credentials at runtime:
-
-```ts
-import { configure } from 'shamela';
+```typescript
+import { configure, getBook } from 'shamela';
 
+// Configure API credentials
 configure({
-    apiKey: process.env.NEXT_PUBLIC_SHAMELA_KEY,
-    booksEndpoint: 'https://shamela.ws/api/books',
-    masterPatchEndpoint: 'https://shamela.ws/api/master_patch',
-    // Optional: host sql-wasm.wasm yourself to control caching/CDN placement
-    sqlJsWasmUrl: '/assets/sql-wasm.wasm',
-    // Optional: integrate with your application's logging system
-    logger: console,
-    // Optional: provide a custom fetch implementation (for tests or SSR)
-    fetchImplementation: fetch,
+  apiKey: process.env.SHAMELA_API_KEY,
+  booksEndpoint: process.env.SHAMELA_BOOKS_ENDPOINT,
+  masterPatchEndpoint: process.env.SHAMELA_MASTER_ENDPOINT,
+  // sqlJsWasmUrl is auto-detected in standard Node.js
 });
-```
-
-You can call `configure` multiple times—values are merged, so later calls update only the keys you pass in.
-
-The optional `logger` must expose `debug`, `info`, `warn`, and `error` methods. When omitted, the library stays silent by default.
 
-## Usage
-
-### Getting Started
-
-First, import the library functions into your project:
-
-```javascript
-import {
-    getMasterMetadata,
-    downloadMasterDatabase,
-    getBookMetadata,
-    downloadBook,
-    getBook,
-    getMaster,
-    getCoverUrl,
-} from 'shamela';
+// Use the library
+const book = await getBook(26592);
+console.log(`Downloaded book with ${book.pages.length} pages`);
 ```
 
 ### Next.js / Bundled Environments
 
-When using this library in Next.js or other bundled environments (webpack/Turbopack), you need some additional configuration to ensure the sql.js WASM file is loaded correctly.
-
-#### 1. Update your Next.js configuration
+For Next.js, webpack, Turbopack, and other bundlers, you need to explicitly configure the WASM file path.
 
-Add the following to your `next.config.js` or `next.config.ts`:
+**1. Update `next.config.ts` or `next.config.js`:**
 
 ```typescript
 import type { NextConfig } from 'next';
 
 const nextConfig: NextConfig = {
-  experimental: {
-    serverComponentsExternalPackages: ['shamela', 'sql.js'],
-  },
   serverExternalPackages: ['shamela', 'sql.js'],
   // ... rest of your config
 };
@@ -148,30 +120,7 @@ const nextConfig: NextConfig = {
 export default nextConfig;
 ```
 
-This tells Next.js to exclude these packages from bundling and load them directly from `node_modules`.
-
-#### 2. Create a server-only configuration file
-
-Create a configuration file that will be imported only in server-side code:
-
-**Option A: Using the `createNodeConfig` helper (Recommended)**
-
-```typescript
-// lib/shamela-server.ts
-import { configure, createNodeConfig } from 'shamela';
-
-// Configure once when this module loads
-configure(createNodeConfig({
-  apiKey: process.env.SHAMELA_API_KEY,
-  booksEndpoint: process.env.SHAMELA_BOOKS_ENDPOINT,
-  masterPatchEndpoint: process.env.SHAMELA_MASTER_ENDPOINT,
-}));
-
-// Re-export the functions you need
-export { getBookMetadata, downloadBook, getMaster, getBook } from 'shamela';
-```
-
-**Option B: Manual configuration**
+**2. Create a server-only configuration file:**
 
 ```typescript
 // lib/shamela-server.ts
@@ -185,10 +134,10 @@ configure({
   masterPatchEndpoint: process.env.SHAMELA_MASTER_ENDPOINT,
 });
 
-export { getBookMetadata, downloadBook, getMaster, getBook } from 'shamela';
+export { downloadBook, getBook, getBookMetadata, getMaster, downloadMasterDatabase } from 'shamela';
 ```
 
-#### 3. Use in Server Actions or API Routes
+**3. Use in Server Actions:**
 
 ```typescript
 'use server';
@@ -197,36 +146,106 @@ import { getBookMetadata, downloadBook } from '@/lib/shamela-server';
 
 export async function downloadBookAction(bookId: number) {
   const metadata = await getBookMetadata(bookId);
-  const result = await downloadBook(bookId, {
+  return await downloadBook(bookId, {
     bookMetadata: metadata,
     outputFile: { path: `./books/${bookId}.db` }
   });
-  return result;
 }
 ```
 
-**Important:** Never import `shamela` directly in your `layout.tsx` or client components. Only use it in server-side code (Server Actions, API Routes, or Server Components).
+**Important:** Only import `shamela` in server-side code (Server Actions, API Routes, or Server Components). Never import in client components or `layout.tsx`.
 
-### API Functions
+### Browser
 
-#### getMasterMetadata
-
-Fetches metadata for the master database.
+In browsers, the library automatically uses a CDN-hosted WASM file:
 
 ```typescript
-getMasterMetadata(version?: number): Promise<GetMasterMetadataResponsePayload>
+import { configure, getBook } from 'shamela';
+
+configure({
+  apiKey: 'your-api-key',
+  booksEndpoint: 'https://SHAMELA_INSTANCE.ws/api/books',
+  masterPatchEndpoint: 'https://SHAMELA_INSTANCE.ws/api/master_patch',
+  // Automatically uses CDN: https://cdn.jsdelivr.net/npm/sql.js@1.13.0/dist/sql-wasm.wasm
+});
+
+const book = await getBook(26592);
 ```
 
-- `version` (optional): The version number of the master database you want to check for updates (defaults to 0)
+## API Reference
+
+### Configuration
 
-**Returns:** Promise that resolves to master database metadata including download URL and version
+#### configure
+
+Initialises runtime configuration including API credentials, custom fetch implementations, sql.js WASM location, and logger overrides.
+
+```typescript
+configure(options: ConfigureOptions): void
+```
 
 **Example:**
 
-```javascript
-const masterMetadata = await getMasterMetadata();
-console.log(masterMetadata.url); // Download URL for master database patch
-console.log(masterMetadata.version); // Latest version number
+```typescript
+import { configure } from 'shamela';
+
+configure({
+  apiKey: process.env.SHAMELA_API_KEY!,
+  booksEndpoint: process.env.SHAMELA_BOOKS_ENDPOINT!,
+  masterPatchEndpoint: process.env.SHAMELA_MASTER_ENDPOINT!,
+});
+```
+
+#### resetConfig
+
+Clears runtime overrides and restores the default silent logger.
+
+```typescript
+resetConfig(): void
+```
+
+Use this in tests or long-running processes when you need a clean configuration slate.
+
+#### getConfig
+
+Returns the merged configuration snapshot combining runtime overrides with environment variables.
+
+```typescript
+getConfig(): ShamelaConfig
+```
+
+#### getConfigValue
+
+Reads a single configuration value without throwing when it is missing.
+
+```typescript
+getConfigValue<Key extends ShamelaConfigKey>(key: Key): ShamelaConfig[Key] | undefined
+```
+
+#### requireConfigValue
+
+Retrieves a configuration entry and throws an error if the value is not present.
+
+```typescript
+requireConfigValue(key: Exclude<ShamelaConfigKey, 'fetchImplementation'>): string
+```
+
+### Metadata & Downloads
+
+#### getMasterMetadata
+
+Fetches metadata for the master database, including download URLs for the latest patches.
+
+```typescript
+getMasterMetadata(version?: number): Promise<GetMasterMetadataResponsePayload>
+```
+
+- `version` (optional): The version number to check for updates (defaults to 0)
+
+```typescript
+const metadata = await getMasterMetadata();
+console.log(metadata.url);     // Download URL
+console.log(metadata.version); // Version number
 
 // Check for updates from a specific version
 const updates = await getMasterMetadata(5);
@@ -234,449 +253,386 @@ const updates = await getMasterMetadata(5);
 
 #### downloadMasterDatabase
 
-Downloads the master database and saves it to a specified path. The master database contains comprehensive information about all books, authors, and categories available in the Shamela library.
+Downloads the master database containing all books, authors, and categories and writes it to disk or a custom writer.
 
 ```typescript
 downloadMasterDatabase(options: DownloadMasterOptions): Promise<string>
 ```
 
-- `options`: Configuration object containing:
-    - `masterMetadata` (optional): Pre-fetched metadata from `getMasterMetadata`
-    - `outputFile`: Object with `path` property specifying the output file path
-
-**Returns:** Promise that resolves to the path of the created output file
-
-**Example:**
+- `options.masterMetadata` (optional): Pre-fetched metadata to avoid an extra HTTP call
+- `options.outputFile.path`: Output file path (`.db`, `.sqlite`, or `.json`)
 
-```javascript
+```typescript
 // Download as SQLite database
 await downloadMasterDatabase({
-    outputFile: { path: './master.db' },
+  outputFile: { path: './master.db' }
 });
 
 // Download as JSON
 await downloadMasterDatabase({
-    outputFile: { path: './master.json' },
-});
-
-// Use pre-fetched metadata for efficiency
-const masterMetadata = await getMasterMetadata();
-await downloadMasterDatabase({
-    masterMetadata,
-    outputFile: { path: './master.db' },
+  outputFile: { path: './master.json' }
 });
 ```
 
 #### getBookMetadata
 
-Fetches metadata for a specific book.
+Fetches metadata for a specific book, including patch release information.
 
 ```typescript
 getBookMetadata(id: number, options?: GetBookMetadataOptions): Promise<GetBookMetadataResponsePayload>
 ```
 
-- `id`: The unique identifier of the book
-- `options` (optional): Configuration object containing:
-    - `majorVersion`: The major version to check against
-    - `minorVersion`: The minor version to check against
-
-**Returns:** Promise that resolves to book metadata including release URLs and versions
-
-**Example:**
+- `id`: Book identifier
+- `options.majorVersion` (optional): Major version to check
+- `options.minorVersion` (optional): Minor version to check
 
-```javascript
+```typescript
 const metadata = await getBookMetadata(26592);
-console.log(metadata.majorReleaseUrl); // URL for downloading the book
-console.log(metadata.majorRelease); // Major version number
-
-// Check specific versions
-const versionedMetadata = await getBookMetadata(26592, {
-    majorVersion: 1,
-    minorVersion: 2,
-});
+console.log(metadata.majorReleaseUrl);
+console.log(metadata.minorReleaseUrl);
 ```
 
 #### downloadBook
 
-Downloads and processes a book from the Shamela database. This function downloads the book's database files, applies patches if available, and exports the data to the specified format.
+Downloads and processes a book from Shamela, writing it to JSON or SQLite on disk.
 
 ```typescript
 downloadBook(id: number, options: DownloadBookOptions): Promise<string>
 ```
 
-- `id`: The unique identifier of the book to download
-- `options`: Configuration object containing:
-    - `bookMetadata` (optional): Pre-fetched metadata from `getBookMetadata`
-    - `outputFile`: Object with `path` property specifying the output file path
-
-**Returns:** Promise that resolves to the path of the created output file
+- `id`: Book identifier
+- `options.bookMetadata` (optional): Pre-fetched metadata to avoid re-fetching
+- `options.outputFile.path`: Output file path (`.db`, `.sqlite`, or `.json`)
 
-**Example:**
-
-```javascript
+```typescript
 // Download as JSON
 await downloadBook(26592, {
-    outputFile: { path: './book.json' },
+  outputFile: { path: './book.json' }
 });
 
-// Download as SQLite database
+// Download as SQLite
 await downloadBook(26592, {
-    outputFile: { path: './book.db' },
+  outputFile: { path: './book.db' }
 });
+```
 
-// Use pre-fetched metadata for efficiency
-const bookMetadata = await getBookMetadata(26592);
-await downloadBook(26592, {
-    bookMetadata,
-    outputFile: { path: './book.db' },
-});
+#### getCoverUrl
+
+Generates the URL for a book's cover image using the configured Shamela host.
+
+```typescript
+getCoverUrl(bookId: number): string
+```
+
+```typescript
+const coverUrl = getCoverUrl(26592);
+// Returns: "https://shamela.ws/covers/26592.jpg"
 ```
 
+### Data Access
+
 #### getBook
 
-Retrieves complete book data as a JavaScript object. This is a convenience function that handles temporary file creation and cleanup automatically.
+Retrieves complete book data as a JavaScript object, returning pages and title entries.
 
 ```typescript
 getBook(id: number): Promise<BookData>
 ```
 
-- `id`: The unique identifier of the book to retrieve
+```typescript
+const book = await getBook(26592);
+console.log(book.pages.length);
+console.log(book.titles?.length);
+console.log(book.pages[0].content);
+```
 
-**Returns:** Promise that resolves to complete book data including pages and titles
+#### getMaster
 
-**Example:**
+Retrieves the entire master dataset as a JavaScript object, including version information.
 
-```javascript
-const bookData = await getBook(26592);
-console.log(bookData.pages.length); // Number of pages in the book
-console.log(bookData.titles?.length); // Number of title entries
-console.log(bookData.pages[0].content); // Content of the first page
+```typescript
+getMaster(): Promise<MasterData>
 ```
 
-#### getMaster
+```typescript
+const master = await getMaster();
+console.log(master.version);
+console.log(master.books.length);
+console.log(master.authors.length);
+console.log(master.categories.length);
+```
+
+### Content Utilities
 
-Retrieves the entire master dataset (authors, books, categories) as a JavaScript object, including the version number that the
-API reports for the snapshot.
+#### parseContentRobust
+
+Parses Shamela HTML snippets into structured lines while preserving title hierarchy and Arabic punctuation.
 
 ```typescript
-getMaster(): Promise<MasterData>
+parseContentRobust(content: string): Line[]
 ```
 
-**Returns:** Promise that resolves to the complete master dataset with version metadata
+```typescript
+const lines = parseContentRobust(rawHtml);
+lines.forEach((line) => console.log(line.id, line.text));
+```
 
-**Example:**
+#### sanitizePageContent
 
-```javascript
-const masterData = await getMaster();
-console.log(masterData.version); // Version of the downloaded master database
-console.log(masterData.books.length); // Number of books available
-console.log(masterData.categories.length); // Number of categories available
+Normalises page content by applying regex-based replacement rules tuned for Shamela sources.
+
+```typescript
+sanitizePageContent(text: string, rules?: Record<string, string>): string
 ```
 
-#### getCoverUrl
+#### splitPageBodyFromFooter
 
-Generates the URL for a book's cover image.
+Separates page body content from trailing footnotes using the default Shamela marker.
 
 ```typescript
-getCoverUrl(bookId: number): string
+splitPageBodyFromFooter(content: string, marker?: string): readonly [string, string]
 ```
 
-- `bookId`: The unique identifier of the book
+#### removeArabicNumericPageMarkers
 
-**Returns:** The complete URL to the book's cover image
+Removes Arabic numeral markers enclosed in ⦗ ⦘, commonly used to denote page numbers.
 
-**Example:**
+```typescript
+removeArabicNumericPageMarkers(text: string): string
+```
 
-```javascript
-const coverUrl = getCoverUrl(26592);
-console.log(coverUrl); // "https://shamela.ws/covers/26592.jpg"
+#### removeTagsExceptSpan
+
+Strips anchor and hadeeth tags while preserving nested `<span>` elements.
+
+```typescript
+removeTagsExceptSpan(content: string): string
+```
+
+### Supporting Utilities
+
+#### buildUrl
+
+Constructs authenticated API URLs with query parameters and optional API key injection.
+
+```typescript
+buildUrl(endpoint: string, queryParams: Record<string, any>, useAuth?: boolean): URL
+```
+
+#### httpsGet
+
+Makes HTTPS GET requests using the configured fetch implementation, automatically parsing JSON responses and returning binary data otherwise.
+
+```typescript
+httpsGet<T extends Uint8Array | Record<string, any>>(url: string | URL, options?: { fetchImpl?: typeof fetch }): Promise<T>
 ```
 
 ## Examples
 
 ### Downloading the Master Database
 
-```javascript
+```typescript
 import { downloadMasterDatabase } from 'shamela';
 
-(async () => {
-    try {
-        // Download as SQLite database
-        const dbPath = await downloadMasterDatabase({
-            outputFile: { path: './shamela_master.db' },
-        });
-        console.log(`Master database downloaded to: ${dbPath}`);
-
-        // Download as JSON for programmatic access
-        const jsonPath = await downloadMasterDatabase({
-            outputFile: { path: './shamela_master.json' },
-        });
-        console.log(`Master data exported to: ${jsonPath}`);
-        console.log('The JSON file includes authors, books, categories, and the master version number.');
-    } catch (error) {
-        console.error('Error downloading master database:', error);
-    }
-})();
+// Download as SQLite
+const dbPath = await downloadMasterDatabase({
+  outputFile: { path: './shamela_master.db' }
+});
+console.log(`Downloaded to: ${dbPath}`);
+
+// Download as JSON
+const jsonPath = await downloadMasterDatabase({
+  outputFile: { path: './shamela_master.json' }
+});
 ```
 
 ### Downloading a Book
 
-```javascript
+```typescript
 import { downloadBook, getBookMetadata } from 'shamela';
 
-(async () => {
-    const bookId = 26592;
+const bookId = 26592;
 
-    try {
-        // Download book as database file
-        await downloadBook(bookId, {
-            outputFile: { path: `./book_${bookId}.db` },
-        });
+// Download book
+await downloadBook(bookId, {
+  outputFile: { path: `./book_${bookId}.db` }
+});
 
-        // Download with pre-fetched metadata
-        const metadata = await getBookMetadata(bookId);
-        await downloadBook(bookId, {
-            bookMetadata: metadata,
-            outputFile: { path: `./book_${bookId}.json` },
-        });
-    } catch (error) {
-        console.error('Error downloading book:', error);
-    }
-})();
+// With pre-fetched metadata
+const metadata = await getBookMetadata(bookId);
+await downloadBook(bookId, {
+  bookMetadata: metadata,
+  outputFile: { path: `./book_${bookId}.json` }
+});
 ```
 
 ### Retrieving Book Data
 
-```javascript
+```typescript
 import { getBook } from 'shamela';
 
-(async () => {
-    try {
-        const bookData = await getBook(26592);
-
-        console.log(`Book has ${bookData.pages.length} pages`);
+const book = await getBook(26592);
 
-        if (bookData.titles) {
-            console.log(`Book has ${bookData.titles.length} titles/chapters`);
-
-            // Display table of contents
-            bookData.titles.forEach((title) => {
-                console.log(`${title.id}: ${title.content} (Page ${title.page})`);
-            });
-        }
-
-        // Access page content
-        const firstPage = bookData.pages[0];
-        console.log(`First page content: ${firstPage.content.substring(0, 100)}...`);
-    } catch (error) {
-        console.error('Error retrieving book:', error);
-    }
-})();
-```
+console.log(`Book has ${book.pages.length} pages`);
 
-### Retrieving Master Data in memory
-
-```javascript
-import { getMaster } from 'shamela';
-
-(async () => {
-    try {
-        const masterData = await getMaster();
+// Display table of contents
+book.titles?.forEach(title => {
+  console.log(`${title.id}: ${title.content} (Page ${title.page})`);
+});
 
-        console.log(`Master snapshot version: ${masterData.version}`);
-        console.log(`Master dataset includes ${masterData.books.length} books`);
-        console.log(`Master dataset includes ${masterData.categories.length} categories`);
-    } catch (error) {
-        console.error('Error retrieving master data:', error);
-    }
-})();
+// Access page content
+const firstPage = book.pages[0];
+console.log(firstPage.content.substring(0, 100));
 ```
 
-### Getting Book Cover URLs
-
-```javascript
-import { getCoverUrl, downloadMasterDatabase } from 'shamela';
+### Getting Book Covers
 
-(async () => {
-    try {
-        // Download master data to get book information
-        const masterData = await downloadMasterDatabase({
-            outputFile: { path: './master.json' },
-        });
+```typescript
+import { getCoverUrl, getMaster } from 'shamela';
 
-        // Read the master data
-        const data = await Bun.file('./master.json').json();
+const master = await getMaster();
 
-        // Generate cover URLs for all books
-        data.books.forEach((book) => {
-            const coverUrl = getCoverUrl(book.id);
-            console.log(`${book.name}: ${coverUrl}`);
-        });
-    } catch (error) {
-        console.error('Error processing covers:', error);
-    }
-})();
+// Generate cover URLs for all books
+master.books.forEach(book => {
+  const coverUrl = getCoverUrl(book.id);
+  console.log(`${book.name}: ${coverUrl}`);
+});
 ```
 
 ## Data Structures
 
-The library provides comprehensive TypeScript types for all data structures:
-
 ### BookData
 
-- `pages`: Array of raw rows from the `page` table, including `content`, `id`, `part`, `page`, `number`, `services`, and `is_deleted`.
-- `titles`: Array of raw rows from the `title` table with `content`, `id`, `page`, `parent`, and `is_deleted`.
+```typescript
+type BookData = {
+  pages: Page[];
+  titles: Title[];
+};
+```
 
 ### MasterData
 
-- `authors`: Raw entries from the `author` table with the original `biography`, `death_text`, `death_number`, `is_deleted`, and `name` fields.
-- `books`: Raw entries from the `book` table containing the original metadata columns (`author`, `bibliography`, `category`, `date`, `hint`, `major_release`, `metadata`, `minor_release`, `pdf_links`, `printed`, `type`, and `is_deleted`).
-- `categories`: Raw entries from the `category` table including `is_deleted`, `order`, and `name`.
-- `version`: Version number reported by the Shamela API for the downloaded master database.
+```typescript
+type MasterData = {
+  authors: Author[];
+  books: Book[];
+  categories: Category[];
+  version: number;
+};
+```
 
 ### Page
 
-- `id`: Unique identifier.
-- `content`: Text content of the page.
-- `part`, `page`, `number`: Numeric references stored exactly as they appear in the source database.
-- `services`: Optional metadata column from the source database.
-- `is_deleted`: Flag indicating whether the page has been marked as deleted in Shamela updates.
+```typescript
+type Page = {
+  id: number;
+  content: string;
+  part?: string;
+  page?: number;
+  number?: string;
+};
+```
 
 ### Title
 
-- `id`: Unique identifier.
-- `content`: Title text.
-- `page`: Page number where title appears (if available).
-- `parent`: Optional parent title ID for hierarchical structure.
-- `is_deleted`: Flag indicating whether the title has been marked as deleted.
-
-### Content helpers
+```typescript
+type Title = {
+  id: number;
+  content: string;
+  page: number;
+  parent?: number;
+};
+```
 
-- `parseContentRobust(content: string)`: Converts Shamela page HTML into a list of structured lines while preserving title markers and punctuation.
-- `sanitizePageContent(content: string)`: Removes common footnote markers and normalises ligatures from Shamela pages.
+### Content Helpers
 
-## Next.js demo
+- `parseContentRobust(content: string)`: Converts Shamela page HTML into structured lines
+- `sanitizePageContent(content: string)`: Removes footnote markers and normalizes text
+- `splitPageBodyFromFooter(content: string)`: Separates page content from footnotes
+- `removeArabicNumericPageMarkers(text: string)`: Removes Arabic page number markers
+- `removeTagsExceptSpan(content: string)`: Strips HTML tags except span elements
 
-A minimal Next.js 16 application in `demo/` replaces the previous Storybook setup and offers an RTL-friendly explorer for the Shamela APIs. The server renders requests so the browser can bypass CORS limits and you only need to provide an API key and book identifier at runtime.
+## Next.js Demo
 
-Create a `demo/.env.local` file (or export the variables in your shell) containing the real endpoints you wish to call:
+A minimal Next.js 16 demo application is available in the `demo/` directory.
 
-```dotenv
-SHAMELA_API_MASTER_PATCH_ENDPOINT=https://dev.shamela.ws/api/v1/patches/master
-SHAMELA_API_BOOKS_ENDPOINT=https://dev.shamela.ws/api/v1/patches/book-updates
-# Optional when hosting the wasm asset yourself
-# SHAMELA_SQLJS_WASM_URL=https://example.com/sql-wasm.wasm
-```
+**Setup:**
 
-Then launch the demo:
+Create `demo/.env.local`:
 
-```bash
-bun run demo
+```env
+SHAMELA_API_KEY=your_api_key
+SHAMELA_API_MASTER_PATCH_ENDPOINT=https://SHAMELA_INSTANCE.ws/api/master_patch
+SHAMELA_API_BOOKS_ENDPOINT=https://SHAMELA_INSTANCE.ws/api/books
 ```
 
-Visit [http://localhost:3000](http://localhost:3000) to enter your API key, choose a book ID, and call helpers like `getMasterMetadata`, `getMaster`, `getBook`, and `downloadMasterDatabase` directly from the interface. For production-style builds use:
+**Run:**
 
 ```bash
-bun run demo:build
-bun run demo:start
+bun run demo              # Development
+bun run demo:build        # Production build
+bun run demo:start        # Production server
 ```
 
-When deploying to Vercel, point the project to the `demo` directory and supply the same environment variables in the dashboard so the API routes can reach Shamela.
+Visit [http://localhost:3000](http://localhost:3000) to explore the API.
 
 ## Troubleshooting
 
-### Error: "Unable to locate sql-wasm.wasm file"
+### Error: "Unable to automatically locate sql-wasm.wasm file"
 
-This error occurs when the library cannot automatically find the WASM file. This is common in bundled environments like Next.js with Turbopack/Webpack.
+This occurs in bundled environments (Next.js, webpack, Turbopack). 
 
-**Solution:** Use the `createNodeConfig` helper or explicitly configure `sqlJsWasmUrl`:
+**Solution:** Add explicit configuration:
 
 ```typescript
-import { configure, createNodeConfig } from 'shamela';
-
-// Option 1: Use the helper (recommended)
-configure(createNodeConfig({
-  apiKey: process.env.SHAMELA_API_KEY,
-  booksEndpoint: process.env.SHAMELA_BOOKS_ENDPOINT,
-  masterPatchEndpoint: process.env.SHAMELA_MASTER_ENDPOINT,
-}));
-
-// Option 2: Manual configuration
+import { configure } from 'shamela';
 import { join } from 'node:path';
 
 configure({
   sqlJsWasmUrl: join(process.cwd(), 'node_modules', 'sql.js', 'dist', 'sql-wasm.wasm'),
   apiKey: process.env.SHAMELA_API_KEY,
-  // ... other config
+  booksEndpoint: process.env.SHAMELA_BOOKS_ENDPOINT,
+  masterPatchEndpoint: process.env.SHAMELA_MASTER_ENDPOINT,
 });
 ```
 
-### Error: "ENOENT: no such file or directory, open 'https://...'"
-
-This means you're in a Node.js environment but providing an HTTPS URL for the WASM file. Node.js requires a filesystem path, not a URL.
-
-**Solution:** Use a filesystem path instead of a URL:
-
-```typescript
-// ❌ Wrong - HTTPS URL in Node.js
-configure({
-  sqlJsWasmUrl: 'https://cdn.jsdelivr.net/npm/sql.js@1.13.0/dist/sql-wasm.wasm'
-});
+### Next.js: Module not found errors
 
-// ✅ Correct - Filesystem path or use createNodeConfig
-import { createNodeConfig } from 'shamela';
+1. Add to `next.config.ts`:
+   ```typescript
+   serverExternalPackages: ['shamela', 'sql.js']
+   ```
 
-configure(createNodeConfig({
-  apiKey: process.env.SHAMELA_API_KEY,
-  // ... other config
-}));
-```
+2. Only import shamela in server-side code
 
-### Next.js: Module not found errors during build
+3. Create a separate `lib/shamela-server.ts` for configuration
 
-If you see webpack/Turbopack errors about not being able to resolve modules during the build phase:
+### Production build works differently than development
 
-1. Make sure you've added `serverExternalPackages` to your `next.config.js` (see [Next.js / Bundled Environments](#nextjs--bundled-environments))
-2. Ensure you're only importing shamela in server-side code (Server Actions, API Routes, Server Components)
-3. Never import shamela in `layout.tsx` or client components
-4. Create a separate `lib/shamela-server.ts` file for configuration
+Ensure `serverExternalPackages` is set in your Next.js config for both development and production.
 
-### Double `node_modules/node_modules` path error
+### Monorepo setup issues
 
-If you see paths like `/path/to/project/node_modules/node_modules/sql.js/...`, this indicates the library's auto-detection failed due to bundling. Use explicit configuration:
+Adjust the WASM path based on your structure:
 
 ```typescript
-import { configure, createNodeConfig } from 'shamela';
-
-configure(createNodeConfig({
-  apiKey: process.env.SHAMELA_API_KEY,
-  booksEndpoint: process.env.SHAMELA_BOOKS_ENDPOINT,
-  masterPatchEndpoint: process.env.SHAMELA_MASTER_ENDPOINT,
-}));
+configure({
+  sqlJsWasmUrl: join(process.cwd(), '../../node_modules/sql.js/dist/sql-wasm.wasm'),
+  // ... other config
+});
 ```
 
 ## Testing
 
-The library includes comprehensive tests powered by `bun test`. To run the unit suite, ensure you have the necessary environment variables set, then execute:
-
-```bash
-bun test src
-```
-
-For end-to-end tests:
-
-```bash
-bun run e2e
-```
-
-### Formatting
-
-Apply Biome formatting across the repository with:
+Run tests with Bun:
 
 ```bash
-bun run format
+bun test src              # Unit tests
+bun run e2e               # End-to-end tests
+bun run format            # Format code
+bun run lint              # Lint code
 ```
 
 ## License
 
-This project is licensed under the MIT License - see the LICENSE file for details.
+MIT License - see LICENSE file for details.