shamela 1.0.6 → 1.2.0

package/README.md

@@ -1,18 +1,18 @@
-# Shamela
+# shamela
 
 [![wakatime](https://wakatime.com/badge/user/a0b906ce-b8e7-4463-8bce-383238df6d4b/project/faef70ab-efdb-448b-ab83-0fc66c95888e.svg)](https://wakatime.com/badge/user/a0b906ce-b8e7-4463-8bce-383238df6d4b/project/faef70ab-efdb-448b-ab83-0fc66c95888e)
 [![E2E](https://github.com/ragaeeb/shamela/actions/workflows/e2e.yml/badge.svg)](https://github.com/ragaeeb/shamela/actions/workflows/e2e.yml)
 [![Node.js CI](https://github.com/ragaeeb/shamela/actions/workflows/build.yml/badge.svg)](https://github.com/ragaeeb/shamela/actions/workflows/build.yml) ![GitHub License](https://img.shields.io/github/license/ragaeeb/shamela)
 ![GitHub Release](https://img.shields.io/github/v/release/ragaeeb/shamela)
 [![codecov](https://codecov.io/gh/ragaeeb/shamela/graph/badge.svg?token=PK55V1R324)](https://codecov.io/gh/ragaeeb/shamela)
-[![Size](https://deno.bundlejs.com/badge?q=shamela@1.0.5)](https://bundlejs.com/?q=shamela%401.0.5)
+[![Size](https://deno.bundlejs.com/badge?q=shamela@latest)](https://bundlejs.com/?q=shamela%40latest)
 ![typescript](https://badgen.net/badge/icon/typescript?icon=typescript&label&color=blue)
 ![npm](https://img.shields.io/npm/v/shamela)
 ![npm](https://img.shields.io/npm/dm/shamela)
 ![GitHub issues](https://img.shields.io/github/issues/ragaeeb/shamela)
 ![GitHub stars](https://img.shields.io/github/stars/ragaeeb/shamela?style=social)
 
-A `NodeJS` library for accessing and downloading Maktabah Shamela v4 APIs. This library provides easy-to-use functions to interact with the Shamela API, download master and book databases, and retrieve book data programmatically.
+A `Node.js` library for accessing and downloading Maktabah Shamela v4 APIs. This library provides easy-to-use functions to interact with the Shamela API, download master and book databases, and retrieve book data programmatically.
 
 ## Table of Contents
 
@@ -26,15 +26,24 @@ A `NodeJS` library for accessing and downloading Maktabah Shamela v4 APIs. This
         - [getBookMetadata](#getbookmetadata)
         - [downloadBook](#downloadbook)
         - [getBook](#getbook)
+        - [getCoverUrl](#getcoverurl)
 - [Examples](#examples)
     - [Downloading the Master Database](#downloading-the-master-database)
     - [Downloading a Book](#downloading-a-book)
     - [Retrieving Book Data](#retrieving-book-data)
+    - [Getting Book Cover URLs](#getting-book-cover-urls)
+- [Data Structures](#data-structures)
 - [Testing](#testing)
 - [License](#license)
 
 ## Installation
 
+```bash
+bun add shamela
+```
+
+or
+
 ```bash
 npm install shamela
 ```
@@ -55,9 +64,10 @@ pnpm install shamela
 
 Before using the library, you need to set up some environment variables for API keys and endpoints:
 
-`SHAMELA_API_KEY`: Your API key for accessing the Shamela API.
-`SHAMELA_API_MASTER_PATCH_ENDPOINT`: The endpoint URL for the master database patches.
-`SHAMELA_API_BOOKS_ENDPOINT`: The base endpoint URL for book-related API calls.
+- `SHAMELA_API_KEY`: Your API key for accessing the Shamela API.
+- `SHAMELA_API_MASTER_PATCH_ENDPOINT`: The endpoint URL for the master database patches.
+- `SHAMELA_API_BOOKS_ENDPOINT`: The base endpoint URL for book-related API calls.
+
 You can set these variables in a `.env` file at the root of your project:
 
 ```dotenv
@@ -73,7 +83,14 @@ SHAMELA_API_BOOKS_ENDPOINT=https://shamela.ws/api/books
 First, import the library functions into your project:
 
 ```javascript
-import { getMasterMetadata, downloadMasterDatabase, getBookMetadata, downloadBook, getBook } from 'shamela';
+import {
+    getMasterMetadata,
+    downloadMasterDatabase,
+    getBookMetadata,
+    downloadBook,
+    getBook,
+    getCoverUrl,
+} from 'shamela';
 ```
 
 ### API Functions
@@ -84,34 +101,54 @@ Fetches metadata for the master database.
 
 ```typescript
 getMasterMetadata(version?: number): Promise<GetMasterMetadataResponsePayload>
-
 ```
 
-- version (optional): The version number of the master database you want to fetch.
+- `version` (optional): The version number of the master database you want to check for updates (defaults to 0)
+
+**Returns:** Promise that resolves to master database metadata including download URL and version
 
-Example:
+**Example:**
 
 ```javascript
 const masterMetadata = await getMasterMetadata();
+console.log(masterMetadata.url); // Download URL for master database patch
+console.log(masterMetadata.version); // Latest version number
+
+// Check for updates from a specific version
+const updates = await getMasterMetadata(5);
 ```
 
 #### downloadMasterDatabase
 
-Downloads the master database and saves it to a specified path.
+Downloads the master database and saves it to a specified path. The master database contains comprehensive information about all books, authors, and categories available in the Shamela library.
 
 ```typescript
 downloadMasterDatabase(options: DownloadMasterOptions): Promise<string>
-
 ```
 
-- options: An object containing:
-    - masterMetadata (optional): The metadata obtained from getMasterMetadata.
-    - outputFile: An object specifying the output path.
+- `options`: Configuration object containing:
+    - `masterMetadata` (optional): Pre-fetched metadata from `getMasterMetadata`
+    - `outputFile`: Object with `path` property specifying the output file path
 
-Example:
+**Returns:** Promise that resolves to the path of the created output file
+
+**Example:**
 
 ```javascript
+// Download as SQLite database
+await downloadMasterDatabase({
+    outputFile: { path: './master.db' },
+});
+
+// Download as JSON
+await downloadMasterDatabase({
+    outputFile: { path: './master.json' },
+});
+
+// Use pre-fetched metadata for efficiency
+const masterMetadata = await getMasterMetadata();
 await downloadMasterDatabase({
+    masterMetadata,
     outputFile: { path: './master.db' },
 });
 ```
@@ -124,33 +161,101 @@ Fetches metadata for a specific book.
 getBookMetadata(id: number, options?: GetBookMetadataOptions): Promise<GetBookMetadataResponsePayload>
 ```
 
-- id: The ID of the book.
-- options (optional): An object containing:
-    - majorVersion: The major version of the book.
-    - minorVersion: The minor version of the book.
+- `id`: The unique identifier of the book
+- `options` (optional): Configuration object containing:
+    - `majorVersion`: The major version to check against
+    - `minorVersion`: The minor version to check against
 
-Example:
+**Returns:** Promise that resolves to book metadata including release URLs and versions
+
+**Example:**
 
 ```javascript
-await downloadMasterDatabase({
-    outputFile: { path: './master.db' },
+const metadata = await getBookMetadata(26592);
+console.log(metadata.majorReleaseUrl); // URL for downloading the book
+console.log(metadata.majorRelease); // Major version number
+
+// Check specific versions
+const versionedMetadata = await getBookMetadata(26592, {
+    majorVersion: 1,
+    minorVersion: 2,
+});
+```
+
+#### downloadBook
+
+Downloads and processes a book from the Shamela database. This function downloads the book's database files, applies patches if available, and exports the data to the specified format.
+
+```typescript
+downloadBook(id: number, options: DownloadBookOptions): Promise<string>
+```
+
+- `id`: The unique identifier of the book to download
+- `options`: Configuration object containing:
+    - `bookMetadata` (optional): Pre-fetched metadata from `getBookMetadata`
+    - `outputFile`: Object with `path` property specifying the output file path
+
+**Returns:** Promise that resolves to the path of the created output file
+
+**Example:**
+
+```javascript
+// Download as JSON
+await downloadBook(26592, {
+    outputFile: { path: './book.json' },
+});
+
+// Download as SQLite database
+await downloadBook(26592, {
+    outputFile: { path: './book.db' },
+});
+
+// Use pre-fetched metadata for efficiency
+const bookMetadata = await getBookMetadata(26592);
+await downloadBook(26592, {
+    bookMetadata,
+    outputFile: { path: './book.db' },
 });
 ```
 
 #### getBook
 
-Retrieves the data of a book as a JavaScript object.
+Retrieves complete book data as a JavaScript object. This is a convenience function that handles temporary file creation and cleanup automatically.
 
 ```typescript
 getBook(id: number): Promise<BookData>
 ```
 
-- id: The ID of the book.
+- `id`: The unique identifier of the book to retrieve
+
+**Returns:** Promise that resolves to complete book data including pages and titles
 
-Example:
+**Example:**
 
 ```javascript
 const bookData = await getBook(26592);
+console.log(bookData.pages.length); // Number of pages in the book
+console.log(bookData.titles?.length); // Number of title entries
+console.log(bookData.pages[0].content); // Content of the first page
+```
+
+#### getCoverUrl
+
+Generates the URL for a book's cover image.
+
+```typescript
+getCoverUrl(bookId: number): string
+```
+
+- `bookId`: The unique identifier of the book
+
+**Returns:** The complete URL to the book's cover image
+
+**Example:**
+
+```javascript
+const coverUrl = getCoverUrl(26592);
+console.log(coverUrl); // "https://shamela.ws/covers/26592.jpg"
 ```
 
 ## Examples
@@ -161,21 +266,47 @@ const bookData = await getBook(26592);
 import { downloadMasterDatabase } from 'shamela';
 
 (async () => {
-    await downloadMasterDatabase({
-        outputFile: { path: './master.db' },
-    });
+    try {
+        // Download as SQLite database
+        const dbPath = await downloadMasterDatabase({
+            outputFile: { path: './shamela_master.db' },
+        });
+        console.log(`Master database downloaded to: ${dbPath}`);
+
+        // Download as JSON for programmatic access
+        const jsonPath = await downloadMasterDatabase({
+            outputFile: { path: './shamela_master.json' },
+        });
+        console.log(`Master data exported to: ${jsonPath}`);
+    } catch (error) {
+        console.error('Error downloading master database:', error);
+    }
 })();
 ```
 
 ### Downloading a Book
 
 ```javascript
-import { downloadBook } from 'shamela';
+import { downloadBook, getBookMetadata } from 'shamela';
 
 (async () => {
-    await downloadBook(26592, {
-        outputFile: { path: './book.db' },
-    });
+    const bookId = 26592;
+
+    try {
+        // Download book as database file
+        await downloadBook(bookId, {
+            outputFile: { path: `./book_${bookId}.db` },
+        });
+
+        // Download with pre-fetched metadata
+        const metadata = await getBookMetadata(bookId);
+        await downloadBook(bookId, {
+            bookMetadata: metadata,
+            outputFile: { path: `./book_${bookId}.json` },
+        });
+    } catch (error) {
+        console.error('Error downloading book:', error);
+    }
 })();
 ```
 
@@ -185,23 +316,109 @@ import { downloadBook } from 'shamela';
 import { getBook } from 'shamela';
 
 (async () => {
-    const bookData = await getBook(26592);
-    console.log(bookData);
+    try {
+        const bookData = await getBook(26592);
+
+        console.log(`Book has ${bookData.pages.length} pages`);
+
+        if (bookData.titles) {
+            console.log(`Book has ${bookData.titles.length} titles/chapters`);
+
+            // Display table of contents
+            bookData.titles.forEach((title) => {
+                console.log(`${title.id}: ${title.content} (Page ${title.page})`);
+            });
+        }
+
+        // Access page content
+        const firstPage = bookData.pages[0];
+        console.log(`First page content: ${firstPage.content.substring(0, 100)}...`);
+    } catch (error) {
+        console.error('Error retrieving book:', error);
+    }
+})();
+```
+
+### Getting Book Cover URLs
+
+```javascript
+import { getCoverUrl, downloadMasterDatabase } from 'shamela';
+
+(async () => {
+    try {
+        // Download master data to get book information
+        const masterData = await downloadMasterDatabase({
+            outputFile: { path: './master.json' },
+        });
+
+        // Read the master data
+        const data = await Bun.file('./master.json').json();
+
+        // Generate cover URLs for all books
+        data.books.forEach((book) => {
+            const coverUrl = getCoverUrl(book.id);
+            console.log(`${book.name}: ${coverUrl}`);
+        });
+    } catch (error) {
+        console.error('Error processing covers:', error);
+    }
 })();
 ```
 
+## Data Structures
+
+The library provides comprehensive TypeScript types for all data structures:
+
+### BookData
+
+- `pages`: Array of raw rows from the `page` table, including `content`, `id`, `part`, `page`, `number`, `services`, and `is_deleted`.
+- `titles`: Array of raw rows from the `title` table with `content`, `id`, `page`, `parent`, and `is_deleted`.
+
+### MasterData
+
+- `authors`: Raw entries from the `author` table with the original `biography`, `death_text`, `death_number`, `is_deleted`, and `name` fields.
+- `books`: Raw entries from the `book` table containing the original metadata columns (`author`, `bibliography`, `category`, `date`, `hint`, `major_release`, `metadata`, `minor_release`, `pdf_links`, `printed`, `type`, and `is_deleted`).
+- `categories`: Raw entries from the `category` table including `is_deleted`, `order`, and `name`.
+
+### Page
+
+- `id`: Unique identifier.
+- `content`: Text content of the page.
+- `part`, `page`, `number`: Numeric references stored exactly as they appear in the source database.
+- `services`: Optional metadata column from the source database.
+- `is_deleted`: Flag indicating whether the page has been marked as deleted in Shamela updates.
+
+### Title
+
+- `id`: Unique identifier.
+- `content`: Title text.
+- `page`: Page number where title appears (if available).
+- `parent`: Optional parent title ID for hierarchical structure.
+- `is_deleted`: Flag indicating whether the title has been marked as deleted.
+
+### Content helpers
+
+- `parseContentRobust(content: string)`: Converts Shamela page HTML into a list of structured lines while preserving title markers and punctuation.
+- `sanitizePageContent(content: string)`: Removes common footnote markers and normalises ligatures from Shamela pages.
+
 ## Testing
 
-The library includes tests to help you understand how the APIs are used. To run the tests, ensure you have the necessary environment variables set, then execute:
+The library includes comprehensive tests. To run them, ensure you have the necessary environment variables set, then execute:
 
 ```bash
-npm run test
+bun test
 ```
 
 For end-to-end tests:
 
 ```bash
-npm run e2e
+bun run e2e
+```
+
+For CI environment:
+
+```bash
+bun run e2e:ci
 ```
 
 ## License