isnady 0.1.7 → 0.1.8

package/LICENSE

@@ -1,21 +1,21 @@
-MIT License
-
-Copyright (c) 2026 Bayram Kotan
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+MIT License
+
+Copyright (c) 2026 Bayram Kotan
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -8,12 +8,12 @@
 
 <p align="center">
   <a href="https://pypi.org/project/isnady/">
-    <img src="https://img.shields.io/badge/PyPI-v0.1.7-1D4777?style=for-the-badge&logo=pypi&logoColor=white" alt="PyPI">
+    <img src="https://img.shields.io/badge/PyPI-v0.1.8-1D4777?style=for-the-badge&logo=pypi&logoColor=white" alt="PyPI">
   </a>
   <img src="https://img.shields.io/pypi/pyversions/isnady?style=for-the-badge&color=A47E24&logo=python&logoColor=white" alt="Python">
   <img src="https://img.shields.io/badge/Platform-Windows%20%7C%20Linux%20%7C%20macOS-5B6878?style=for-the-badge" alt="Platform">
   <a href="https://www.npmjs.com/package/isnady">
-    <img src="https://img.shields.io/badge/npm-v0.1.7-14304F?style=for-the-badge&logo=npm" alt="npm">
+    <img src="https://img.shields.io/badge/npm-v0.1.8-14304F?style=for-the-badge&logo=npm" alt="npm">
   </a>
   <img src="https://img.shields.io/badge/License-MIT-D6B25E?style=for-the-badge" alt="License">
 </p>
@@ -43,7 +43,7 @@
   <img src="https://raw.githubusercontent.com/bayramkotan/isnady/main/assets/screenshots/search-arabic.png" alt="Searching النيات finds بالنيات in Sahih al-Bukhari 1, with the chain above the text" width="850">
 </p>
 
-> **Pre-alpha (0.1.7).** Search and chains of transmission work today, on data you import
+> **Pre-alpha (0.1.8).** Search and chains of transmission work today, on data you import
 > in one command. Narrators, scholars, gradings and the rest are being built, in the open.
 
 ---
@@ -170,21 +170,26 @@ it opens it.
 
 ## 🚀 Quick Start
 
-isnady ships without hadith data: you choose the sources. One command brings in Sahih
-al-Bukhari and Sunan Abi Dawud in Arabic and Turkish from the open
-[fawazahmed0/hadith-api](https://github.com/fawazahmed0/hadith-api) collection (public
-domain):
+isnady ships without hadith data; it reads it from a **data repository**. The default is
+isnady's own, [**bayramkotan/isnady-data**](https://github.com/bayramkotan/isnady-data), which
+holds every Sunni and Shia collection and rijal work isnady can read — 35 sources, from Sahih
+al-Bukhari to al-Kafi (see [Data sources](#-data-sources) below). Open the window with `iy`,
+choose **File → Data Sources**, and import any of them with one click — or **Import all** — in
+the languages you tick. The same from the command line:
 
 ```bash
-iy import fawazahmed0 https://cdn.jsdelivr.net/gh/fawazahmed0/hadith-api@1/editions.json \
-   --book bukhari --book abudawud --language ara --language tur --license Unlicense
+iy catalog list                         # the sources of the repository in use
+iy catalog import sahih_al_bukhari --language ara --language tur
+iy catalog import musnad_ahmad_ibn_hanbal
+iy catalog import al_kafi_al_kulayni    # with al-Majlisi's and al-Behbudi's grades
+iy catalog import taqrib_al_tahdhib_ibn_hajar_al_asqalani   # identifies the narrators in the chains
+iy catalog import --all                 # everything
 ```
 
-Or open the window with `iy` and choose **File → Data Sources**: every built-in collection
-(al-Bukhari, Muslim, Abu Dawud, al-Tirmidhi, al-Nasa'i, Ibn Maja, the Muwatta and three
-forty-hadith books) and the Taqrib import with one click — or **Import all** — in the
-languages you tick. You can add your own files or links there too, and move the data
-folder anywhere you like.
+Other repositories can be chosen instead — the datasets others publish on GitHub and Hugging
+Face are in the list — and any repository with an isnady catalog can be added
+(`iy repo add https://github.com/owner/name`). Your own files and links can be added too, and
+the data folder can be moved anywhere you like.
 
 On the command line:
 
@@ -196,13 +201,8 @@ iy chain bukhari 1           # one chain, narrator by narrator
 
 Importing takes a minute or two; the search index and the chains are built as part of it.
 
-To identify the narrators in those chains, add Ibn Hajar's *Taqrib al-Tahdhib* — 8,824
-narrators with his verdict on each — from the [OpenITI](https://github.com/OpenITI) corpus:
-
-```bash
-iy import taqrib https://raw.githubusercontent.com/OpenITI/0875AH/master/data/0852IbnHajarCasqalani/0852IbnHajarCasqalani.TaqribTahdhib/0852IbnHajarCasqalani.TaqribTahdhib.JK000121-ara1.completed
-iy narrator الزهري
-```
+With Ibn Hajar's *Taqrib al-Tahdhib* imported (8,824 narrators, his verdict on each), every
+chain is read narrator by narrator: `iy narrator الزهري`.
 
 ---
 
@@ -214,7 +214,9 @@ iy narrator الزهري
 | **Attached prefixes** | `النيات` finds `بالنيات`, because Arabic joins bi-, wa-, fa- and al- to the word. *Whole words only* turns this off |
 | **Latin-script text** | Case and accents are ignored: `NIYET`, `niyet` and `nîyet` are one word; `Muʿādh`, `Mu'adh` and `Muadh` too |
 | **Match** | All words, any word, or the exact phrase |
-| **Filters** | Book and language; the language filter also chooses which translations appear |
+| **Books** | All books, all Sunni, all Shia, or any few ticked one by one — Sunni collections listed first, Shia after them |
+| **Language** | One language to search in; the languages you do not read are folded behind an arrow on each hadith (**Settings → Books and languages**), a click opens them |
+| **Active books** | A book can be switched off in **Settings → Books and languages**: it stays imported but is left out of searches, book lists and statistics until switched on |
 | **Grades** | Shown under each hadith with the scholar's name, exactly as given |
 | **Highlighting** | On the original, diacritised text, diacritics included |
 
@@ -311,7 +313,12 @@ al-Din 'Abd al-Hamid on Sunan Abi Dawud) and shows:
 
 Two graders who agree far too often to be independent (98.1%) count as one voice in the model,
 and the page says so. Results are computed once and kept in a file. `iy stats graders`.
-Narrators, books, hadith, chains, correlations and the models follow, tab by tab.
+Narrators, books, hadith, chains, correlations and the models follow, tab by tab. **Correlations
+and tests** sets every measure against every other — Spearman's ρ with Benjamini–Hochberg q-values,
+and the partial ρ with all the others held equal (what each factor of the chain says of each
+scholar's grade on its own) — and tests what differs: the scholars' grades pair by pair
+(Stuart–Maxwell), the books (Kruskal–Wallis, Mann–Whitney with Cliff's δ), and whether the hadith
+model's shape holds (Brant's test). `iy stats lab`.
 
 <p align="center">
   <img src="https://raw.githubusercontent.com/bayramkotan/isnady/main/assets/screenshots/statistics.png" alt="Statistics — the graders of Sunan Abi Dawud: Krippendorff's alpha, agreement with intervals, the model's certainty" width="850">
@@ -477,24 +484,46 @@ command line.
 
 ---
 
-## 📚 Data and licences
-
-isnady reads open formats and keeps the source of everything:
-
-- **Supported now:** the [fawazahmed0/hadith-api](https://github.com/fawazahmed0/hadith-api)
-  JSON format, from a file or a URL, with or without authentication (username and
-  password, bearer token, or API key; credentials are never stored).
-- **Rijal:** Ibn Hajar's *Taqrib al-Tahdhib* in OpenITI mARkdown — each narrator's name,
-  kunya, verdict, rank, tabaqa, death year and book marks, read by the rules Ibn Hajar sets
-  out in his own introduction. The OpenITI release does not state its licence in the
-  repository, so it is kept as tier C (used on your computer, never redistributed) until
-  it is confirmed.
-- **Coming:** CSV, SQL/SQLite, OpenITI mARkdown, REST APIs and Shamela, so any collection
-  you have can be brought in.
-
-Every source records a licence **tier**: **A** may be redistributed, **B** may be
-redistributed under its conditions, **C** may not. Sources you add yourself stay on your
-computer. **Help → Licences** lists every source in your database with its licence.
+## 📚 Data sources
+
+isnady reads its sources from a **repository**: a `catalog.json` listing books and rijal works,
+where each file is, its format, its licence and its tradition. Choose one in **Settings → Data
+sources** (or `iy repo use ID`); add your own with **Add a repository…** (`iy repo add URL`: a
+GitHub repository, a Hugging Face dataset, a catalog URL or a folder).
+
+**The default — [isnady-data](https://github.com/bayramkotan/isnady-data)** holds every source
+below in one place and in isnady's own format, each book in a folder named by its full title and
+its compiler — `sunni/sahih_al_bukhari/`, `sunni/musnad_ahmad_ibn_hanbal/`, `shia/al_kafi_al_kulayni/`
+(a `book.json` and one file per language) — and the works on the narrators in `rijal/sunni/`,
+`rijal/shia/`.
+
+| | Sunni | Shia |
+|---|---|---|
+| **Hadith** | Sahih al-Bukhari, Sahih Muslim, Sunan Abi Dawud, Jami' al-Tirmidhi, Sunan al-Nasa'i, Sunan Ibn Maja, the Muwatta, the Forty of al-Nawawi, Forty Hadith Qudsi, the Forty of Shah Waliullah — Arabic and up to eight translations, with grades · Musnad Ahmad, Sunan al-Darimi · Riyad al-Salihin, al-Shama'il al-Muhammadiyya, Bulugh al-Maram, al-Adab al-Mufrad, Mishkat al-Masabih | al-Kulayni's al-Kafi (8 volumes, with al-Majlisi's and al-Behbudi's grades) · al-Saduq's 'Uyun Akhbar al-Rida, al-Amali, al-Khisal, al-Tawhid, Ma'ani al-Akhbar, Thawab al-A'mal, Sifat al-Shi'a, Fada'il al-Shi'a · al-Mufid's al-Amali · Ibn Qulawayh's Kamil al-Ziyarat · al-Nu'mani's and al-Tusi's Kitab al-Ghayba · al-Ahwazi's al-Mu'min and al-Zuhd · al-Muhsini's Mu'jam al-Ahadith al-Mu'tabara — Arabic and English |
+| **Narrators** | Ibn Hajar, *Taqrib al-Tahdhib* | al-Najashi, *Rijal* |
+
+The credits for where each text was first published are in the repository's README.
+
+**Optional repositories**, each the dataset as its authors publish it (their books under their
+own keys, e.g. `bukhari-lk`, so editions with other numbering never mix):
+
+| Repository | What it holds | Licence |
+|---|---|---|
+| [fawazahmed0/hadith-api](https://github.com/fawazahmed0/hadith-api) | 10 collections, Arabic + 8 languages, grades | Unlicense |
+| [Thaqalayn](https://github.com/MohammedArab1/ThaqalaynAPI) | 16 Shia collections, Arabic + English, grades of al-Kafi | not stated (from thaqalayn.net) |
+| [Open Hadith Data](https://github.com/mhashim6/Open-Hadith-Data) | the nine books in Arabic with diacritics, seven with a commentary of every hadith | ODbL |
+| [hadith-json](https://github.com/AhmedBaset/hadith-json) | 17 books, Arabic + English (50,884 hadith, from sunnah.com) | not stated |
+| [LK Hadith Corpus](https://github.com/ShathaTm/LK-Hadith-Corpus) | the six books, Arabic + English, isnad and matn marked apart, grades | not stated (cite the authors' paper) |
+| [Hugging Face — meeAtif/hadith_datasets](https://huggingface.co/datasets/meeAtif/hadith_datasets) | the six books, Arabic + English, chapter titles, grades | MIT (as declared; text from sunnah.com) |
+| [Hadith-Data-Sets](https://github.com/abdelrahmaan/Hadith-Data-Sets) | the nine books in Arabic (62,169 hadith) | not stated |
+| [OpenITI](https://github.com/OpenITI) | the Taqrib and al-Najashi's Rijal | not stated in the repository |
+
+Every source records a licence **tier**: **A** may be redistributed, **B** may be redistributed
+under its conditions, **C** may not. Sources you add yourself stay on your computer. **Help →
+Licences** lists every source in your database with its licence. Any CSV or JSON table of hadith
+can be read through a catalog that names its columns (format `table`); the fawazahmed0 JSON
+format, OpenITI mARkdown (the Taqrib, al-Najashi) and files of your own with or without
+authentication (credentials are never stored) are read as well.
 
 ---
 
@@ -514,15 +543,18 @@ script. The command line never loads Qt.
 | `iy shortcut` | `isnady shortcut --no-desktop` | A desktop and applications-menu shortcut, with isnady's icon |
 | `iy term [NAME]` | `isnady term --search tadlis` | The glossary: a term's meaning, English or Turkish (`--lang tr`) |
 | `iy stats graders` | `isnady stats graders --book abudawud` | The graders of a book in depth: agreement, model, strictness |
+| `iy stats model` | `isnady stats model --grader albani` | What each scholar's grade follows: ordinal regression, odds ratios, surprises |
+| `iy stats lab` | `isnady stats lab --csv ./lab` | Correlations (Spearman, partial, BH q) and tests (Stuart–Maxwell, Kruskal–Wallis, Mann–Whitney, Brant) |
 | `iy book [KEY [CHAPTER]]` | `isnady book bukhari 2` | The books; a book's chapters; a chapter read in order |
 | `iy scholar [NAME]` | `isnady scholar albani` | The scholars in the data, or one of them with his measured statistics |
 | `iy tahric BOOK NUMBER` | `isnady tahric bukhari 1` | Other narrations of a hadith in the imported books (takhrij) |
 | `iy search --mode meaning WORDS` | `isnady search --mode meaning "komşu hakları"` | Search by meaning and across languages |
 | `iy narrator NAME` | `isnady narrator "Ibn Umar"` | A narrator: Ibn Hajar's verdict and rank, tabaqa, death year, books, other names; with `--limit 1` also teachers, students and hadith |
 | `iy import FORMAT FILE-OR-URL` | `isnady import …` | Import a source; `--book`, `--language`, `--license`, `--user`, `--token-env`, `--api-key-env` |
-| `iy catalog list` | `isnady catalog list` | Built-in sources and your own, with what is imported — the same as File → Data Sources |
-| `iy catalog import ID` | `isnady catalog import fawaz-muslim --language tur` | Import a listed source; `remove`, `add`, `delete` manage the list |
-| `iy catalog import --all` | `isnady catalog import --all --language ara` | Every built-in source at once |
+| `iy repo [list\|use\|show\|add\|remove]` | `isnady repo use thaqalayn` | The data repositories: isnady's own (default), the optional ones, yours |
+| `iy catalog list` | `isnady catalog list --repo lk-hadith-corpus` | The sources of the repository in use, and your own, with what is imported — the same as File → Data Sources |
+| `iy catalog import ID` | `isnady catalog import sahih_muslim --language tur` | Import a listed source; `remove`, `add`, `delete` manage the list |
+| `iy catalog import --all` | `isnady catalog import --all --language ara` | Every source of the repository at once |
 | `iy datadir [FOLDER]` | `isnady datadir /data/isnady` | Show or change where the database and settings live; `--as-is`, `--default` |
 | `iy formats` | `isnady formats` | Formats that can be imported |
 | `iy sources` | `isnady sources` | Imported sources, their licence and tier |

package/index.d.ts

@@ -1,11 +1,11 @@
-export declare const version: string;
-export declare const name: string;
-
-export interface IsnadyInfo {
-  name: string;
-  version: string;
-  dataLoaded: boolean;
-  homepage: string;
-}
-
-export declare function info(): IsnadyInfo;
+export declare const version: string;
+export declare const name: string;
+
+export interface IsnadyInfo {
+  name: string;
+  version: string;
+  dataLoaded: boolean;
+  homepage: string;
+}
+
+export declare function info(): IsnadyInfo;

package/index.js

@@ -6,7 +6,7 @@
  * database (used by both the Python and JavaScript packages) exists.
  */
 
-export const version = "0.1.7";
+export const version = "0.1.8";
 export const name = "isnady";
 
 /** Returns basic package information. */

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "isnady",
-  "version": "0.1.7",
+  "version": "0.1.8",
   "description": "Hadith search with full isnad chains, narrator biographies, jarh wa ta'dil verdicts and reliability scores",
   "type": "module",
   "main": "./index.js",