mypii 1.0.0__tar.gz

mypii-1.0.0/.gitignore

@@ -0,0 +1,23 @@
+.venv/
+__pycache__/
+.ruff_cache/
+.pytest_cache/
+hasil/*
+!hasil/cache/
+!hasil/cache/**
+!hasil/dev-masa.json
+# PELAN-C (ADR-006): rekod iterasi + carta + statistik dev dikomit; hasil/ramalan/ dan hasil/*ujian* KEKAL diabaikan
+!hasil/iterasi.tsv
+!hasil/kemajuan.svg
+!hasil/dev-statistik.json
+# PELAN-IAA P9 (ADR-015): skor persetujuan + kadar ralat label (kiraan sahaja, tiada teks)
+!hasil/iaa.json
+# PELAN-CLAUDE P3 (ADR-016): anggaran kos Claude dev (kiraan token + USD sahaja)
+!hasil/dev-kos-claude.json
+# PELAN-MODEL-LAIN P3/P4 (ADR-017): probe format eksplorasi (kiraan + jenis ralat sahaja)
+!hasil/dev-probe-llm.json
+# PELAN-V2 P12/P13: statistik + kuasa dev3 (kiraan sahaja) — sumber angka pindaan kuasa v2
+!hasil/dev3-statistik.json
+# 6 Nov 2026: salinan arkib larian ujian muktamad (v1 ujian/ujian2 + v2 ujian3) — keputusan + ramalan + cache, MANIFEST.sha256
+!hasil/arkib/
+!hasil/arkib/**

mypii-1.0.0/CHANGELOG.md

@@ -0,0 +1,42 @@
+# Changelog
+
+**BM** · Semua perubahan penting projek ini direkod di sini. Format ikut [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/)
+dan versi ikut [Semantic Versioning](https://semver.org/). Keputusan reka bentuk penuh ada dalam `docs/adr/`.
+**EN** · All notable changes to this project are recorded here. The format follows
+[Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/) and versions follow [Semantic Versioning](https://semver.org/).
+Full design decisions live in `docs/adr/`.
+
+## [1.0.0] - 2026-11-06
+
+Keluaran awam pertama; membawa pengecam v2. / First public release; ships recognizer v2.
+
+### Added
+
+- Three Presidio recognizers: `MY_NRIC` (MyKad number with birth date and official JPN place-of-birth code check), `MY_SSM`
+  (SSM registration number: new 12-digit format, old format, LLP) and `MY_PHONE` (Malaysian phone numbers), with context words for
+  Malay, English and Manglish text (ADR-001).
+- Recognizer v2: OCR normalisation inside numbers, line-broken numbers, bracketed Sabah and Sarawak area codes, the LLP rule
+  (year 2012 or later) and a small local context classifier (logistic regression, numpy only, sha256-checked artifact) (ADR-018).
+- `all_recognizers()`, an English alias of `semua_pengecam()`.
+- MyPII-Bench: 11,000 synthetic documents in seven splits, three of them sealed test splits locked by sha256 (`ujian`, `ujian2`,
+  `ujian3`), with a dataset card and a canary string (`data/README.md`).
+- Evaluation script with span-level precision, recall and F1, bootstrap intervals, Holm-corrected tests and an iteration log
+  (ADR-006, ADR-007, ADR-014).
+- Baselines scored by the same evaluation script: stock Presidio, `malaysia-pii-guard` and zero-shot LLMs, with LLM answers cached in the repo so
+  scores reproduce offline (ADR-011, ADR-016, ADR-017).
+- Pre-registration (`docs/PRA-DAFTAR.md`) and the archive of every test run with manifests (`hasil/arkib/`, `docs/PROVENANS.md`).
+- Human label review protocol and results on a 300-document sample: one human plus one LLM reviewer (ADR-015).
+- Comparison reports `docs/BANDINGAN.md` (dev, v1) and `docs/BANDINGAN-V2.md` (dev3 and `ujian3`, v2).
+
+### Changed
+
+- The wheel now carries `config/` and the classifier artifact inside the package, so `pip install mypii` works without a clone.
+- Evaluation and training tools moved from extras to dependency groups (`uv sync --group nilai --group garis --group latih`),
+  because PyPI rejects direct URL dependencies in package metadata.
+
+### Known limitations
+
+- All benchmark text is synthetic and labels are machine-generated; results on real text are not measured.
+- On the older-style `dev` split, v2 scores lower than v1 (F1 0.840 versus 0.888), mostly `MY_SSM` precision.
+- Phone recall is optimistic: synthetic phone labels only use number ranges that libphonenumber accepts (ADR-002).
+- The label review has one human reviewer, not two.

mypii-1.0.0/CITATION.cff

@@ -0,0 +1,32 @@
+cff-version: 1.2.0
+message: "If you use MyPII or MyPII-Bench, please cite it using the metadata in this file."
+type: software
+title: "MyPII: Malaysian PII recognizers for Presidio and the MyPII-Bench synthetic benchmark"
+abstract: >-
+  MyPII is a set of Malaysian PII recognizers for Presidio. It detects MyKad (NRIC) numbers,
+  SSM company registration numbers and Malaysian phone numbers in Malay, English and Manglish text.
+  It runs entirely on the local machine (rules, validation and a small logistic-regression context classifier; no external API), so text
+  never leaves the machine and every decision can be audited. The repository also ships MyPII-Bench,
+  a synthetic benchmark with machine-generated labels for evaluating recognizers like these.
+authors:
+  - family-names: Malib
+    given-names: Aiman
+    orcid: "https://orcid.org/0009-0000-1719-9800"
+    affiliation: "Sasa Teguh"
+version: 1.0.0
+date-released: "2026-11-06"
+license: MIT
+repository-code: "https://github.com/sasateguh/mypii"
+keywords:
+  - privacy
+  - nlp
+  - benchmark
+  - synthetic-data
+  - named-entity-recognition
+  - pii
+  - data-privacy
+  - anonymization
+  - pii-detection
+  - malaysia
+  - presidio
+  - phone-numbers

mypii-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Aiman Malib (Sasa Teguh)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mypii-1.0.0/PKG-INFO

@@ -0,0 +1,242 @@
+Metadata-Version: 2.5
+Name: mypii
+Version: 1.0.0
+Summary: Malaysian PII recognizers (MyKad, SSM, phone) for Presidio, plus the MyPII-Bench synthetic BM/EN/Manglish benchmark
+Project-URL: Homepage, https://github.com/sasateguh/mypii
+Project-URL: Source, https://github.com/sasateguh/mypii
+Project-URL: Issues, https://github.com/sasateguh/mypii/issues
+Project-URL: Changelog, https://github.com/sasateguh/mypii/blob/main/CHANGELOG.md
+Author: Aiman Malib
+License-Expression: MIT
+License-File: LICENSE
+Keywords: anonymization,data-privacy,malaysia,mykad,nlp,nric,pii,pii-detection,presidio
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Natural Language :: English
+Classifier: Natural Language :: Malay
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Security
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: <3.14,>=3.11
+Requires-Dist: numpy>=1.26
+Requires-Dist: phonenumbers>=8.13
+Requires-Dist: presidio-analyzer>=2.2.366
+Requires-Dist: pyyaml>=6
+Requires-Dist: regex>=2023.0
+Description-Content-Type: text/markdown
+
+# MyPII
+
+**BM** · MyPII mengesan nombor MyKad, nombor pendaftaran SSM dan nombor telefon Malaysia dalam teks BM, EN dan Manglish, sebagai
+pengecam tambahan untuk [Presidio](https://github.com/data-privacy-stack/presidio). Ia berjalan sepenuhnya pada mesin anda: peraturan,
+pengesahan (tarikh lahir dan kod tempat lahir JPN, format SSM, libphonenumber) dan satu pengelas konteks kecil (regresi logistik,
+numpy). Tiada API luar, jadi teks tidak keluar dari mesin. Repo ini juga membawa MyPII-Bench, penanda aras sintetik untuk menilai
+pengecam seperti ini, berserta pra-daftar hipotesis dan arkib setiap larian ujian.
+
+**EN** · MyPII detects Malaysian MyKad (NRIC) numbers, SSM company registration numbers and Malaysian phone numbers in Malay, English
+and Manglish text, as add-on recognizers for [Presidio](https://github.com/data-privacy-stack/presidio). It runs entirely on your
+machine: rules, validation (JPN birth date and place-of-birth codes, SSM format, libphonenumber) and one small context classifier
+(logistic regression, numpy). There is no external API, so text never leaves the machine. The repo also ships MyPII-Bench, a synthetic
+benchmark for evaluating recognizers like these, together with a pre-registration and the archive of every test run.
+
+## Pasang / Install
+
+```bash
+pip install mypii
+python -m spacy download en_core_web_sm
+```
+
+**BM** · Python 3.11 hingga 3.13. Model spaCy kecil itu ialah enjin NLP untuk Presidio dalam contoh di bawah. Sebelum pakej ada di
+PyPI, pasang terus dari GitHub: `pip install "mypii @ git+https://github.com/sasateguh/mypii@v1.0.0"`.
+**EN** · Python 3.11 to 3.13. The small spaCy model is the NLP engine Presidio uses in the example below. If the package is not on
+PyPI yet, install straight from GitHub: `pip install "mypii @ git+https://github.com/sasateguh/mypii@v1.0.0"`.
+
+## Contoh / Example
+
+```python
+from presidio_analyzer import AnalyzerEngine
+from presidio_analyzer.nlp_engine import NlpEngineProvider
+from mypii import semua_pengecam
+
+nlp = NlpEngineProvider(nlp_configuration={
+    "nlp_engine_name": "spacy",
+    "models": [{"lang_code": "en", "model_name": "en_core_web_sm"}],
+}).create_engine()
+engine = AnalyzerEngine(nlp_engine=nlp, supported_languages=["en"])
+for r in semua_pengecam():
+    engine.registry.add_recognizer(r)
+
+teks = "No IC saya 900101-08-1234, call 012-345 6789. Syarikat SSM 201901000005."
+hasil = engine.analyze(teks, language="en", entities=["MY_NRIC", "MY_SSM", "MY_PHONE"])
+for r in sorted(hasil, key=lambda r: r.start):
+    print(r.entity_type, teks[r.start:r.end], r.score)
+```
+
+Output:
+
+```text
+MY_NRIC 900101-08-1234 1.0
+MY_PHONE 012-345 6789 1.0
+MY_SSM 201901000005 1.0
+```
+
+**BM** · `all_recognizers()` ialah nama Inggeris untuk fungsi yang sama dengan `semua_pengecam()`. Konfigurasi `en_core_web_sm` itu
+sengaja: `AnalyzerEngine()` lalai memuat turun `en_core_web_lg` (382 MB). Tanpa `entities=`, pengecam asal Presidio turut
+memulangkan jenis lain (contohnya `UK_NHS`, `DATE_TIME`) atas nombor yang sama.
+**EN** · `all_recognizers()` is the English name for the same function as `semua_pengecam()`. The `en_core_web_sm` configuration is
+deliberate: a default `AnalyzerEngine()` downloads `en_core_web_lg` (382 MB). Without `entities=`, stock Presidio recognizers also
+return other types (for example `UK_NHS`, `DATE_TIME`) on the same numbers.
+
+## Entiti / Entities
+
+| Entiti | Contoh (sintetik) | Pengesahan / Validation |
+|---|---|---|
+| `MY_NRIC` | `900101-08-1234` | Tarikh lahir sah + kod tempat lahir JPN (86 kod, [JPN](https://www.jpn.gov.my/my/kod-negeri)) / valid birth date + official place-of-birth code |
+| `MY_SSM` | `201901000005` | Tahun + kod jenis entiti 01 hingga 06 (format SSM sejak 11 Okt 2019), format lama, LLP / year + entity type code, old format, LLP |
+| `MY_PHONE` | `+60 12-345 6789` | libphonenumber, rantau MY / region MY |
+
+## Keputusan / Results
+
+F1 padanan tepat mikro, semua entiti / exact-match micro F1, all entities. Versi 1.0.0 membawa pengecam **v2** / release 1.0.0 ships
+recognizer **v2**.
+
+| Split | Teks / Text | MyPII v1 | MyPII v2 (1.0.0) | Claude Haiku 5.5 | Claude Sonnet 5.5 | Presidio asal / stock |
+|---|---|---|---|---|---|---|
+| `dev` | pembangunan v1 / v1 development | 0.888 | 0.840 | 0.915 | 0.878 | 0.431 |
+| `ujian` † | templat simpanan v1 / held-out v1 templates | **0.993** | – | 0.965 | 0.982 | 0.526 |
+| `ujian2` † | templat Kimi K3 / Kimi K3 templates | **0.940** | – | 0.873 | 0.885 | 0.419 |
+| `dev3` | pembangunan v2, templat Nemotron / v2 development | 0.615 | 0.998 | 0.791 | – | 0.290 |
+| `ujian3` † | templat Claude Sonnet 5.5 / Claude Sonnet 5.5 templates | 0.587 | **0.980** | 0.871 | – | 0.325 |
+
+† Split ujian: dikunci sha256 sebelum dinilai, dibuka sekali sahaja, hipotesis didaftar dahulu dalam
+[`docs/PRA-DAFTAR.md`](docs/PRA-DAFTAR.md) / test split: locked by sha256 before scoring, opened once, hypotheses registered in
+advance. `dev` dan `dev3` dipakai semasa pembangunan, jadi angka MyPII di situ optimistik / `dev` and `dev3` were used during
+development, so MyPII's numbers there are optimistic. – = tidak dijalankan / not run.
+
+**BM** · Cara membaca jadual ini:
+- Hipotesis pra-daftar yang diuji semuanya disokong selepas pembetulan Holm: v1 mengatasi enam garis dasar atas `ujian2` (H1a hingga
+  H1f, utama) dan atas `ujian` (H2a hingga H2f, sekunder); v2 tidak kalah kepada Haiku (Δ +0.109) dan mengatasi v1 (Δ +0.393) atas
+  `ujian3` (H4a, H4b). Beza v1 dengan Sonnet atas `ujian` kecil (Δ +0.0115) walaupun signifikan. Hipotesis sekunder H3 (setiap
+  entiti) belum dilapor.
+- v2 menambah normalisasi OCR dalam nombor, nombor yang dipecah baris, kod kawasan Sabah dan Sarawak berkurung, peraturan LLP dan
+  pengelas konteks. Atas teks gaya baharu (`dev3`, `ujian3`) ia jauh lebih baik. Atas teks gaya lama (`dev`) ia lebih rendah daripada
+  v1 (0.840 lawan 0.888), terutamanya `MY_SSM` (ketepatan 0.51). Tiada versi yang menang di setiap gaya teks, jadi nilai atas data anda
+  sendiri dahulu.
+- v2 tidak dinilai atas `ujian` dan `ujian2` kerana kedua-dua split itu sudah dibuka untuk v1. Larian v2 di situ sekarang ialah
+  analisis selepas kejadian, bukan ujian.
+- Selang keyakinan, ketepatan dan dapatan semula setiap entiti, setiap bahasa dan setiap kategori ada dalam
+  [`docs/BANDINGAN-V2.md`](docs/BANDINGAN-V2.md) dan [`docs/BANDINGAN.md`](docs/BANDINGAN.md). Keputusan `ujian` dan `ujian2` ada
+  dalam [`docs/LOG-ITERASI.md`](docs/LOG-ITERASI.md) dan arkib `hasil/arkib/`.
+
+**EN** · How to read this table:
+- Every pre-registered hypothesis tested so far is supported after Holm correction: v1 beats six baselines on `ujian2` (H1a to H1f,
+  primary) and on `ujian` (H2a to H2f, secondary); v2 is not worse than Haiku (Δ +0.109) and beats v1 (Δ +0.393) on `ujian3` (H4a,
+  H4b). The v1 margin over Sonnet on `ujian` is small (Δ +0.0115) although significant. The secondary per-entity hypothesis H3 is not
+  reported yet.
+- v2 adds OCR normalisation inside numbers, line-broken numbers, bracketed Sabah and Sarawak area codes, the LLP rule and the context
+  classifier. On new-style text (`dev3`, `ujian3`) it is far better. On old-style text (`dev`) it is lower than v1 (0.840 versus
+  0.888), mostly `MY_SSM` (precision 0.51). No version wins on every text style, so evaluate on your own data first.
+- v2 was not scored on `ujian` and `ujian2` because both splits were already opened for v1. A v2 run there now would be a post-hoc
+  analysis, not a test.
+- Confidence intervals and per-entity, per-language and per-category precision and recall are in
+  [`docs/BANDINGAN-V2.md`](docs/BANDINGAN-V2.md) and [`docs/BANDINGAN.md`](docs/BANDINGAN.md). The `ujian` and `ujian2` results are in
+  [`docs/LOG-ITERASI.md`](docs/LOG-ITERASI.md) and the `hasil/arkib/` archive.
+
+## Had kajian / Limitations
+
+**BM**
+- **Sintetik sepenuhnya.** Teks ditulis LLM dengan slot bertanda, kemudian diisi nilai rekaan, jadi label diketahui tanpa anotasi
+  manual. Pembina yang sama menulis penjana data dan pengecam. Keputusan atas teks sebenar belum diukur dan mungkin lebih rendah.
+- **Keluarga Claude terlibat di tiga tempat.** Kod ditulis dengan ejen Claude, templat `ujian3` dijana Claude Sonnet, dan Claude Haiku
+  ialah pembanding H4a. Arah bias itu menentang MyPII bagi H4a.
+- **Semakan label = satu manusia + satu LLM, bukan dua manusia.** Atas sampel 300 dokumen, penyemak manusia mendapati 0.000 span
+  salah dan 0.067 [0.027, 0.113] dokumen ada PII yang tidak bertanda. κ tidak bermakna kerana hampir tiada varians.
+- **Dapatan semula telefon optimistik.** Label telefon dijana dalam julat sah libphonenumber, pustaka yang sama dipakai pengecam
+  ([ADR-002](docs/adr/002-penanda-aras-sintetik.md)).
+- **MyPII mengesan span, bukan menilai risiko.** Ia bukan alat anonimisasi lengkap dan bukan bukti pematuhan PDPA.
+
+**EN**
+- **Fully synthetic.** Text is written by LLMs with marked slots and filled with invented values, so labels are known without manual
+  annotation. The same builder wrote the data generator and the recognizer. Results on real text are not measured and may be lower.
+- **The Claude family appears in three places.** The code was written with Claude agents, the `ujian3` templates were generated by
+  Claude Sonnet, and Claude Haiku is the H4a comparator. That bias runs against MyPII for H4a.
+- **Label review = one human + one LLM, not two humans.** On a 300-document sample the human reviewer found 0.000 wrong spans and
+  0.067 [0.027, 0.113] of documents with unmarked PII. κ is not meaningful because there is almost no variance.
+- **Phone recall is optimistic.** Phone labels are drawn from libphonenumber's valid ranges, the same library the recognizer uses
+  ([ADR-002](docs/adr/002-penanda-aras-sintetik.md)).
+- **MyPII finds spans; it does not assess risk.** It is not a complete anonymisation tool and not evidence of PDPA compliance.
+
+## Penanda aras / Benchmark
+
+**BM** · MyPII-Bench (`data/`, CC-BY-4.0) ada lima split 1000 dokumen setiap satu (`dev`, `ujian`, `ujian2`, `dev3`, `ujian3`)
+serta split latihan. Kad set data: [`data/README.md`](data/README.md). Split ujian kini awam, jadi model yang dilatih selepas tarikh
+keluaran mungkin pernah melihatnya; rentetan canary dalam kad set data membantu pembina model menapisnya.
+**EN** · MyPII-Bench (`data/`, CC-BY-4.0) has five 1000-document splits (`dev`, `ujian`, `ujian2`, `dev3`, `ujian3`) plus training
+splits. Dataset card: [`data/README.md`](data/README.md). The test splits are now public, so models trained after the release date may
+have seen them; the canary string in the dataset card helps model builders filter them out.
+
+## Jana semula / Reproduce
+
+```bash
+git clone https://github.com/sasateguh/mypii.git && cd mypii
+uv sync --all-groups      # alat penilaian: model spaCy, malaysia-pii-guard, SDK Anthropic, scikit-learn
+make uji                  # ujian unit + kunci sha256 setiap split
+make bandingan-v2         # docs/BANDINGAN-V2.md dari arkib ujian3, tanpa rangkaian
+make bandingan            # docs/BANDINGAN.md dari cache LLM yang dikomit, tanpa rangkaian
+```
+
+**BM** · Jawapan LLM disimpan dalam cache yang dikomit, jadi skor garis dasar boleh dijana semula tanpa kunci API. Bukti meterai,
+arkib larian ujian dan suntingan dokumen diterangkan dalam [`docs/PROVENANS.md`](docs/PROVENANS.md).
+**EN** · LLM answers are stored in the committed cache, so baseline scores regenerate without API keys. Seal evidence, test-run
+archives and document redactions are explained in [`docs/PROVENANS.md`](docs/PROVENANS.md).
+
+## Dokumen / Documents
+
+- [`docs/PRA-DAFTAR.md`](docs/PRA-DAFTAR.md): hipotesis, split, ujian statistik dan pindaan / hypotheses, splits, statistical tests and amendments.
+- [`docs/BANDINGAN-V2.md`](docs/BANDINGAN-V2.md), [`docs/BANDINGAN.md`](docs/BANDINGAN.md): laporan bandingan / comparison reports.
+- [`docs/LOG-ITERASI.md`](docs/LOG-ITERASI.md): log setiap langkah dan ukuran / log of every step and measurement.
+- [`docs/ANALISIS-RALAT.md`](docs/ANALISIS-RALAT.md): analisis ralat v1 / v1 error analysis.
+- [`docs/adr/`](docs/adr/): rekod keputusan / decision records.
+- [`docs/IAA-SEMAKAN.md`](docs/IAA-SEMAKAN.md): cara semakan label manusia / how the human label review works.
+- [`docs/PENDEDAHAN-AI.md`](docs/PENDEDAHAN-AI.md): apa yang dibuat ejen AI dan apa yang dibuat manusia / what AI agents did and what humans did.
+- [`docs/PROVENANS.md`](docs/PROVENANS.md): meterai, arkib dan suntingan / seals, archives and redactions.
+
+## Keputusan dev v1 (sejarah) / v1 dev results (history)
+
+**BM** · Jadual ini dijana sebelum v2 wujud, jadi baris `mypii` ialah **MyPII v1**. Enam garis dasar ialah pengesahan (Holm k = 6);
+Mistral Small 3.1 dan GLM-5.3-Flash ialah eksplorasi (ADR-017), bukan dakwaan.
+**EN** · This table was generated before v2 existed, so the `mypii` row is **MyPII v1**. Six baselines are confirmatory (Holm k = 6);
+Mistral Small 3.1 and GLM-5.3-Flash are exploratory (ADR-017), not claims.
+
+<!-- jadual-dev:mula -->
+<!-- dijana `make readme` (scripts/jana_readme.py) dari hasil/dev-*.json; jangan sunting dengan tangan -->
+
+Split `dev`, 1000 dokumen / documents, padanan tepat / exact match, `semua|*`. Selang = bootstrap 95% (1000 ulangan / resamples, unit `dokumen`). Δ = F1 MyPII tolak F1 sistem / MyPII minus system (negatif = sistem lebih baik / negative = system is better). P/R: `docs/BANDINGAN.md`.
+
+| Sistem / System | F1 | Selang F1 / F1 interval | Δ F1 [selang / interval] |
+|---|---|---|---|
+| `mypii` | **0.8877** | [0.8701, 0.9037] | rujukan / reference |
+| `anthropic/claude-haiku-5-5` | **0.9149** | [0.8979, 0.9312] | -0.0272 [-0.0465, -0.0094] |
+| `anthropic/claude-sonnet-5-5` | **0.8782** | [0.8578, 0.8966] | +0.0095 [-0.0086, 0.0268] |
+| `cf/@cf/meta/llama-3.3-70b-instruct-fp8-fast` | **0.7060** | [0.6777, 0.7335] | +0.1817 [0.1591, 0.2028] |
+| `cf/@cf/mistralai/mistral-small-3.1-24b-instruct` (eksplorasi / exploratory) | **0.8513** | [0.8290, 0.8710] | +0.0364 [0.0165, 0.0583] |
+| `rv/deepseek-ai/DeepSeek-V4.1-Flash` | **0.7193** | [0.6899, 0.7483] | +0.1684 [0.1467, 0.1897] |
+| `rv/zai-org/GLM-5.3-Flash` (eksplorasi / exploratory) | **0.8889** | [0.8699, 0.9059] | -0.0012 [-0.0197, 0.0169] |
+| `pii-guard` | **0.7209** | [0.6977, 0.7434] | +0.1668 [0.1415, 0.1908] |
+| `presidio-asal` | **0.4309** | [0.4078, 0.4554] | +0.4568 [0.4299, 0.4813] |
+<!-- jadual-dev:tamat -->
+
+## Petik / Cite
+
+**BM** · Metadata petikan ada dalam [`CITATION.cff`](CITATION.cff) (GitHub memaparkannya sebagai "Cite this repository"). DOI Zenodo
+ditambah di sini selepas keluaran pertama.
+**EN** · Citation metadata lives in [`CITATION.cff`](CITATION.cff) (GitHub shows it as "Cite this repository"). The Zenodo DOI is
+added here after the first release.
+
+## Lesen / License
+
+Kod / code: MIT ([`LICENSE`](LICENSE)). Set data / dataset (`data/`): CC-BY-4.0. © 2026 Aiman Malib (Sasa Teguh).

mypii-1.0.0/README.md

@@ -0,0 +1,211 @@
+# MyPII
+
+**BM** · MyPII mengesan nombor MyKad, nombor pendaftaran SSM dan nombor telefon Malaysia dalam teks BM, EN dan Manglish, sebagai
+pengecam tambahan untuk [Presidio](https://github.com/data-privacy-stack/presidio). Ia berjalan sepenuhnya pada mesin anda: peraturan,
+pengesahan (tarikh lahir dan kod tempat lahir JPN, format SSM, libphonenumber) dan satu pengelas konteks kecil (regresi logistik,
+numpy). Tiada API luar, jadi teks tidak keluar dari mesin. Repo ini juga membawa MyPII-Bench, penanda aras sintetik untuk menilai
+pengecam seperti ini, berserta pra-daftar hipotesis dan arkib setiap larian ujian.
+
+**EN** · MyPII detects Malaysian MyKad (NRIC) numbers, SSM company registration numbers and Malaysian phone numbers in Malay, English
+and Manglish text, as add-on recognizers for [Presidio](https://github.com/data-privacy-stack/presidio). It runs entirely on your
+machine: rules, validation (JPN birth date and place-of-birth codes, SSM format, libphonenumber) and one small context classifier
+(logistic regression, numpy). There is no external API, so text never leaves the machine. The repo also ships MyPII-Bench, a synthetic
+benchmark for evaluating recognizers like these, together with a pre-registration and the archive of every test run.
+
+## Pasang / Install
+
+```bash
+pip install mypii
+python -m spacy download en_core_web_sm
+```
+
+**BM** · Python 3.11 hingga 3.13. Model spaCy kecil itu ialah enjin NLP untuk Presidio dalam contoh di bawah. Sebelum pakej ada di
+PyPI, pasang terus dari GitHub: `pip install "mypii @ git+https://github.com/sasateguh/mypii@v1.0.0"`.
+**EN** · Python 3.11 to 3.13. The small spaCy model is the NLP engine Presidio uses in the example below. If the package is not on
+PyPI yet, install straight from GitHub: `pip install "mypii @ git+https://github.com/sasateguh/mypii@v1.0.0"`.
+
+## Contoh / Example
+
+```python
+from presidio_analyzer import AnalyzerEngine
+from presidio_analyzer.nlp_engine import NlpEngineProvider
+from mypii import semua_pengecam
+
+nlp = NlpEngineProvider(nlp_configuration={
+    "nlp_engine_name": "spacy",
+    "models": [{"lang_code": "en", "model_name": "en_core_web_sm"}],
+}).create_engine()
+engine = AnalyzerEngine(nlp_engine=nlp, supported_languages=["en"])
+for r in semua_pengecam():
+    engine.registry.add_recognizer(r)
+
+teks = "No IC saya 900101-08-1234, call 012-345 6789. Syarikat SSM 201901000005."
+hasil = engine.analyze(teks, language="en", entities=["MY_NRIC", "MY_SSM", "MY_PHONE"])
+for r in sorted(hasil, key=lambda r: r.start):
+    print(r.entity_type, teks[r.start:r.end], r.score)
+```
+
+Output:
+
+```text
+MY_NRIC 900101-08-1234 1.0
+MY_PHONE 012-345 6789 1.0
+MY_SSM 201901000005 1.0
+```
+
+**BM** · `all_recognizers()` ialah nama Inggeris untuk fungsi yang sama dengan `semua_pengecam()`. Konfigurasi `en_core_web_sm` itu
+sengaja: `AnalyzerEngine()` lalai memuat turun `en_core_web_lg` (382 MB). Tanpa `entities=`, pengecam asal Presidio turut
+memulangkan jenis lain (contohnya `UK_NHS`, `DATE_TIME`) atas nombor yang sama.
+**EN** · `all_recognizers()` is the English name for the same function as `semua_pengecam()`. The `en_core_web_sm` configuration is
+deliberate: a default `AnalyzerEngine()` downloads `en_core_web_lg` (382 MB). Without `entities=`, stock Presidio recognizers also
+return other types (for example `UK_NHS`, `DATE_TIME`) on the same numbers.
+
+## Entiti / Entities
+
+| Entiti | Contoh (sintetik) | Pengesahan / Validation |
+|---|---|---|
+| `MY_NRIC` | `900101-08-1234` | Tarikh lahir sah + kod tempat lahir JPN (86 kod, [JPN](https://www.jpn.gov.my/my/kod-negeri)) / valid birth date + official place-of-birth code |
+| `MY_SSM` | `201901000005` | Tahun + kod jenis entiti 01 hingga 06 (format SSM sejak 11 Okt 2019), format lama, LLP / year + entity type code, old format, LLP |
+| `MY_PHONE` | `+60 12-345 6789` | libphonenumber, rantau MY / region MY |
+
+## Keputusan / Results
+
+F1 padanan tepat mikro, semua entiti / exact-match micro F1, all entities. Versi 1.0.0 membawa pengecam **v2** / release 1.0.0 ships
+recognizer **v2**.
+
+| Split | Teks / Text | MyPII v1 | MyPII v2 (1.0.0) | Claude Haiku 5.5 | Claude Sonnet 5.5 | Presidio asal / stock |
+|---|---|---|---|---|---|---|
+| `dev` | pembangunan v1 / v1 development | 0.888 | 0.840 | 0.915 | 0.878 | 0.431 |
+| `ujian` † | templat simpanan v1 / held-out v1 templates | **0.993** | – | 0.965 | 0.982 | 0.526 |
+| `ujian2` † | templat Kimi K3 / Kimi K3 templates | **0.940** | – | 0.873 | 0.885 | 0.419 |
+| `dev3` | pembangunan v2, templat Nemotron / v2 development | 0.615 | 0.998 | 0.791 | – | 0.290 |
+| `ujian3` † | templat Claude Sonnet 5.5 / Claude Sonnet 5.5 templates | 0.587 | **0.980** | 0.871 | – | 0.325 |
+
+† Split ujian: dikunci sha256 sebelum dinilai, dibuka sekali sahaja, hipotesis didaftar dahulu dalam
+[`docs/PRA-DAFTAR.md`](docs/PRA-DAFTAR.md) / test split: locked by sha256 before scoring, opened once, hypotheses registered in
+advance. `dev` dan `dev3` dipakai semasa pembangunan, jadi angka MyPII di situ optimistik / `dev` and `dev3` were used during
+development, so MyPII's numbers there are optimistic. – = tidak dijalankan / not run.
+
+**BM** · Cara membaca jadual ini:
+- Hipotesis pra-daftar yang diuji semuanya disokong selepas pembetulan Holm: v1 mengatasi enam garis dasar atas `ujian2` (H1a hingga
+  H1f, utama) dan atas `ujian` (H2a hingga H2f, sekunder); v2 tidak kalah kepada Haiku (Δ +0.109) dan mengatasi v1 (Δ +0.393) atas
+  `ujian3` (H4a, H4b). Beza v1 dengan Sonnet atas `ujian` kecil (Δ +0.0115) walaupun signifikan. Hipotesis sekunder H3 (setiap
+  entiti) belum dilapor.
+- v2 menambah normalisasi OCR dalam nombor, nombor yang dipecah baris, kod kawasan Sabah dan Sarawak berkurung, peraturan LLP dan
+  pengelas konteks. Atas teks gaya baharu (`dev3`, `ujian3`) ia jauh lebih baik. Atas teks gaya lama (`dev`) ia lebih rendah daripada
+  v1 (0.840 lawan 0.888), terutamanya `MY_SSM` (ketepatan 0.51). Tiada versi yang menang di setiap gaya teks, jadi nilai atas data anda
+  sendiri dahulu.
+- v2 tidak dinilai atas `ujian` dan `ujian2` kerana kedua-dua split itu sudah dibuka untuk v1. Larian v2 di situ sekarang ialah
+  analisis selepas kejadian, bukan ujian.
+- Selang keyakinan, ketepatan dan dapatan semula setiap entiti, setiap bahasa dan setiap kategori ada dalam
+  [`docs/BANDINGAN-V2.md`](docs/BANDINGAN-V2.md) dan [`docs/BANDINGAN.md`](docs/BANDINGAN.md). Keputusan `ujian` dan `ujian2` ada
+  dalam [`docs/LOG-ITERASI.md`](docs/LOG-ITERASI.md) dan arkib `hasil/arkib/`.
+
+**EN** · How to read this table:
+- Every pre-registered hypothesis tested so far is supported after Holm correction: v1 beats six baselines on `ujian2` (H1a to H1f,
+  primary) and on `ujian` (H2a to H2f, secondary); v2 is not worse than Haiku (Δ +0.109) and beats v1 (Δ +0.393) on `ujian3` (H4a,
+  H4b). The v1 margin over Sonnet on `ujian` is small (Δ +0.0115) although significant. The secondary per-entity hypothesis H3 is not
+  reported yet.
+- v2 adds OCR normalisation inside numbers, line-broken numbers, bracketed Sabah and Sarawak area codes, the LLP rule and the context
+  classifier. On new-style text (`dev3`, `ujian3`) it is far better. On old-style text (`dev`) it is lower than v1 (0.840 versus
+  0.888), mostly `MY_SSM` (precision 0.51). No version wins on every text style, so evaluate on your own data first.
+- v2 was not scored on `ujian` and `ujian2` because both splits were already opened for v1. A v2 run there now would be a post-hoc
+  analysis, not a test.
+- Confidence intervals and per-entity, per-language and per-category precision and recall are in
+  [`docs/BANDINGAN-V2.md`](docs/BANDINGAN-V2.md) and [`docs/BANDINGAN.md`](docs/BANDINGAN.md). The `ujian` and `ujian2` results are in
+  [`docs/LOG-ITERASI.md`](docs/LOG-ITERASI.md) and the `hasil/arkib/` archive.
+
+## Had kajian / Limitations
+
+**BM**
+- **Sintetik sepenuhnya.** Teks ditulis LLM dengan slot bertanda, kemudian diisi nilai rekaan, jadi label diketahui tanpa anotasi
+  manual. Pembina yang sama menulis penjana data dan pengecam. Keputusan atas teks sebenar belum diukur dan mungkin lebih rendah.
+- **Keluarga Claude terlibat di tiga tempat.** Kod ditulis dengan ejen Claude, templat `ujian3` dijana Claude Sonnet, dan Claude Haiku
+  ialah pembanding H4a. Arah bias itu menentang MyPII bagi H4a.
+- **Semakan label = satu manusia + satu LLM, bukan dua manusia.** Atas sampel 300 dokumen, penyemak manusia mendapati 0.000 span
+  salah dan 0.067 [0.027, 0.113] dokumen ada PII yang tidak bertanda. κ tidak bermakna kerana hampir tiada varians.
+- **Dapatan semula telefon optimistik.** Label telefon dijana dalam julat sah libphonenumber, pustaka yang sama dipakai pengecam
+  ([ADR-002](docs/adr/002-penanda-aras-sintetik.md)).
+- **MyPII mengesan span, bukan menilai risiko.** Ia bukan alat anonimisasi lengkap dan bukan bukti pematuhan PDPA.
+
+**EN**
+- **Fully synthetic.** Text is written by LLMs with marked slots and filled with invented values, so labels are known without manual
+  annotation. The same builder wrote the data generator and the recognizer. Results on real text are not measured and may be lower.
+- **The Claude family appears in three places.** The code was written with Claude agents, the `ujian3` templates were generated by
+  Claude Sonnet, and Claude Haiku is the H4a comparator. That bias runs against MyPII for H4a.
+- **Label review = one human + one LLM, not two humans.** On a 300-document sample the human reviewer found 0.000 wrong spans and
+  0.067 [0.027, 0.113] of documents with unmarked PII. κ is not meaningful because there is almost no variance.
+- **Phone recall is optimistic.** Phone labels are drawn from libphonenumber's valid ranges, the same library the recognizer uses
+  ([ADR-002](docs/adr/002-penanda-aras-sintetik.md)).
+- **MyPII finds spans; it does not assess risk.** It is not a complete anonymisation tool and not evidence of PDPA compliance.
+
+## Penanda aras / Benchmark
+
+**BM** · MyPII-Bench (`data/`, CC-BY-4.0) ada lima split 1000 dokumen setiap satu (`dev`, `ujian`, `ujian2`, `dev3`, `ujian3`)
+serta split latihan. Kad set data: [`data/README.md`](data/README.md). Split ujian kini awam, jadi model yang dilatih selepas tarikh
+keluaran mungkin pernah melihatnya; rentetan canary dalam kad set data membantu pembina model menapisnya.
+**EN** · MyPII-Bench (`data/`, CC-BY-4.0) has five 1000-document splits (`dev`, `ujian`, `ujian2`, `dev3`, `ujian3`) plus training
+splits. Dataset card: [`data/README.md`](data/README.md). The test splits are now public, so models trained after the release date may
+have seen them; the canary string in the dataset card helps model builders filter them out.
+
+## Jana semula / Reproduce
+
+```bash
+git clone https://github.com/sasateguh/mypii.git && cd mypii
+uv sync --all-groups      # alat penilaian: model spaCy, malaysia-pii-guard, SDK Anthropic, scikit-learn
+make uji                  # ujian unit + kunci sha256 setiap split
+make bandingan-v2         # docs/BANDINGAN-V2.md dari arkib ujian3, tanpa rangkaian
+make bandingan            # docs/BANDINGAN.md dari cache LLM yang dikomit, tanpa rangkaian
+```
+
+**BM** · Jawapan LLM disimpan dalam cache yang dikomit, jadi skor garis dasar boleh dijana semula tanpa kunci API. Bukti meterai,
+arkib larian ujian dan suntingan dokumen diterangkan dalam [`docs/PROVENANS.md`](docs/PROVENANS.md).
+**EN** · LLM answers are stored in the committed cache, so baseline scores regenerate without API keys. Seal evidence, test-run
+archives and document redactions are explained in [`docs/PROVENANS.md`](docs/PROVENANS.md).
+
+## Dokumen / Documents
+
+- [`docs/PRA-DAFTAR.md`](docs/PRA-DAFTAR.md): hipotesis, split, ujian statistik dan pindaan / hypotheses, splits, statistical tests and amendments.
+- [`docs/BANDINGAN-V2.md`](docs/BANDINGAN-V2.md), [`docs/BANDINGAN.md`](docs/BANDINGAN.md): laporan bandingan / comparison reports.
+- [`docs/LOG-ITERASI.md`](docs/LOG-ITERASI.md): log setiap langkah dan ukuran / log of every step and measurement.
+- [`docs/ANALISIS-RALAT.md`](docs/ANALISIS-RALAT.md): analisis ralat v1 / v1 error analysis.
+- [`docs/adr/`](docs/adr/): rekod keputusan / decision records.
+- [`docs/IAA-SEMAKAN.md`](docs/IAA-SEMAKAN.md): cara semakan label manusia / how the human label review works.
+- [`docs/PENDEDAHAN-AI.md`](docs/PENDEDAHAN-AI.md): apa yang dibuat ejen AI dan apa yang dibuat manusia / what AI agents did and what humans did.
+- [`docs/PROVENANS.md`](docs/PROVENANS.md): meterai, arkib dan suntingan / seals, archives and redactions.
+
+## Keputusan dev v1 (sejarah) / v1 dev results (history)
+
+**BM** · Jadual ini dijana sebelum v2 wujud, jadi baris `mypii` ialah **MyPII v1**. Enam garis dasar ialah pengesahan (Holm k = 6);
+Mistral Small 3.1 dan GLM-5.3-Flash ialah eksplorasi (ADR-017), bukan dakwaan.
+**EN** · This table was generated before v2 existed, so the `mypii` row is **MyPII v1**. Six baselines are confirmatory (Holm k = 6);
+Mistral Small 3.1 and GLM-5.3-Flash are exploratory (ADR-017), not claims.
+
+<!-- jadual-dev:mula -->
+<!-- dijana `make readme` (scripts/jana_readme.py) dari hasil/dev-*.json; jangan sunting dengan tangan -->
+
+Split `dev`, 1000 dokumen / documents, padanan tepat / exact match, `semua|*`. Selang = bootstrap 95% (1000 ulangan / resamples, unit `dokumen`). Δ = F1 MyPII tolak F1 sistem / MyPII minus system (negatif = sistem lebih baik / negative = system is better). P/R: `docs/BANDINGAN.md`.
+
+| Sistem / System | F1 | Selang F1 / F1 interval | Δ F1 [selang / interval] |
+|---|---|---|---|
+| `mypii` | **0.8877** | [0.8701, 0.9037] | rujukan / reference |
+| `anthropic/claude-haiku-5-5` | **0.9149** | [0.8979, 0.9312] | -0.0272 [-0.0465, -0.0094] |
+| `anthropic/claude-sonnet-5-5` | **0.8782** | [0.8578, 0.8966] | +0.0095 [-0.0086, 0.0268] |
+| `cf/@cf/meta/llama-3.3-70b-instruct-fp8-fast` | **0.7060** | [0.6777, 0.7335] | +0.1817 [0.1591, 0.2028] |
+| `cf/@cf/mistralai/mistral-small-3.1-24b-instruct` (eksplorasi / exploratory) | **0.8513** | [0.8290, 0.8710] | +0.0364 [0.0165, 0.0583] |
+| `rv/deepseek-ai/DeepSeek-V4.1-Flash` | **0.7193** | [0.6899, 0.7483] | +0.1684 [0.1467, 0.1897] |
+| `rv/zai-org/GLM-5.3-Flash` (eksplorasi / exploratory) | **0.8889** | [0.8699, 0.9059] | -0.0012 [-0.0197, 0.0169] |
+| `pii-guard` | **0.7209** | [0.6977, 0.7434] | +0.1668 [0.1415, 0.1908] |
+| `presidio-asal` | **0.4309** | [0.4078, 0.4554] | +0.4568 [0.4299, 0.4813] |
+<!-- jadual-dev:tamat -->
+
+## Petik / Cite
+
+**BM** · Metadata petikan ada dalam [`CITATION.cff`](CITATION.cff) (GitHub memaparkannya sebagai "Cite this repository"). DOI Zenodo
+ditambah di sini selepas keluaran pertama.
+**EN** · Citation metadata lives in [`CITATION.cff`](CITATION.cff) (GitHub shows it as "Cite this repository"). The Zenodo DOI is
+added here after the first release.
+
+## Lesen / License
+
+Kod / code: MIT ([`LICENSE`](LICENSE)). Set data / dataset (`data/`): CC-BY-4.0. © 2026 Aiman Malib (Sasa Teguh).