kintsugi 0.1.0__tar.gz → 0.2.0__tar.gz

{kintsugi-0.1.0 → kintsugi-0.2.0}/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## 0.2.0
+
+- Full language + variant registry (`kintsugi.languages`): every Google
+  Translate and Translated language, plus curated diglossia/dialect variants, as
+  BCP-47 codes with base language, script, region, register and listing engines
+  (304 entries). A language with no bundled frequency band is still a valid
+  attack target.
+- Frequency bands expanded to every FLORES+ language (233 configs) alongside
+  wordfreq, with script-aware tokenisation.
+- `kintsugi.locales` is now a view over the full registry; `diglossia_pairs()`
+  spans all standard-vs-vernacular contrasts (Singlish, Hinglish, Schweizerdeutsch,
+  Arabic dialects, Cantonese, ...).
+- `list-languages` command and `--lang-of` locale support on `frequency`.
+
 ## 0.1.0
 
 - Provider-agnostic targets and judges: Ollama, Hugging Face, your own callable,

{kintsugi-0.1.0 → kintsugi-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: kintsugi
-Version: 0.1.0
+Version: 0.2.0
 Summary: Red-teaming for machine translation: attacks, detectors, and judges from a large MT red-team project.
 Project-URL: Homepage, https://github.com/alvations/kintsugi
 Project-URL: Repository, https://github.com/alvations/kintsugi
@@ -126,7 +126,9 @@ common = rt.summary(rt.run(bands["frequent"]))
 rare   = rt.summary(rt.run(bands["rare"]))               # expect a higher catastrophic rate
 ```
 
-Bundled languages: `ar de en es fr hi id it ja kn ko nl pl pt ru sv tr vi yue zh zh-Hant`.
+Frequency bands are bundled for every FLORES+ language (233 configs) plus the
+`wordfreq` set and the Japanese gold dictionary; `frequency.languages()` lists
+them. Friendly aliases (`yue`, `kn`, `zh-Hant`, ...) resolve to the right band.
 
 ## Locale variants: region, script, register
 
@@ -147,11 +149,11 @@ rt = RedTeam(OllamaProvider("qwen3.5:0.8b"),
 ```
 
 Covered: region (`pt-BR`/`pt-PT`, `fr-CA`/`fr-CH`, `de-AT`/`de-CH`, `zh-CN`/
-`zh-TW`/`zh-HK`/`zh-MO`/`zh-SG`, ...), script (`zh-Hans`/`zh-Hant`, `sr-Cyrl`/
-`sr-Latn`, `ms-Latn`/`ms-Arab`, `hi-Deva`/`hi-Latn`, ...), and register
-(Singapore Standard English vs Singlish, Indian English vs Hinglish, Jamaican
-Standard English vs Patois, Swiss Standard German vs Schweizerdeutsch, MSA vs
-Egyptian/Moroccan Arabic, HK written Chinese vs spoken Cantonese).
+`zh-Hant-TW`/`zh-Hant-HK`/`zh-Hant-MO`/`zh-SG`, ...), script (`zh-Hans`/`zh-Hant`,
+`sr-Latn`, `hi-Latn`, ...), and register (Singapore Standard English vs Singlish,
+Indian English vs Hinglish, Jamaican Standard English vs Patois, Swiss Standard
+German vs Schweizerdeutsch, MSA vs Egyptian/Moroccan Arabic, HK written Chinese
+vs spoken Cantonese).
 
 ```bash
 kintsugi list-locales --register creole
@@ -159,9 +161,30 @@ kintsugi list-locales --script Hant
 kintsugi frequency --target ollama:qwen3.5:0.8b --lang-of zh-Hant --band rare --n 30
 ```
 
-The full Google Translate / Translated / Apple coverage catalogues are reference
-material in [`docs/LANGUAGE_COVERAGE.md`](docs/LANGUAGE_COVERAGE.md), kept out of
-the code on purpose (that list is never-ending).
+## Full language registry
+
+`kintsugi.languages` holds **every Google Translate and Translated language**
+plus the curated diglossia/dialect variants above (304 entries), each a BCP-47
+code with base language, script, region, register, listing engines, and a
+frequency band where one is bundled. A language with no band is still a valid
+attack target; frequency bands are bundled for all FLORES+ languages and the
+`wordfreq` set.
+
+```python
+from kintsugi import languages
+languages.by_engine("google")        # every Google Translate language/variant
+languages.by_engine("translated")    # every translated.com language/variant
+languages.with_band()                # those with a bundled frequency band
+languages.get("zh-Hant-HK")          # {'code':'zh-Hant-HK','script':'Hant','region':'HK',...}
+```
+
+```bash
+kintsugi list-languages --engine google
+kintsugi list-languages --with-band
+```
+
+The engine coverage is also written out, with the Apple `language-Script-REGION`
+convention, in [`docs/LANGUAGE_COVERAGE.md`](docs/LANGUAGE_COVERAGE.md).
 
 ## Command line
 

{kintsugi-0.1.0 → kintsugi-0.2.0}/README.md

@@ -93,7 +93,9 @@ common = rt.summary(rt.run(bands["frequent"]))
 rare   = rt.summary(rt.run(bands["rare"]))               # expect a higher catastrophic rate
 ```
 
-Bundled languages: `ar de en es fr hi id it ja kn ko nl pl pt ru sv tr vi yue zh zh-Hant`.
+Frequency bands are bundled for every FLORES+ language (233 configs) plus the
+`wordfreq` set and the Japanese gold dictionary; `frequency.languages()` lists
+them. Friendly aliases (`yue`, `kn`, `zh-Hant`, ...) resolve to the right band.
 
 ## Locale variants: region, script, register
 
@@ -114,11 +116,11 @@ rt = RedTeam(OllamaProvider("qwen3.5:0.8b"),
 ```
 
 Covered: region (`pt-BR`/`pt-PT`, `fr-CA`/`fr-CH`, `de-AT`/`de-CH`, `zh-CN`/
-`zh-TW`/`zh-HK`/`zh-MO`/`zh-SG`, ...), script (`zh-Hans`/`zh-Hant`, `sr-Cyrl`/
-`sr-Latn`, `ms-Latn`/`ms-Arab`, `hi-Deva`/`hi-Latn`, ...), and register
-(Singapore Standard English vs Singlish, Indian English vs Hinglish, Jamaican
-Standard English vs Patois, Swiss Standard German vs Schweizerdeutsch, MSA vs
-Egyptian/Moroccan Arabic, HK written Chinese vs spoken Cantonese).
+`zh-Hant-TW`/`zh-Hant-HK`/`zh-Hant-MO`/`zh-SG`, ...), script (`zh-Hans`/`zh-Hant`,
+`sr-Latn`, `hi-Latn`, ...), and register (Singapore Standard English vs Singlish,
+Indian English vs Hinglish, Jamaican Standard English vs Patois, Swiss Standard
+German vs Schweizerdeutsch, MSA vs Egyptian/Moroccan Arabic, HK written Chinese
+vs spoken Cantonese).
 
 ```bash
 kintsugi list-locales --register creole
@@ -126,9 +128,30 @@ kintsugi list-locales --script Hant
 kintsugi frequency --target ollama:qwen3.5:0.8b --lang-of zh-Hant --band rare --n 30
 ```
 
-The full Google Translate / Translated / Apple coverage catalogues are reference
-material in [`docs/LANGUAGE_COVERAGE.md`](docs/LANGUAGE_COVERAGE.md), kept out of
-the code on purpose (that list is never-ending).
+## Full language registry
+
+`kintsugi.languages` holds **every Google Translate and Translated language**
+plus the curated diglossia/dialect variants above (304 entries), each a BCP-47
+code with base language, script, region, register, listing engines, and a
+frequency band where one is bundled. A language with no band is still a valid
+attack target; frequency bands are bundled for all FLORES+ languages and the
+`wordfreq` set.
+
+```python
+from kintsugi import languages
+languages.by_engine("google")        # every Google Translate language/variant
+languages.by_engine("translated")    # every translated.com language/variant
+languages.with_band()                # those with a bundled frequency band
+languages.get("zh-Hant-HK")          # {'code':'zh-Hant-HK','script':'Hant','region':'HK',...}
+```
+
+```bash
+kintsugi list-languages --engine google
+kintsugi list-languages --with-band
+```
+
+The engine coverage is also written out, with the Apple `language-Script-REGION`
+convention, in [`docs/LANGUAGE_COVERAGE.md`](docs/LANGUAGE_COVERAGE.md).
 
 ## Command line
 

{kintsugi-0.1.0 → kintsugi-0.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "kintsugi"
-version = "0.1.0"
+version = "0.2.0"
 description = "Red-teaming for machine translation: attacks, detectors, and judges from a large MT red-team project."
 readme = "README.md"
 requires-python = ">=3.9"

{kintsugi-0.1.0 → kintsugi-0.2.0}/src/kintsugi/__init__.py

@@ -19,7 +19,7 @@ Quick start::
 """
 from __future__ import annotations
 
-from . import frequency, framings, locales, transforms
+from . import frequency, framings, languages, locales, transforms
 from .attacks import (
     Attack,
     categories,
@@ -39,13 +39,13 @@ from .providers import (
     Provider,
 )
 
-__version__ = "0.1.0"
+__version__ = "0.2.0"
 
 __all__ = [
     "RedTeam", "Finding", "LANG_PAIRS",
     "Attack", "load_attacks", "iter_attacks", "categories", "category_names", "observed_rates",
     "Provider", "OllamaProvider", "HuggingFaceProvider", "CallableProvider", "AnthropicProvider",
     "Judge", "Verdict", "prefilter", "detect_profanity",
-    "transforms", "framings", "frequency", "locales",
+    "transforms", "framings", "frequency", "locales", "languages",
     "__version__",
 ]