champollion 0.3.3 → 0.3.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -11
- package/lib/cards/reader.js +17 -3
- package/lib/command-help.js +3 -3
- package/lib/commands/register-corpus.js +21 -9
- package/lib/commands/tm.js +1 -1
- package/lib/corpus-registration.mjs +3 -3
- package/lib/scripts.js +187 -105
- package/lib/seal.mjs +2 -2
- package/lib/sealed-qualifier.mjs +1 -1
- package/package.json +3 -2
- package/shared/DATA-SOVEREIGNTY.md +19 -20
- package/shared/LANGUAGE-CARD-FIELDS.md +1 -1
- package/shared/cards-fallback.json +1 -1
- package/shared/catalogue/card-config.json +1 -1
- package/shared/docent/corpus.json +1156 -750
- package/shared/docent/faq.en.json +14 -16
- package/shared/docent/system-prompt.md +17 -19
- package/shared/explainers/tc-features.json +15 -15
- package/shared/human-services.json +1 -1
- package/shared/schemas/corpora-card.schema.json +12 -8
- package/shared/schemas/human-services.schema.json +2 -2
- package/shared/schemas/language-card.schema.json +1 -1
- package/shared/schemas/method-card.schema.json +1 -1
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "champollion",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.4",
|
|
4
4
|
"description": "Research-grade translation engine for i18n projects. Pluggable methods, per-pair quality tiers, and deterministic script converters. Supports JSON (next-intl, i18next), TOML, and YAML (Hugo).",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -88,6 +88,7 @@
|
|
|
88
88
|
"language-subtag-registry": "^0.4.2"
|
|
89
89
|
},
|
|
90
90
|
"dependencies": {
|
|
91
|
-
"@translated/lara": "^1.12.1"
|
|
91
|
+
"@translated/lara": "^1.12.1",
|
|
92
|
+
"cree-sro-syllabics": "2021.6.8"
|
|
92
93
|
}
|
|
93
94
|
}
|
|
@@ -39,19 +39,19 @@ Each sovereignty-related field maps to one or more external standards. The table
|
|
|
39
39
|
|
|
40
40
|
| Field / Concept | Primary Standard | Standard URL | Relationship |
|
|
41
41
|
|---|---|---|---|
|
|
42
|
-
| `sovereignty.frameworks` |
|
|
43
|
-
| `sovereignty.governanceOrg` |
|
|
44
|
-
| `sovereignty.custodian` |
|
|
45
|
-
| `sovereignty.consentModel` |
|
|
42
|
+
| `sovereignty.frameworks` | CARE, Te Mana Raraunga, FAIR, IEEE 2890, community ownership-and-control assertions | See individual URLs below | Records which frameworks have been **explicitly invoked** by the governing body |
|
|
43
|
+
| `sovereignty.governanceOrg` | Community ownership | — | Who owns the data |
|
|
44
|
+
| `sovereignty.custodian` | Community possession | — | Who physically holds the data |
|
|
45
|
+
| `sovereignty.consentModel` | Community control, CARE Authority | [https://www.gida-global.org/care](https://www.gida-global.org/care) | Who controls access — maps to CARE's Authority to Control principle |
|
|
46
46
|
| `sovereignty.tkLabels[]` | Local Contexts TK/BC Labels | [https://localcontexts.org/](https://localcontexts.org/) | Community-applied labels from the Local Contexts Hub |
|
|
47
47
|
| `usageRestrictions.training` | DUO (Data Use Ontology), ODRL | [DUO](https://github.com/EBISPOT/DUO), [ODRL](https://www.w3.org/TR/odrl-model/) | `prohibited-by-license` maps to DUO:0000004 (no general methods research) and ODRL `odrl:prohibition`. `prohibited-by-community` maps to a governance assertion (no DUO equivalent — DUO is license-derived) |
|
|
48
48
|
| `usageRestrictions.commercialUse` | DUO:0000018 (not for profit), ODRL | [DUO](https://github.com/EBISPOT/DUO), [ODRL](https://www.w3.org/TR/odrl-model/) | `prohibited-by-license` maps to DUO:0000018 and ODRL `odrl:prohibition` with `odrl:commercialize` action |
|
|
49
49
|
| `usageRestrictions.redistribution` | ODRL `odrl:distribute` | [ODRL](https://www.w3.org/TR/odrl-model/) | Maps to an ODRL permission/prohibition on the `odrl:distribute` action |
|
|
50
50
|
| `usageRestrictions.communityNotes` | CARE Ethics | [https://www.gida-global.org/care](https://www.gida-global.org/care) | Maps to CARE's Ethics principle — ensuring legitimate concerns are surfaced |
|
|
51
51
|
| `doNotTrain` | IEEE 2890 | No public URL (IEEE standard, paywall) | Methodological constraint for evaluation data; IEEE 2890 §7.4 recommends flagging datasets not intended for model training |
|
|
52
|
-
| `stewardship.authorizationModel` |
|
|
53
|
-
| `submission.transfer.*` |
|
|
54
|
-
| `submission.admissibility.selfHostable` |
|
|
52
|
+
| `stewardship.authorizationModel` | Community control | — | Community controls how evaluation access is granted |
|
|
53
|
+
| `submission.transfer.*` | Community ownership + possession, CARE Collective Benefit | [CARE](https://www.gida-global.org/care) | Transfer provisions mean the community owns and possesses the method. Revenue model maps to CARE Collective Benefit |
|
|
54
|
+
| `submission.admissibility.selfHostable` | Community possession | — | If the community can't run it independently, they don't possess it |
|
|
55
55
|
|
|
56
56
|
### Standard References
|
|
57
57
|
|
|
@@ -61,7 +61,6 @@ Each sovereignty-related field maps to one or more external standards. The table
|
|
|
61
61
|
| DUO | Data Use Ontology | [https://github.com/EBISPOT/DUO](https://github.com/EBISPOT/DUO) |
|
|
62
62
|
| Local Contexts | TK and BC Labels | [https://localcontexts.org/](https://localcontexts.org/) |
|
|
63
63
|
| CARE | Collective Benefit, Authority to Control, Responsibility, Ethics | [https://www.gida-global.org/care](https://www.gida-global.org/care) |
|
|
64
|
-
| OCAP® | Ownership, Control, Access, Possession | [https://fnigc.ca/ocap-training/](https://fnigc.ca/ocap-training/) |
|
|
65
64
|
| IEEE 2890 | IEEE Standard for Recommended Practice for Provenance of Datasets | IEEE (paywall; no stable public URL) |
|
|
66
65
|
| Te Mana Raraunga | Māori Data Sovereignty Network | [https://www.temanararaunga.maori.nz/](https://www.temanararaunga.maori.nz/) |
|
|
67
66
|
| FAIR | Findable, Accessible, Interoperable, Reusable | [https://www.go-fair.org/fair-principles/](https://www.go-fair.org/fair-principles/) |
|
|
@@ -108,7 +107,7 @@ The `sovereignty` object records governance structures that actually exist for t
|
|
|
108
107
|
| Field | Type | Allowed Values | Description | When to use `null` |
|
|
109
108
|
|---|---|---|---|---|
|
|
110
109
|
| `sovereignty` | `object \| null` | — | Container for governance metadata. | No governance body has been identified. No frameworks have been invoked. No labels have been applied. |
|
|
111
|
-
| `.frameworks` | `array` of `string` | `"
|
|
110
|
+
| `.frameworks` | `array` of `string` | `"community-ownership-control"`, `"CARE"`, `"Te-Mana-Raraunga"`, `"FAIR"`, `"IEEE-2890"` | Data sovereignty frameworks the data creators or governing body have **explicitly invoked**. Only list frameworks where there is documented evidence of adoption. | *(Field is an array — use `[]` if sovereignty object exists but no frameworks have been invoked. But if no sovereignty object at all, the whole object is null.)* |
|
|
112
111
|
| `.governanceOrg` | `string \| null` | Free text | Organization or body with governance authority. For Indigenous-governed corpora, this is the language trust, tribal council, or delegated body. | No governance body has been identified or established. |
|
|
113
112
|
| `.custodian` | `string \| null` | Free text | Who physically holds the data. Distinct from `governanceOrg` — a university may be custodian while a community is the governance authority. | Custodian is the same as `source.publisher` (no need to repeat). |
|
|
114
113
|
| `.consentModel` | `string \| null` | `"per-submission"`, `"blanket"`, `"open-access"` | How consent is structured for use of this data in method evaluation. | No consent model has been established. |
|
|
@@ -123,7 +122,7 @@ The `sovereignty` object records governance structures that actually exist for t
|
|
|
123
122
|
```json
|
|
124
123
|
{
|
|
125
124
|
"sovereignty": {
|
|
126
|
-
"frameworks": ["
|
|
125
|
+
"frameworks": ["community-ownership-control", "CARE"],
|
|
127
126
|
"governanceOrg": "Plains Cree Language Trust (hypothetical)",
|
|
128
127
|
"custodian": "University of Alberta — ALTLab",
|
|
129
128
|
"consentModel": "per-submission",
|
|
@@ -264,13 +263,13 @@ The `submission` field (§7) defines what happens when a method is accepted: wha
|
|
|
264
263
|
|
|
265
264
|
## 7. Admissibility Rules
|
|
266
265
|
|
|
267
|
-
The `submission.admissibility` fields define what methods are eligible for prize evaluation. These constraints are technical, not arbitrary — they flow directly from
|
|
266
|
+
The `submission.admissibility` fields define what methods are eligible for prize evaluation. These constraints are technical, not arbitrary — they flow directly from the community-possession requirement (the community must be able to hold and run the method itself) and the air-gapped sandbox architecture.
|
|
268
267
|
|
|
269
268
|
### Why Coached API Calls Are Inadmissible
|
|
270
269
|
|
|
271
270
|
A "coached API call" is a method that wraps a third-party API (e.g., GPT-4, Google Translate) with prompt engineering. These are inadmissible for prize evaluation because:
|
|
272
271
|
|
|
273
|
-
1. **
|
|
272
|
+
1. **Community-possession violation.** The community cannot possess a method that depends on someone else's API. When the API key expires, the method dies. The community doesn't control the model, its weights, or its availability.
|
|
274
273
|
|
|
275
274
|
2. **Non-reproducible.** API providers change models, pricing, and availability without notice. A score measured today may not be reproducible tomorrow.
|
|
276
275
|
|
|
@@ -286,9 +285,9 @@ A "coached API call" is a method that wraps a third-party API (e.g., GPT-4, Goog
|
|
|
286
285
|
- No license-restricted runtime dependencies that the community cannot independently obtain
|
|
287
286
|
- The community can inspect, modify, and deploy the method on their own hardware
|
|
288
287
|
|
|
289
|
-
###
|
|
288
|
+
### Community-Possession Logic
|
|
290
289
|
|
|
291
|
-
The
|
|
290
|
+
The community-possession principle states that the community must physically hold the data and tools. In the context of prize evaluation:
|
|
292
291
|
|
|
293
292
|
| What | Possession requirement |
|
|
294
293
|
|---|---|
|
|
@@ -311,15 +310,15 @@ The evaluation sandbox operates with no network access:
|
|
|
311
310
|
|---|---|---|
|
|
312
311
|
| `selfHostable` | `boolean` | Method must run on community infrastructure without any third-party dependencies. |
|
|
313
312
|
| `inadmissible` | `array` of `string` | Explicitly inadmissible method types. e.g., `"coached-api-calls"`, `"proprietary-api-wrappers"`, `"external-api-dependencies"`. |
|
|
314
|
-
| `notes` | `string \| null` | Rationale for admissibility constraints. Should reference
|
|
313
|
+
| `notes` | `string \| null` | Rationale for admissibility constraints. Should reference community possession and the air-gapped sandbox architecture. |
|
|
315
314
|
|
|
316
315
|
### `submission.transfer` Fields — What the Community Gets
|
|
317
316
|
|
|
318
|
-
| Field | Type | Description |
|
|
317
|
+
| Field | Type | Description | Sovereignty/CARE mapping |
|
|
319
318
|
|---|---|---|---|
|
|
320
|
-
| `sourceCode` | `boolean` | Source code ownership transfers to governance org |
|
|
321
|
-
| `modelWeights` | `boolean` | Trained model weights transfer to governance org |
|
|
322
|
-
| `deploymentRights` | `boolean` | Exclusive deployment rights transfer to governance org |
|
|
319
|
+
| `sourceCode` | `boolean` | Source code ownership transfers to governance org | Community ownership + possession |
|
|
320
|
+
| `modelWeights` | `boolean` | Trained model weights transfer to governance org | Community possession |
|
|
321
|
+
| `deploymentRights` | `boolean` | Exclusive deployment rights transfer to governance org | Community control |
|
|
323
322
|
| `revenueModel` | `string \| null` | Community-set terms for any commercial deployment, held by the governance org (e.g., `"commercial use by written permission only"`) — Champollion is non-commercial and takes no share | CARE Collective Benefit |
|
|
324
323
|
|
|
325
324
|
### `submission.retained` Fields — What the Researcher Keeps
|
|
@@ -438,7 +437,7 @@ These are documented mistakes to avoid when populating sovereignty fields. Each
|
|
|
438
437
|
|
|
439
438
|
### Don't derive sovereignty from vitality
|
|
440
439
|
|
|
441
|
-
**Wrong:** Seeing that a language is `"severely-endangered"` in `vitality.unescoStatus` and automatically populating
|
|
440
|
+
**Wrong:** Seeing that a language is `"severely-endangered"` in `vitality.unescoStatus` and automatically populating sovereignty frameworks, TK Labels, or consent models.
|
|
442
441
|
|
|
443
442
|
**Right:** Sovereignty fields are populated from **documented evidence of governance adoption** — a formal resolution, a published data governance policy, or a registered Local Contexts Hub project. Endangerment status and sovereignty are orthogonal. A safe, widely-spoken language may have strong data sovereignty frameworks (e.g., Māori → Te Mana Raraunga). A critically endangered language may have no formal governance body.
|
|
444
443
|
|
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
|
|
8
8
|
This document describes every field in the Champollion language card schema. For each field you will find the JSON key, type, data source(s), a plain-English description, and a real value from an existing card.
|
|
9
9
|
|
|
10
|
-
For
|
|
10
|
+
For provenance rules and how disagreeing sources are shown, see the [language card specification](https://champollion.dev/docs/reference/language-card-spec).
|
|
11
11
|
For license obligations see [ATTRIBUTION.md](./ATTRIBUTION.md).
|
|
12
12
|
|
|
13
13
|
---
|