tarja 0.5.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- tarja-0.5.0/.github/dependabot.yml +16 -0
- tarja-0.5.0/.github/workflows/ci.yml +104 -0
- tarja-0.5.0/.github/workflows/release.yml +63 -0
- tarja-0.5.0/.gitignore +26 -0
- tarja-0.5.0/.hypothesis/.gitignore +9 -0
- tarja-0.5.0/.hypothesis/constants/058c4eff3064e337 +4 -0
- tarja-0.5.0/.hypothesis/constants/09e605d5e9e39a96 +4 -0
- tarja-0.5.0/.hypothesis/constants/1536fa76fd657e74 +4 -0
- tarja-0.5.0/.hypothesis/constants/17a5860e69a6b020 +4 -0
- tarja-0.5.0/.hypothesis/constants/1c4637e6644a91b0 +4 -0
- tarja-0.5.0/.hypothesis/constants/1dcb52ad40b0f0fb +4 -0
- tarja-0.5.0/.hypothesis/constants/2087400ae26b175e +4 -0
- tarja-0.5.0/.hypothesis/constants/28bfcc1da69b8037 +4 -0
- tarja-0.5.0/.hypothesis/constants/34cfed09127a3555 +4 -0
- tarja-0.5.0/.hypothesis/constants/353ecff50d0482da +4 -0
- tarja-0.5.0/.hypothesis/constants/37c2ec14c524c10a +4 -0
- tarja-0.5.0/.hypothesis/constants/3ba8d594853d5fd4 +4 -0
- tarja-0.5.0/.hypothesis/constants/4284de0ec053b45b +4 -0
- tarja-0.5.0/.hypothesis/constants/45c36dc67e2208c7 +4 -0
- tarja-0.5.0/.hypothesis/constants/47d93baee4409816 +4 -0
- tarja-0.5.0/.hypothesis/constants/4af2feaccdad10de +4 -0
- tarja-0.5.0/.hypothesis/constants/59d4d6878cb5e5d5 +4 -0
- tarja-0.5.0/.hypothesis/constants/5ceb5bd990dc88cf +4 -0
- tarja-0.5.0/.hypothesis/constants/6ec1e0cb046c2482 +4 -0
- tarja-0.5.0/.hypothesis/constants/6f25a12da37e56a4 +4 -0
- tarja-0.5.0/.hypothesis/constants/7d4705152db9f69a +4 -0
- tarja-0.5.0/.hypothesis/constants/84255d0506f067a4 +4 -0
- tarja-0.5.0/.hypothesis/constants/88c71be08c3c7953 +4 -0
- tarja-0.5.0/.hypothesis/constants/8fddf9fc5ef6f086 +4 -0
- tarja-0.5.0/.hypothesis/constants/a5e0f6bc9addb970 +4 -0
- tarja-0.5.0/.hypothesis/constants/ab9c66bd57150038 +4 -0
- tarja-0.5.0/.hypothesis/constants/acf8ff06749ec0aa +4 -0
- tarja-0.5.0/.hypothesis/constants/af06ae6c4a1561df +4 -0
- tarja-0.5.0/.hypothesis/constants/b19283ba6b6d51d6 +4 -0
- tarja-0.5.0/.hypothesis/constants/bd81274d33edd06c +4 -0
- tarja-0.5.0/.hypothesis/constants/c32370e0a38c7a3f +4 -0
- tarja-0.5.0/.hypothesis/constants/c614a0131d75a43a +4 -0
- tarja-0.5.0/.hypothesis/constants/c7161542775cf0ab +4 -0
- tarja-0.5.0/.hypothesis/constants/cf4059c41f39462d +4 -0
- tarja-0.5.0/.hypothesis/constants/de3b013fc2801347 +4 -0
- tarja-0.5.0/.hypothesis/constants/eaa0d877aabafbc9 +4 -0
- tarja-0.5.0/.hypothesis/constants/ec53ec3e8273fca4 +4 -0
- tarja-0.5.0/.hypothesis/constants/ed06308583341041 +4 -0
- tarja-0.5.0/.hypothesis/constants/ed7a86c745dd90cb +4 -0
- tarja-0.5.0/.hypothesis/constants/f259e6e756fa50ce +4 -0
- tarja-0.5.0/.hypothesis/constants/f56df33f2df71fe5 +4 -0
- tarja-0.5.0/.hypothesis/constants/f9dfd68dc6c264dd +4 -0
- tarja-0.5.0/.hypothesis/unicode_data/15.0.0/charmap.json.gz +0 -0
- tarja-0.5.0/.hypothesis/unicode_data/15.0.0/codec-v2-utf-8.json.gz +0 -0
- tarja-0.5.0/.pre-commit-config.yaml +28 -0
- tarja-0.5.0/CODE_OF_CONDUCT.md +10 -0
- tarja-0.5.0/CONTRIBUTING.md +57 -0
- tarja-0.5.0/LICENSE +202 -0
- tarja-0.5.0/NOTICE +7 -0
- tarja-0.5.0/PKG-INFO +189 -0
- tarja-0.5.0/README.md +153 -0
- tarja-0.5.0/SECURITY.md +42 -0
- tarja-0.5.0/bench/ANNOTATION_GUIDE.md +36 -0
- tarja-0.5.0/bench/DATASHEET.md +119 -0
- tarja-0.5.0/bench/README.md +103 -0
- tarja-0.5.0/bench/__init__.py +6 -0
- tarja-0.5.0/bench/data/v0.1/manifest.json +27 -0
- tarja-0.5.0/bench/evaluate.py +55 -0
- tarja-0.5.0/bench/fetch_public_texts.py +152 -0
- tarja-0.5.0/bench/generate.py +158 -0
- tarja-0.5.0/bench/iaa.py +69 -0
- tarja-0.5.0/bench/ids.py +208 -0
- tarja-0.5.0/bench/metrics.py +85 -0
- tarja-0.5.0/bench/paper/by_entity_exact.csv +35 -0
- tarja-0.5.0/bench/paper/fig_f1_by_difficulty__synthetic_adversarial.test.pdf +0 -0
- tarja-0.5.0/bench/paper/fig_f1_by_difficulty__synthetic_controlled.test.pdf +0 -0
- tarja-0.5.0/bench/paper/fig_recall_by_entity__synthetic_adversarial.test.pdf +0 -0
- tarja-0.5.0/bench/paper/fig_recall_by_entity__synthetic_controlled.test.pdf +0 -0
- tarja-0.5.0/bench/paper/overall.csv +7 -0
- tarja-0.5.0/bench/paper/table_overall__synthetic_adversarial.test.tex +8 -0
- tarja-0.5.0/bench/paper/table_overall__synthetic_controlled.test.tex +8 -0
- tarja-0.5.0/bench/prompts/llm_v1.txt +18 -0
- tarja-0.5.0/bench/reference.py +131 -0
- tarja-0.5.0/bench/report.py +182 -0
- tarja-0.5.0/bench/run.py +60 -0
- tarja-0.5.0/bench/run_all.sh +43 -0
- tarja-0.5.0/bench/runners/__init__.py +27 -0
- tarja-0.5.0/bench/runners/base.py +48 -0
- tarja-0.5.0/bench/runners/cloud.py +148 -0
- tarja-0.5.0/bench/runners/ner_llm.py +125 -0
- tarja-0.5.0/bench/runners/presidio_runner.py +49 -0
- tarja-0.5.0/bench/runners/tarja_runner.py +19 -0
- tarja-0.5.0/bench/semireal.py +114 -0
- tarja-0.5.0/bench/templates.py +63 -0
- tarja-0.5.0/bench/zenodo/.zenodo.json +17 -0
- tarja-0.5.0/bench/zenodo/CITATION.cff +16 -0
- tarja-0.5.0/bench/zenodo/README.md +30 -0
- tarja-0.5.0/bench/zenodo/__init__.py +0 -0
- tarja-0.5.0/bench/zenodo/build.py +85 -0
- tarja-0.5.0/contrib/presidio-pr/PR_DESCRIPTION.md +27 -0
- tarja-0.5.0/contrib/presidio-pr/README.md +51 -0
- tarja-0.5.0/contrib/presidio-pr/default_recognizers.yaml.snippet +15 -0
- tarja-0.5.0/contrib/presidio-pr/predefined_recognizers__init__.snippet +9 -0
- tarja-0.5.0/contrib/presidio-pr/presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific/br/__init__.py +6 -0
- tarja-0.5.0/contrib/presidio-pr/presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific/br/br_cnpj_recognizer.py +101 -0
- tarja-0.5.0/contrib/presidio-pr/presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific/br/br_cpf_recognizer.py +94 -0
- tarja-0.5.0/contrib/presidio-pr/presidio-analyzer/tests/test_br_cnpj_recognizer.py +53 -0
- tarja-0.5.0/contrib/presidio-pr/presidio-analyzer/tests/test_br_cpf_recognizer.py +52 -0
- tarja-0.5.0/contrib/presidio-pr/supported_entities.snippet.md +8 -0
- tarja-0.5.0/docs/SOURCES.md +30 -0
- tarja-0.5.0/docs/presidio.md +52 -0
- tarja-0.5.0/examples/presidio_pt.py +35 -0
- tarja-0.5.0/examples/rag_pipeline.py +119 -0
- tarja-0.5.0/packages/presidio-br/README.md +26 -0
- tarja-0.5.0/packages/presidio-br/pyproject.toml +28 -0
- tarja-0.5.0/packages/presidio-br/src/presidio_br/__init__.py +81 -0
- tarja-0.5.0/packages/presidio-br/tests/test_presidio_br.py +104 -0
- tarja-0.5.0/pyproject.toml +82 -0
- tarja-0.5.0/spec/entities/br_cartao.yaml +51 -0
- tarja-0.5.0/spec/entities/br_cep.yaml +47 -0
- tarja-0.5.0/spec/entities/br_cib.yaml +55 -0
- tarja-0.5.0/spec/entities/br_cnh.yaml +49 -0
- tarja-0.5.0/spec/entities/br_cnj.yaml +54 -0
- tarja-0.5.0/spec/entities/br_cnm.yaml +52 -0
- tarja-0.5.0/spec/entities/br_cnpj.yaml +59 -0
- tarja-0.5.0/spec/entities/br_cns.yaml +55 -0
- tarja-0.5.0/spec/entities/br_cpf.yaml +55 -0
- tarja-0.5.0/spec/entities/br_iptu.yaml +47 -0
- tarja-0.5.0/spec/entities/br_matricula_imovel.yaml +56 -0
- tarja-0.5.0/spec/entities/br_nis.yaml +52 -0
- tarja-0.5.0/spec/entities/br_pix_evp.yaml +47 -0
- tarja-0.5.0/spec/entities/br_placa.yaml +52 -0
- tarja-0.5.0/spec/entities/br_renavam.yaml +53 -0
- tarja-0.5.0/spec/entities/br_telefone.yaml +45 -0
- tarja-0.5.0/spec/entities/br_titulo_eleitor.yaml +51 -0
- tarja-0.5.0/spec/registry.yaml +21 -0
- tarja-0.5.0/spec/schema.json +291 -0
- tarja-0.5.0/src/tarja/__init__.py +65 -0
- tarja-0.5.0/src/tarja/__main__.py +5 -0
- tarja-0.5.0/src/tarja/cli.py +129 -0
- tarja-0.5.0/src/tarja/detect.py +187 -0
- tarja-0.5.0/src/tarja/entities.py +446 -0
- tarja-0.5.0/src/tarja/mask.py +121 -0
- tarja-0.5.0/src/tarja/normalise.py +77 -0
- tarja-0.5.0/src/tarja/py.typed +0 -0
- tarja-0.5.0/src/tarja/registry.py +87 -0
- tarja-0.5.0/src/tarja/validators/__init__.py +4 -0
- tarja-0.5.0/src/tarja/validators/cartao.py +46 -0
- tarja-0.5.0/src/tarja/validators/cep.py +28 -0
- tarja-0.5.0/src/tarja/validators/cib.py +61 -0
- tarja-0.5.0/src/tarja/validators/cnh.py +51 -0
- tarja-0.5.0/src/tarja/validators/cnj.py +67 -0
- tarja-0.5.0/src/tarja/validators/cnm.py +50 -0
- tarja-0.5.0/src/tarja/validators/cnpj.py +93 -0
- tarja-0.5.0/src/tarja/validators/cns.py +82 -0
- tarja-0.5.0/src/tarja/validators/cpf.py +78 -0
- tarja-0.5.0/src/tarja/validators/iptu.py +29 -0
- tarja-0.5.0/src/tarja/validators/matricula.py +29 -0
- tarja-0.5.0/src/tarja/validators/nis.py +66 -0
- tarja-0.5.0/src/tarja/validators/pix.py +19 -0
- tarja-0.5.0/src/tarja/validators/placa.py +44 -0
- tarja-0.5.0/src/tarja/validators/renavam.py +47 -0
- tarja-0.5.0/src/tarja/validators/telefone.py +43 -0
- tarja-0.5.0/src/tarja/validators/titulo.py +77 -0
- tarja-0.5.0/src/tarja/vault.py +201 -0
- tarja-0.5.0/tests/test_api.py +23 -0
- tarja-0.5.0/tests/test_bench.py +303 -0
- tarja-0.5.0/tests/test_cli.py +96 -0
- tarja-0.5.0/tests/test_cnpj.py +96 -0
- tarja-0.5.0/tests/test_cpf.py +65 -0
- tarja-0.5.0/tests/test_detect.py +135 -0
- tarja-0.5.0/tests/test_e9.py +158 -0
- tarja-0.5.0/tests/test_fuzz.py +93 -0
- tarja-0.5.0/tests/test_mask.py +67 -0
- tarja-0.5.0/tests/test_normalise.py +36 -0
- tarja-0.5.0/tests/test_perf.py +50 -0
- tarja-0.5.0/tests/test_property_ids.py +106 -0
- tarja-0.5.0/tests/test_registry.py +62 -0
- tarja-0.5.0/tests/test_regressions.py +59 -0
- tarja-0.5.0/tests/test_spec.py +108 -0
- tarja-0.5.0/tests/test_validators_e2.py +127 -0
- tarja-0.5.0/tests/test_validators_e29.py +150 -0
- tarja-0.5.0/tests/test_vault_scope.py +134 -0
- tarja-0.5.0/tools/ci_contrib.py +70 -0
- tarja-0.5.0/tools/gen_entities.py +175 -0
- tarja-0.5.0/tools/mutation_check.py +153 -0
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# .github/dependabot.yml
|
|
2
|
+
# [DEPENDABOT] weekly PRs for GitHub Actions (keeps pinned commit SHAs current) and pip extras
|
|
3
|
+
version: 2
|
|
4
|
+
updates:
|
|
5
|
+
- package-ecosystem: github-actions
|
|
6
|
+
directory: /
|
|
7
|
+
schedule:
|
|
8
|
+
interval: weekly
|
|
9
|
+
groups:
|
|
10
|
+
actions:
|
|
11
|
+
patterns: ["*"]
|
|
12
|
+
- package-ecosystem: pip
|
|
13
|
+
directory: /
|
|
14
|
+
schedule:
|
|
15
|
+
interval: weekly
|
|
16
|
+
open-pull-requests-limit: 5
|
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
# .github/workflows/ci.yml
|
|
2
|
+
# [CI] EN: on every push/PR: lint + tests with coverage >= 90% on Python 3.10 to 3.13 (E1.2)
|
|
3
|
+
# [CI] PT: em todo push/PR: lint + testes c/ cobertura >= 90% no Python 3.10 a 3.13 (E1.2)
|
|
4
|
+
|
|
5
|
+
name: ci
|
|
6
|
+
|
|
7
|
+
on:
|
|
8
|
+
push:
|
|
9
|
+
branches: [main]
|
|
10
|
+
pull_request:
|
|
11
|
+
|
|
12
|
+
# EN: least privilege / PT: permissao minima
|
|
13
|
+
permissions:
|
|
14
|
+
contents: read
|
|
15
|
+
|
|
16
|
+
jobs:
|
|
17
|
+
# [CI-LINT] EN: ruff lint + format check, once / PT: ruff lint + format, 1 vez so
|
|
18
|
+
lint:
|
|
19
|
+
runs-on: ubuntu-24.04
|
|
20
|
+
steps:
|
|
21
|
+
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
22
|
+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
|
|
23
|
+
with:
|
|
24
|
+
python-version: "3.12"
|
|
25
|
+
# EN: pinned, same version as pre-commit / PT: versao fixa, igual ao pre-commit
|
|
26
|
+
- run: pip install "ruff==0.15.11"
|
|
27
|
+
- run: ruff check .
|
|
28
|
+
- run: ruff format --check .
|
|
29
|
+
# [CI-GEN-ENTITIES] src/tarja/entities.py must match spec/*.yaml (single source of truth)
|
|
30
|
+
- run: pip install "pyyaml>=6"
|
|
31
|
+
- run: python tools/gen_entities.py --check
|
|
32
|
+
|
|
33
|
+
# [CI-TEST] EN: test matrix, 4 Python versions / PT: matriz de teste, 4 versoes do Python
|
|
34
|
+
test:
|
|
35
|
+
runs-on: ubuntu-24.04
|
|
36
|
+
strategy:
|
|
37
|
+
fail-fast: false
|
|
38
|
+
matrix:
|
|
39
|
+
python-version: ["3.10", "3.11", "3.12", "3.13"]
|
|
40
|
+
steps:
|
|
41
|
+
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
42
|
+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
|
|
43
|
+
with:
|
|
44
|
+
python-version: ${{ matrix.python-version }}
|
|
45
|
+
# EN: install package + dev extras / PT: instala o pacote + extras de dev
|
|
46
|
+
- run: pip install -e ".[dev]"
|
|
47
|
+
# EN: pytest config (coverage >= 90%) lives in pyproject.toml / PT: config do pytest (cobertura >= 90%) fica no pyproject.toml
|
|
48
|
+
# EN: TARJA_NO_SKIP=1 -> a missing test dependency is an error, never a silent skip
|
|
49
|
+
# PT: TARJA_NO_SKIP=1 -> dependencia de teste faltando e erro, nunca pulo silencioso
|
|
50
|
+
# [CI-DEV-DIRS] proves the editable install exposes bench/ without pytest's pythonpath (run outside the repo)
|
|
51
|
+
- run: cd /tmp && python -c "import bench.reference, tarja; print('bench importable')"
|
|
52
|
+
- run: pytest tests
|
|
53
|
+
env:
|
|
54
|
+
TARJA_NO_SKIP: "1"
|
|
55
|
+
# [CI-PERF] EN: E2.8, print the speed number in the log (pytest hides prints) / PT: E2.8, mostra a velocidade no log
|
|
56
|
+
- run: python -m unittest tests.test_perf -v
|
|
57
|
+
|
|
58
|
+
# [CI-TYPES] static type check of the public package (py.typed is shipped, so types are part of the API)
|
|
59
|
+
types:
|
|
60
|
+
runs-on: ubuntu-24.04
|
|
61
|
+
steps:
|
|
62
|
+
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
63
|
+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
|
|
64
|
+
with:
|
|
65
|
+
python-version: "3.12"
|
|
66
|
+
- run: pip install -e ".[dev]"
|
|
67
|
+
- run: mypy
|
|
68
|
+
|
|
69
|
+
# [CI-MUTATION] EN: proves the tests catch real bugs: 27 deliberate bugs, all must make the suite fail
|
|
70
|
+
# [CI-MUTATION] PT: prova q os testes pegam bug de verdade: 27 bugs de proposito, todos tem q derrubar a suite
|
|
71
|
+
mutation:
|
|
72
|
+
runs-on: ubuntu-24.04
|
|
73
|
+
steps:
|
|
74
|
+
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
75
|
+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
|
|
76
|
+
with:
|
|
77
|
+
python-version: "3.12"
|
|
78
|
+
- run: pip install -e ".[dev]"
|
|
79
|
+
- run: python tools/mutation_check.py
|
|
80
|
+
|
|
81
|
+
# [CI-PRESIDIO-BR] EN: plugin tests against the REAL presidio-analyzer, skipping is forbidden here
|
|
82
|
+
# [CI-PRESIDIO-BR] PT: testes do plugin contra o presidio-analyzer DE VERDADE, pular e proibido aqui
|
|
83
|
+
presidio-br:
|
|
84
|
+
runs-on: ubuntu-24.04
|
|
85
|
+
steps:
|
|
86
|
+
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
87
|
+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
|
|
88
|
+
with:
|
|
89
|
+
python-version: "3.12"
|
|
90
|
+
- run: pip install -e ".[dev]" -e packages/presidio-br
|
|
91
|
+
- run: pytest packages/presidio-br/tests -p no:cacheprovider --no-cov
|
|
92
|
+
env:
|
|
93
|
+
TARJA_REQUIRE_PRESIDIO: "1"
|
|
94
|
+
|
|
95
|
+
# [CI-CONTRIB] the Presidio PR kit's tests against the real presidio-analyzer. Delete after the PR merges.
|
|
96
|
+
contrib:
|
|
97
|
+
runs-on: ubuntu-24.04
|
|
98
|
+
steps:
|
|
99
|
+
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
100
|
+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
|
|
101
|
+
with:
|
|
102
|
+
python-version: "3.12"
|
|
103
|
+
- run: pip install presidio-analyzer pytest
|
|
104
|
+
- run: python tools/ci_contrib.py
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# .github/workflows/release.yml
|
|
2
|
+
# [RELEASE] EN: publish to PyPI when a tag like v0.1.0 is pushed (E1.6). Uses PyPI Trusted Publishing, no token stored.
|
|
3
|
+
# [RELEASE] PT: publica no PyPI qdo sobe uma tag tipo v0.1.0 (E1.6). Usa Trusted Publishing do PyPI, sem token guardado.
|
|
4
|
+
#
|
|
5
|
+
# EN: one-off setup (manual, see PENDING.md): register this workflow as a trusted publisher on PyPI
|
|
6
|
+
# (project "tarja", owner macmaia, repo tarja, workflow release.yml, environment pypi)
|
|
7
|
+
# PT: setup 1x (manual, ver PENDING.md): cadastrar este workflow como trusted publisher no PyPI
|
|
8
|
+
# (projeto "tarja", owner macmaia, repo tarja, workflow release.yml, environment pypi)
|
|
9
|
+
#
|
|
10
|
+
# EN: to release: bump __version__ in src/tarja/__init__.py, commit, then: git tag v0.1.0 && git push origin v0.1.0
|
|
11
|
+
# PT: p/ lancar: sobe o __version__ em src/tarja/__init__.py, commit, dps: git tag v0.1.0 && git push origin v0.1.0
|
|
12
|
+
|
|
13
|
+
name: release
|
|
14
|
+
|
|
15
|
+
on:
|
|
16
|
+
push:
|
|
17
|
+
tags: ["v*"]
|
|
18
|
+
|
|
19
|
+
permissions:
|
|
20
|
+
contents: read
|
|
21
|
+
|
|
22
|
+
jobs:
|
|
23
|
+
# [RELEASE-BUILD] EN: tests again, checks tag == package version, builds sdist + wheel
|
|
24
|
+
# [RELEASE-BUILD] PT: roda os testes de novo, confere tag == versao do pacote, gera sdist + wheel
|
|
25
|
+
build:
|
|
26
|
+
runs-on: ubuntu-24.04
|
|
27
|
+
steps:
|
|
28
|
+
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
|
29
|
+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
|
|
30
|
+
with:
|
|
31
|
+
python-version: "3.12"
|
|
32
|
+
- run: pip install -e ".[dev]" build
|
|
33
|
+
- run: pytest
|
|
34
|
+
# EN: tag v0.1.0 must match __version__ 0.1.0, otherwise fail
|
|
35
|
+
# PT: tag v0.1.0 tem q bater c/ __version__ 0.1.0, senao falha
|
|
36
|
+
- name: check version / confere versao
|
|
37
|
+
run: |
|
|
38
|
+
PKG=$(python -c "import tarja; print(tarja.__version__)")
|
|
39
|
+
TAG=${GITHUB_REF_NAME#v}
|
|
40
|
+
echo "package=$PKG tag=$TAG"
|
|
41
|
+
test "$PKG" = "$TAG"
|
|
42
|
+
- run: python -m build
|
|
43
|
+
- uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5
|
|
44
|
+
with:
|
|
45
|
+
name: dist
|
|
46
|
+
path: dist/
|
|
47
|
+
|
|
48
|
+
# [RELEASE-PUBLISH] EN: uploads to PyPI via OIDC (id-token), protected by the "pypi" environment
|
|
49
|
+
# [RELEASE-PUBLISH] PT: sobe p/ o PyPI via OIDC (id-token), protegido pelo environment "pypi"
|
|
50
|
+
publish:
|
|
51
|
+
needs: build
|
|
52
|
+
runs-on: ubuntu-24.04
|
|
53
|
+
environment:
|
|
54
|
+
name: pypi
|
|
55
|
+
url: https://pypi.org/project/tarja/
|
|
56
|
+
permissions:
|
|
57
|
+
id-token: write
|
|
58
|
+
steps:
|
|
59
|
+
- uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5
|
|
60
|
+
with:
|
|
61
|
+
name: dist
|
|
62
|
+
path: dist/
|
|
63
|
+
- uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
|
tarja-0.5.0/.gitignore
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
# EN: python / PT: python
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
.venv/
|
|
5
|
+
dist/
|
|
6
|
+
build/
|
|
7
|
+
*.egg-info/
|
|
8
|
+
# EN: tests/lint / PT: testes/lint
|
|
9
|
+
.coverage
|
|
10
|
+
htmlcov/
|
|
11
|
+
.pytest_cache/
|
|
12
|
+
.ruff_cache/
|
|
13
|
+
# EN: macOS
|
|
14
|
+
.DS_Store
|
|
15
|
+
# EN: local pre-publish checklist, not versioned / PT: checklist local pre-publicacao, fora do git
|
|
16
|
+
PENDING.md
|
|
17
|
+
# EN: NEVER version real data / PT: NUNCA versionar dado real
|
|
18
|
+
data/real/
|
|
19
|
+
*.real.jsonl
|
|
20
|
+
# EN: benchmark: data is regenerated from the seed, results/cache are local / PT: dados regerados pela seed, resultados/cache locais
|
|
21
|
+
bench/data/**/*.jsonl
|
|
22
|
+
bench/results/
|
|
23
|
+
bench/cache/
|
|
24
|
+
# EN: private notes / PT: notas privadas
|
|
25
|
+
docs/decisions.md
|
|
26
|
+
bench/paper/*.png
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
# This .gitignore file was automatically created by Hypothesis. Hypothesis gitignores
|
|
2
|
+
# .hypothesis by default, because we generally recommend that .hypothesis not be checked
|
|
3
|
+
# into version control.
|
|
4
|
+
#
|
|
5
|
+
# If you *would* like to check .hypothesis into version control, you should delete this
|
|
6
|
+
# file. Hypothesis will not re-create this .gitignore unless .hypothesis is deleted (and
|
|
7
|
+
# if it does, that's a bug - please report it!)
|
|
8
|
+
|
|
9
|
+
*
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/bench/semireal.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
[0.3, 200, 1200, 2000, '(?<=[.;:])\\s+', '*.txt', '--n', '--out', '--seed', '--src', 'BR_CARTAO', 'BR_CEP', 'BR_CIB', 'BR_CNH', 'BR_CNJ', 'BR_CNM', 'BR_CNPJ', 'BR_CNS', 'BR_CPF', 'BR_IPTU', 'BR_MATRICULA_IMOVEL', 'BR_NIS', 'BR_PIX_EVP', 'BR_PLACA', 'BR_RENAVAM', 'BR_TELEFONE', 'BR_TITULO_ELEITOR', 'Condutor com CNH {}.', 'D0', 'Endereco no CEP {}.', 'Imovel de CNM {}.', 'Veiculo de placa {}.', '\\n\\s*\\n', '__main__', 'bench/data/v0.1', 'compact', 'difficulty', 'domain', 'end', 'entity', 'file', 'formatted', 'id', 'semireal', 'semireal.dev', 'semireal.test', 'source', 'sources.json', 'spans', 'start', 'text', 'utf-8', '{}']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/bench/run.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
['--data', '--model', '--out', '--provider', '--system', '__main__', 'bench/results', 'id', 'info', 'latency_ms_median', 'model', 'predict_doc', 'predictions', 'provider', 'system', 'text', 'utf-8']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/src/tarja/entities.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.85, 0.9, 0.95, 'BR_CARTAO', 'BR_CEP', 'BR_CIB', 'BR_CNH', 'BR_CNJ', 'BR_CNM', 'BR_CNPJ', 'BR_CNS', 'BR_CPF', 'BR_IPTU', 'BR_MATRICULA_IMOVEL', 'BR_NIS', 'BR_PIX_EVP', 'BR_PLACA', 'BR_RENAVAM', 'BR_TELEFONE', 'BR_TITULO_ELEITOR', 'N1', 'N2', 'N3', 'N4', '\\b[12789]\\d{14}\\b', '\\b[A-Z]{3}-?\\d{4}\\b', '\\b\\d{1,7}\\b', '\\b\\d{11}\\b', '\\b\\d{13,19}\\b', '\\b\\d{20}\\b', '\\b\\d{5}-?\\d{3}\\b', '\\b\\d{6}[23]\\d{9}\\b', '\\b\\d{9}\\b', 'acao', 'amex', 'automovel', 'autos', 'cadastro imobiliario', 'cadastro municipal', 'carro', 'cartao', 'cartao do sus', 'cartao sus', 'cartao_16', 'cartao_amex', 'cartao_compact', 'cel', 'celular', 'cep', 'chave', 'chave aleatoria', 'chave pix', 'cib', 'cib_spaced', 'cnh', 'cnh_compact', 'cnj_compact', 'cnj_formatted', 'cnj_spaced', 'cnm', 'cnm_compact', 'cnm_formatted', 'cnm_spaced', 'cnpj', 'cnpj_compact', 'cnpj_formatted', 'cnpj_spaced', 'cns', 'cns_compact', 'cns_formatted', 'codigo postal', 'codigo renavam', 'contato', 'contribuinte', 'cpf', 'cpf_compact', 'cpf_formatted', 'credito', 'cri', 'debito', 'eleitor', 'elo', 'empresa', 'final', 'fone', 'habilitacao', 'hipercard', 'imovel', 'indice cadastral', 'inscricao do imovel', 'inscricao eleitoral', 'inscrita no cnpj', 'inscrita sob o', 'inscrito no cpf', 'iptu', 'ligar', 'mastercard', 'matricula', 'matricula do imovel', 'matricula n', 'matricula no', 'matricula sob o n', 'matricula_dotted', 'matricula_plain', 'moto', 'nis', 'nis_compact', 'nis_formatted', 'nis_spaced', 'nit', 'numero do processo', 'pasep', 'pessoa juridica', 'pis', 'pis/pasep', 'pix', 'pix_uuid', 'placa', 'placa_mercosur', 'placa_old', 'portador do cpf', 'proc.', 'processo', 'recurso', 'registro da cnh', 'registro de imoveis', 'renavam', 'renavam_11', 'renavam_9', 'rgi', 'sql', 'sus', 'tel', 'telefone', 'titulo de eleitor', 'titulo eleitoral', 'titulo_spaced', 'veiculo', 'visa', 'whats', 'whatsapp']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/bench/fetch_public_texts.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
[2.0, '--out', '--pause', 'Accept', 'Accept-Language', 'User-Agent', '\\n\\s*\\n', '__main__', 'br', 'constituicao_1988', 'cp1252', 'del', 'div', 'h1', 'h2', 'h3', 'h4', 'head', 'latin-1', 'lei_13105_cpc', 'lei_13709_lgpd', 'lei_14133_licitacoes', 'lei_5172_ctn', 'lei_8078_cdc', 'lei_8080_sus', 'lei_8112_servidores', 'lei_8213_previdencia', 'lei_9503_ctb', 'leis/l8080.htm', 'leis/l9784.htm', 'li', 'licence', 'p', 'pt-BR,pt;q=0.9', 'retrieved', 's', 'script', 'sources.json', 'strike', 'style', 'table', 'tr', 'url', 'utf-8']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/src/tarja/cli.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
[1024, '*', ',', '-', '--entities', '--format', '--max-mb', '--min-score', '--salt', '--show-values', '--strategy', '--suspect', '--version', 'TARJA_SALT', '__main__', 'command', 'jsonl', 'mask', 'path', 'redact', 'scan', 'store_true', 'table', 'tarja', 'utf-8', 'version']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/bench/generate.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
[0.2, 0.25, 0.3, 0.4, 0.5, 0.6, 3000, 5000, ' |', '--adversarial', '--controlled', '--out', '--seed', '.-/ ', '0', '01', '1', ':', 'BR_CNJ', 'BR_CNPJ', 'BR_CNS', 'BR_CPF', 'BR_NIS', 'BR_PIX_EVP', 'BR_TITULO_ELEITOR', 'D0', 'D1', 'D2', 'D3', 'D4', 'D5', 'O', '\\{(BR_[A-Z_]+)\\}', '__main__', 'bench/data/v0.1', 'bench_version', 'compact', 'difficulty', 'docs', 'domain', 'end', 'entity', 'files', 'formatted', 'id', 'l', 'manifest.json', 'seed', 'sha256', 'spaced', 'spans', 'start', 'synthetic_controlled', 'tarja_version', 'text', 'utf-8', '| ']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/src/tarja/mask.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
['[^0-9A-Za-z]', '_-.', 'admin', 'change-me', 'changeme', 'default', 'example', 'hash', 'letmein', 'mysalt', 'mysecret', 'passwd', 'password', 'pseudonym', 'pseudonym_stable', 'redact', 'replace', 's3cr3t', 'salt', 'sample', 'secret', 'tarja', 'test', 'testing', 'utf-8', 'your-secret', 'yoursecret']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/bench/metrics.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
[0.025, 0.975, 1000, 'by_difficulty', 'by_entity', 'difficulty', 'docs', 'end', 'entity', 'exact', 'f1', 'f1_ci95', 'fn', 'fp', 'id', 'mode', 'overall', 'partial', 'precision', 'recall', 'spans', 'start', 'tp', 'untyped']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/bench/runners/ner_llm.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
[1.0, 2048, 'BR_CEP', 'BR_CIB', 'BR_CNH', 'BR_CNJ', 'BR_CNM', 'BR_CNPJ', 'BR_CNS', 'BR_CPF', 'BR_IPTU', 'BR_MATRICULA_IMOVEL', 'BR_NIS', 'BR_PIX_EVP', 'BR_PLACA', 'BR_RENAVAM', 'BR_TELEFONE', 'BR_TITULO_ELEITOR', 'OTHER', '\\{.*\\}', 'a', 'anthropic', 'bench/cache', 'content', 'date', 'entities', 'key', 'llm', 'llm_v1.txt', 'model', 'openai', 'prompt', 'prompts', 'provider', 'pt_core_news_lg', 'raw', 'role', 'spacy', 'temperature', 'text', 'type', 'user', 'utf-8', '{TEXT}']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/src/tarja/__init__.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
['0.5.0', 'ENTITIES', 'Match', 'ProtectedText', 'Vault', 'VaultCollisionError', 'VaultConsumedError', 'VaultError', 'VaultExpiredError', 'VaultScopeError', 'cartao', 'cnj', 'cnpj', 'cns', 'cpf', 'find', 'mask', 'nis', 'register_entity', 'residual', 'resolve_overlaps', 'unregister_entity', 'validate']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/src/tarja/validators/telefone.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
['11', '12', '13', '14', '15', '16', '17', '18', '19', '21', '22', '2345', '24', '27', '28', '31', '32', '33', '34', '35', '37', '38', '41', '42', '43', '44', '45', '46', '47', '48', '49', '51', '53', '54', '55', '61', '62', '63', '64', '65', '66', '67', '68', '69', '71', '73', '74', '75', '77', '79', '81', '82', '83', '84', '85', '86', '87', '88', '89', '9', '91', '92', '93', '94', '95', '96', '97', '98', '99', '\\D']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/bench/ids.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
[0.5, 0.7, 100, 128, 999999, ',', '.', '0', '0001', '1', '12', '23', '2345', '4', '789', '9', 'BR_CARTAO', 'BR_CEP', 'BR_CIB', 'BR_CNH', 'BR_CNJ', 'BR_CNM', 'BR_CNPJ', 'BR_CNS', 'BR_CPF', 'BR_IPTU', 'BR_MATRICULA_IMOVEL', 'BR_NIS', 'BR_PIX_EVP', 'BR_PLACA', 'BR_RENAVAM', 'BR_TELEFONE', 'BR_TITULO_ELEITOR', '[.\\-/]', '[\\s.\\-/()]', 'compact', 'formatted', 'reference', 'spaced']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/bench/runners/cloud.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
[1.0, '.txt', '/', 'AZURE_LANGUAGE_KEY', 'Amazon Macie', 'BRAZIL_CEP_CODE', 'BRAZIL_CNPJ_NUMBER', 'BRAZIL_CPF_NUMBER', 'BRAZIL_PHONE_NUMBER', 'BRAZIL_RG_NUMBER', 'BRCPFNumber', 'BRLegalEntityNumber', 'BR_CEP', 'BR_CNPJ', 'BR_CPF', 'BR_TELEFONE', 'Brazil CPF Number', 'Google Cloud SDP', 'MACIE_FINDINGS_JSON', 'OTHER', 'PHONE_NUMBER', 'PURVIEW_EXPORT_CSV', 'PhoneNumber', 'azure', 'date', 'detections', 'doc_id', 'end', 'endColumn', 'google-sdp', 'ignore', 'include_quote', 'info_types', 'inspect_config', 'item', 'key', 'language', 'lineRanges', 'macie', 'name', 'occurrences', 'parent', 'pt-BR', 'purview', 'resourcesAffected', 'result', 's3Object', 'sensitiveData', 'service', 'sit_name', 'start', 'startColumn', 'type', 'utf-8', 'value']
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# file: /home/runner/work/tarja/tarja/src/tarja/validators/titulo.py
|
|
2
|
+
# hypothesis_version: 6.168.0
|
|
3
|
+
|
|
4
|
+
['01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', 'AC', 'AL', 'AM', 'AP', 'BA', 'CE', 'DF', 'ES', 'GO', 'MA', 'MG', 'MS', 'MT', 'PA', 'PB', 'PE', 'PI', 'PR', 'RJ', 'RN', 'RO', 'RR', 'RS', 'SC', 'SE', 'SP', 'TO', 'ZZ', '[\\s.\\-/]', '\\d{12}', '\\d{8}']
|
|
Binary file
|
|
Binary file
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# .pre-commit-config.yaml
|
|
2
|
+
# EN: runs lint/format + basic hygiene before every commit. Setup once: pip install pre-commit && pre-commit install
|
|
3
|
+
# PT: roda lint/format + higiene basica antes de cada commit. Setup 1x: pip install pre-commit && pre-commit install
|
|
4
|
+
# EN: bump versions with `pre-commit autoupdate` / PT: atualizar versoes c/ `pre-commit autoupdate`
|
|
5
|
+
|
|
6
|
+
repos:
|
|
7
|
+
# [PRECOMMIT-HYGIENE] EN: whitespace, EOF, yaml/json/toml syntax, big files, merge markers, private keys
|
|
8
|
+
# [PRECOMMIT-HYGIENE] PT: espaco sobrando, fim de arquivo, sintaxe yaml/json/toml, arquivo grande, conflito de merge, chave privada
|
|
9
|
+
- repo: https://github.com/pre-commit/pre-commit-hooks
|
|
10
|
+
rev: v6.0.0
|
|
11
|
+
hooks:
|
|
12
|
+
- id: trailing-whitespace
|
|
13
|
+
- id: end-of-file-fixer
|
|
14
|
+
- id: check-yaml
|
|
15
|
+
- id: check-json
|
|
16
|
+
- id: check-toml
|
|
17
|
+
- id: check-added-large-files
|
|
18
|
+
args: ["--maxkb=500"]
|
|
19
|
+
- id: check-merge-conflict
|
|
20
|
+
- id: detect-private-key
|
|
21
|
+
|
|
22
|
+
# [PRECOMMIT-RUFF] EN: lint (with autofix) + format / PT: lint (c/ autofix) + format
|
|
23
|
+
- repo: https://github.com/astral-sh/ruff-pre-commit
|
|
24
|
+
rev: v0.15.11
|
|
25
|
+
hooks:
|
|
26
|
+
- id: ruff-check
|
|
27
|
+
args: ["--fix"]
|
|
28
|
+
- id: ruff-format
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# code of conduct / código de conduta
|
|
2
|
+
|
|
3
|
+
EN: we follow the Contributor Covenant 2.1: https://www.contributor-covenant.org/version/2/1/code_of_conduct/
|
|
4
|
+
PT: a gente segue o Contributor Covenant 2.1: https://www.contributor-covenant.org/pt-br/version/2/1/code_of_conduct/
|
|
5
|
+
|
|
6
|
+
EN: in short: respect everyone, criticise code and ideas (not people), no harassment, no personal attacks. And since this project is about personal data: never post anyone's PII in an issue or PR.
|
|
7
|
+
PT: resumindo: respeito c/ todo mundo, crítica é p/ código e ideia (não p/ pessoa), sem assédio, sem ataque pessoal. E, como o projeto é sobre dado pessoal: não publica PII de ninguém em issue/PR.
|
|
8
|
+
|
|
9
|
+
EN: problem? email tarja@micah6ai.com or DM [@macmaia](https://github.com/macmaia). Kept confidential.
|
|
10
|
+
PT: problema? manda e-mail p/ tarja@micah6ai.com ou msg privada p/ [@macmaia](https://github.com/macmaia). Fica em sigilo.
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# contributing / contribuindo
|
|
2
|
+
|
|
3
|
+
## setup
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
git clone https://github.com/macmaia/tarja && cd tarja
|
|
7
|
+
python -m venv .venv && source .venv/bin/activate
|
|
8
|
+
pip install -e ".[dev]" pre-commit
|
|
9
|
+
pre-commit install # EN: lint on every commit / PT: lint em todo commit
|
|
10
|
+
pytest # EN: min. 90% coverage / PT: cobertura min. 90%
|
|
11
|
+
ruff check . && ruff format --check .
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
## new entity / entidade nova
|
|
15
|
+
|
|
16
|
+
EN:
|
|
17
|
+
1. Open an issue first with the entity, the official standard and a few examples.
|
|
18
|
+
2. Create `spec/entities/<id>.yaml` following `spec/schema.json` (id with `BR_` prefix). Fill in `description` (EN-UK) and `description_pt`.
|
|
19
|
+
3. `sources` must point to the official standard for the algorithm (Receita, TSE, Ministry of Health, CNJ...). No official source = goes in as `experimental`.
|
|
20
|
+
4. Validator in `src/tarja/validators/<name>.py`: pure function, stdlib only, source repeated at the top, every block commented in short English with a `[NAME-THING]` tag. Public docstrings are bilingual, EN first then PT.
|
|
21
|
+
5. Tests: official case, wrong check digit, bad format, repeated digits (where it makes sense) and one property test (generate, validate, mutate).
|
|
22
|
+
6. Add the id to `spec/registry.yaml` and run `python tools/gen_entities.py`. Never edit `src/tarja/entities.py` by hand: it is generated and CI checks it.
|
|
23
|
+
|
|
24
|
+
EN: an identifier that only matters to you (a company ID, one state's IE) does not need a PR: use `tarja.register_entity()` at start-up.
|
|
25
|
+
|
|
26
|
+
PT:
|
|
27
|
+
1. Abre uma issue antes, c/ a entidade, a norma oficial e uns exemplos.
|
|
28
|
+
2. Cria `spec/entities/<id>.yaml` seguindo `spec/schema.json` (id c/ prefixo `BR_`). Preenche `description` (EN-UK) e `description_pt`.
|
|
29
|
+
3. `sources` c/ a norma oficial do algoritmo (Receita, TSE, MS, CNJ...). Sem fonte oficial entra como `experimental`.
|
|
30
|
+
4. Validador em `src/tarja/validators/<nome>.py`: função pura, só stdlib, fonte repetida no topo, todo bloco comentado em inglês curto c/ tag `[NOME-ALGO]`. Docstring pública bilíngue, EN primeiro e PT dps.
|
|
31
|
+
5. Testes: caso oficial, DV errado, formato ruim, repetidos (qdo fizer sentido) e 1 teste de propriedade (gera, valida, muta).
|
|
32
|
+
6. Põe o id no `spec/registry.yaml` e roda `python tools/gen_entities.py`. Nunca edite o `src/tarja/entities.py` à mão: ele é gerado e o CI confere.
|
|
33
|
+
|
|
34
|
+
PT: identificador q só importa p/ vc (matrícula de empresa, IE de 1 estado) não precisa de PR: use `tarja.register_entity()` na inicialização.
|
|
35
|
+
|
|
36
|
+
## real data: no / dado real: não
|
|
37
|
+
|
|
38
|
+
EN: never, anywhere (code, tests, issues, PRs, logs). Only generated numbers with valid check digits or examples published by a public body. Found real data here? See `SECURITY.md`.
|
|
39
|
+
PT: nunca, em lugar nenhum (código, teste, issue, PR, log). Só número gerado c/ DV válido ou exemplo publicado por órgão oficial. Achou dado real aqui? Ver `SECURITY.md`.
|
|
40
|
+
|
|
41
|
+
## commits / PRs
|
|
42
|
+
|
|
43
|
+
EN:
|
|
44
|
+
- `git commit -s` (DCO, https://developercertificate.org)
|
|
45
|
+
- one PR per entity or per change
|
|
46
|
+
- in the PR, say how you tested accuracy
|
|
47
|
+
- code identifiers in UK English (`normalise`, not `normalize`)
|
|
48
|
+
|
|
49
|
+
PT:
|
|
50
|
+
- `git commit -s` (DCO, https://developercertificate.org)
|
|
51
|
+
- 1 PR por entidade ou por mudança
|
|
52
|
+
- no PR, conta como vc testou a acurácia
|
|
53
|
+
- nomes no código em inglês UK (`normalise`, não `normalize`)
|
|
54
|
+
|
|
55
|
+
## conduct / conduta
|
|
56
|
+
|
|
57
|
+
EN: see `CODE_OF_CONDUCT.md`. PT: ver `CODE_OF_CONDUCT.md`.
|