WeTextProcessing 1.2.0__tar.gz → 1.3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- wetextprocessing-1.3.0/.github/workflows/android.yml +91 -0
- wetextprocessing-1.3.0/.github/workflows/runtime.yml +69 -0
- wetextprocessing-1.3.0/.github/workflows/unittest.yml +86 -0
- wetextprocessing-1.3.0/.github/workflows/wheels.yml +74 -0
- wetextprocessing-1.3.0/.gitignore +48 -0
- wetextprocessing-1.3.0/.pre-commit-config.yaml +23 -0
- wetextprocessing-1.3.0/.style.yapf +3 -0
- wetextprocessing-1.3.0/PKG-INFO +290 -0
- wetextprocessing-1.3.0/README.md +267 -0
- wetextprocessing-1.3.0/WeTextProcessing.egg-info/PKG-INFO +290 -0
- wetextprocessing-1.3.0/WeTextProcessing.egg-info/SOURCES.txt +2120 -0
- wetextprocessing-1.3.0/WeTextProcessing.egg-info/scm_file_list.json +2115 -0
- wetextprocessing-1.3.0/WeTextProcessing.egg-info/scm_version.json +8 -0
- wetextprocessing-1.3.0/conftest.py +19 -0
- wetextprocessing-1.3.0/docs/python-rule-architecture.md +106 -0
- wetextprocessing-1.3.0/itn/README.md +68 -0
- wetextprocessing-1.3.0/itn/arabic/data/measure/measurements.tsv +14 -0
- wetextprocessing-1.3.0/itn/arabic/data/money/currency_major.tsv +24 -0
- wetextprocessing-1.3.0/itn/arabic/data/money/currency_minor_plural.tsv +4 -0
- wetextprocessing-1.3.0/itn/arabic/data/money/currency_minor_singular.tsv +3 -0
- wetextprocessing-1.3.0/itn/arabic/data/money/local_currency.tsv +20 -0
- wetextprocessing-1.3.0/itn/arabic/data/number/digit.tsv +9 -0
- wetextprocessing-1.3.0/itn/arabic/data/number/digit_100.tsv +7 -0
- wetextprocessing-1.3.0/itn/arabic/data/number/digit_1000.tsv +7 -0
- wetextprocessing-1.3.0/itn/arabic/data/number/flops.tsv +72 -0
- wetextprocessing-1.3.0/itn/arabic/data/number/fraction_dual.tsv +9 -0
- wetextprocessing-1.3.0/itn/arabic/data/number/fraction_plural.tsv +8 -0
- wetextprocessing-1.3.0/itn/arabic/data/number/fraction_singular.tsv +9 -0
- wetextprocessing-1.3.0/itn/arabic/data/number/quantities.tsv +6 -0
- wetextprocessing-1.3.0/itn/arabic/data/number/teens.tsv +9 -0
- wetextprocessing-1.3.0/itn/arabic/data/number/tens.tsv +10 -0
- wetextprocessing-1.3.0/itn/arabic/inverse_normalizer.py +60 -0
- wetextprocessing-1.3.0/itn/arabic/rules/cardinal.py +66 -0
- wetextprocessing-1.3.0/itn/arabic/rules/decimal.py +58 -0
- wetextprocessing-1.3.0/itn/arabic/rules/fraction.py +72 -0
- wetextprocessing-1.3.0/itn/arabic/rules/measure.py +49 -0
- wetextprocessing-1.3.0/itn/arabic/rules/money.py +73 -0
- wetextprocessing-1.3.0/itn/arabic/rules/punctuation.py +29 -0
- wetextprocessing-1.3.0/itn/arabic/rules/word.py +28 -0
- wetextprocessing-1.3.0/itn/arabic/test/data/test_cases_cardinal.txt +34 -0
- wetextprocessing-1.3.0/itn/arabic/test/data/test_cases_decimal.txt +30 -0
- wetextprocessing-1.3.0/itn/arabic/test/data/test_cases_fraction.txt +20 -0
- wetextprocessing-1.3.0/itn/arabic/test/data/test_cases_measure.txt +15 -0
- wetextprocessing-1.3.0/itn/arabic/test/data/test_cases_money.txt +21 -0
- wetextprocessing-1.3.0/itn/arabic/test/data/test_cases_whitelist.txt +6 -0
- wetextprocessing-1.3.0/itn/arabic/test/normalizer_test.py +137 -0
- wetextprocessing-1.3.0/itn/armenian/data/currency.tsv +53 -0
- wetextprocessing-1.3.0/itn/armenian/data/measurement_dates.tsv +4 -0
- wetextprocessing-1.3.0/itn/armenian/data/measurements.tsv +48 -0
- wetextprocessing-1.3.0/itn/armenian/data/numbers/digit.tsv +11 -0
- wetextprocessing-1.3.0/itn/armenian/data/numbers/digits_no_one.tsv +10 -0
- wetextprocessing-1.3.0/itn/armenian/data/numbers/ties.tsv +10 -0
- wetextprocessing-1.3.0/itn/armenian/data/ordinals/digit.tsv +10 -0
- wetextprocessing-1.3.0/itn/armenian/data/time/hours.tsv +29 -0
- wetextprocessing-1.3.0/itn/armenian/data/time/minutes.tsv +65 -0
- wetextprocessing-1.3.0/itn/armenian/data/time/minutes_to.tsv +59 -0
- wetextprocessing-1.3.0/itn/armenian/data/time/to_hour.tsv +25 -0
- wetextprocessing-1.3.0/itn/armenian/data/whitelist.tsv +7 -0
- wetextprocessing-1.3.0/itn/armenian/inverse_normalizer.py +66 -0
- wetextprocessing-1.3.0/itn/armenian/rules/cardinal.py +67 -0
- wetextprocessing-1.3.0/itn/armenian/rules/decimal.py +67 -0
- wetextprocessing-1.3.0/itn/armenian/rules/fraction.py +40 -0
- wetextprocessing-1.3.0/itn/armenian/rules/measure.py +52 -0
- wetextprocessing-1.3.0/itn/armenian/rules/money.py +38 -0
- wetextprocessing-1.3.0/itn/armenian/rules/ordinal.py +37 -0
- wetextprocessing-1.3.0/itn/armenian/rules/punctuation.py +29 -0
- wetextprocessing-1.3.0/itn/armenian/rules/time.py +59 -0
- wetextprocessing-1.3.0/itn/armenian/rules/utils.py +58 -0
- wetextprocessing-1.3.0/itn/armenian/rules/whitelist.py +27 -0
- wetextprocessing-1.3.0/itn/armenian/rules/word.py +28 -0
- wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_cardinal.txt +23 -0
- wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_decimal.txt +14 -0
- wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_fraction.txt +24 -0
- wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_measure.txt +12 -0
- wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_money.txt +10 -0
- wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_ordinal.txt +11 -0
- wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_time.txt +9 -0
- wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_whitelist.txt +7 -0
- wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_word.txt +50 -0
- wetextprocessing-1.3.0/itn/armenian/test/normalizer_test.py +158 -0
- wetextprocessing-1.3.0/itn/chinese/inverse_normalizer.py +104 -0
- wetextprocessing-1.3.0/itn/chinese/rules/cardinal.py +151 -0
- wetextprocessing-1.3.0/itn/chinese/rules/date.py +63 -0
- wetextprocessing-1.3.0/itn/chinese/rules/fraction.py +57 -0
- wetextprocessing-1.3.0/itn/chinese/rules/license_plate.py +38 -0
- wetextprocessing-1.3.0/itn/chinese/rules/math.py +37 -0
- wetextprocessing-1.3.0/itn/chinese/rules/measure.py +101 -0
- wetextprocessing-1.3.0/itn/chinese/rules/money.py +53 -0
- wetextprocessing-1.3.0/itn/chinese/rules/postprocessor.py +32 -0
- wetextprocessing-1.3.0/itn/chinese/rules/time.py +59 -0
- wetextprocessing-1.3.0/itn/chinese/rules/train_number.py +37 -0
- wetextprocessing-1.3.0/itn/chinese/rules/whitelist.py +34 -0
- wetextprocessing-1.3.0/itn/chinese/test/normalizer_test.py +242 -0
- wetextprocessing-1.3.0/itn/english/inverse_normalizer.py +91 -0
- wetextprocessing-1.3.0/itn/english/rules/cardinal.py +111 -0
- wetextprocessing-1.3.0/itn/english/rules/date.py +163 -0
- wetextprocessing-1.3.0/itn/english/rules/decimal.py +82 -0
- wetextprocessing-1.3.0/itn/english/rules/electronic.py +79 -0
- wetextprocessing-1.3.0/itn/english/rules/measure.py +106 -0
- wetextprocessing-1.3.0/itn/english/rules/money.py +121 -0
- wetextprocessing-1.3.0/itn/english/rules/ordinal.py +50 -0
- wetextprocessing-1.3.0/itn/english/rules/telephone.py +147 -0
- wetextprocessing-1.3.0/itn/english/rules/time.py +138 -0
- wetextprocessing-1.3.0/itn/english/rules/whitelist.py +33 -0
- wetextprocessing-1.3.0/itn/english/rules/word.py +26 -0
- wetextprocessing-1.3.0/itn/english/test/normalizer_test.py +130 -0
- wetextprocessing-1.3.0/itn/french/data/electronic/domain.tsv +25 -0
- wetextprocessing-1.3.0/itn/french/data/electronic/server_name.tsv +17 -0
- wetextprocessing-1.3.0/itn/french/data/electronic/symbols.tsv +12 -0
- wetextprocessing-1.3.0/itn/french/data/fractions.tsv +41 -0
- wetextprocessing-1.3.0/itn/french/data/measurements/magnitudes.tsv +16 -0
- wetextprocessing-1.3.0/itn/french/data/measurements/measurements.tsv +27 -0
- wetextprocessing-1.3.0/itn/french/data/money/currency_major.tsv +40 -0
- wetextprocessing-1.3.0/itn/french/data/money/currency_minor.tsv +5 -0
- wetextprocessing-1.3.0/itn/french/data/months.tsv +12 -0
- wetextprocessing-1.3.0/itn/french/data/numbers/digit.tsv +10 -0
- wetextprocessing-1.3.0/itn/french/data/numbers/teen.tsv +6 -0
- wetextprocessing-1.3.0/itn/french/data/numbers/ties.tsv +13 -0
- wetextprocessing-1.3.0/itn/french/data/numbers/ties_unique.tsv +25 -0
- wetextprocessing-1.3.0/itn/french/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/itn/french/data/ordinals/digits_root_change.tsv +17 -0
- wetextprocessing-1.3.0/itn/french/data/ordinals/firsts.tsv +4 -0
- wetextprocessing-1.3.0/itn/french/data/ordinals/key_nouns.tsv +1 -0
- wetextprocessing-1.3.0/itn/french/data/ordinals/second.tsv +4 -0
- wetextprocessing-1.3.0/itn/french/data/roman/digits_large.tsv +9 -0
- wetextprocessing-1.3.0/itn/french/data/roman/hundreds_large.tsv +9 -0
- wetextprocessing-1.3.0/itn/french/data/roman/ties_large.tsv +9 -0
- wetextprocessing-1.3.0/itn/french/data/time/hour_to_night.tsv +12 -0
- wetextprocessing-1.3.0/itn/french/data/time/hours.tsv +26 -0
- wetextprocessing-1.3.0/itn/french/data/time/hours_to.tsv +25 -0
- wetextprocessing-1.3.0/itn/french/data/time/minutes.tsv +63 -0
- wetextprocessing-1.3.0/itn/french/data/time/minutes_to.tsv +59 -0
- wetextprocessing-1.3.0/itn/french/data/time/time_suffix_am.tsv +1 -0
- wetextprocessing-1.3.0/itn/french/data/time/time_suffix_pm.tsv +2 -0
- wetextprocessing-1.3.0/itn/french/data/whitelist.tsv +16 -0
- wetextprocessing-1.3.0/itn/french/inverse_normalizer.py +74 -0
- wetextprocessing-1.3.0/itn/french/rules/cardinal.py +115 -0
- wetextprocessing-1.3.0/itn/french/rules/date.py +45 -0
- wetextprocessing-1.3.0/itn/french/rules/decimal.py +70 -0
- wetextprocessing-1.3.0/itn/french/rules/electronic.py +67 -0
- wetextprocessing-1.3.0/itn/french/rules/fraction.py +55 -0
- wetextprocessing-1.3.0/itn/french/rules/measure.py +55 -0
- wetextprocessing-1.3.0/itn/french/rules/money.py +67 -0
- wetextprocessing-1.3.0/itn/french/rules/ordinal.py +63 -0
- wetextprocessing-1.3.0/itn/french/rules/punctuation.py +29 -0
- wetextprocessing-1.3.0/itn/french/rules/telephone.py +59 -0
- wetextprocessing-1.3.0/itn/french/rules/time.py +70 -0
- wetextprocessing-1.3.0/itn/french/rules/whitelist.py +27 -0
- wetextprocessing-1.3.0/itn/french/rules/word.py +29 -0
- wetextprocessing-1.3.0/itn/french/test/data/test_cases_cardinal.txt +106 -0
- wetextprocessing-1.3.0/itn/french/test/data/test_cases_date.txt +6 -0
- wetextprocessing-1.3.0/itn/french/test/data/test_cases_decimal.txt +15 -0
- wetextprocessing-1.3.0/itn/french/test/data/test_cases_electronic.txt +10 -0
- wetextprocessing-1.3.0/itn/french/test/data/test_cases_fraction.txt +30 -0
- wetextprocessing-1.3.0/itn/french/test/data/test_cases_measure.txt +15 -0
- wetextprocessing-1.3.0/itn/french/test/data/test_cases_money.txt +22 -0
- wetextprocessing-1.3.0/itn/french/test/data/test_cases_ordinal.txt +23 -0
- wetextprocessing-1.3.0/itn/french/test/data/test_cases_telephone.txt +5 -0
- wetextprocessing-1.3.0/itn/french/test/data/test_cases_time.txt +18 -0
- wetextprocessing-1.3.0/itn/french/test/data/test_cases_whitelist.txt +8 -0
- wetextprocessing-1.3.0/itn/french/test/data/test_cases_word.txt +49 -0
- wetextprocessing-1.3.0/itn/french/test/normalizer_test.py +155 -0
- wetextprocessing-1.3.0/itn/german/inverse_normalizer.py +85 -0
- wetextprocessing-1.3.0/itn/german/rules/cardinal.py +48 -0
- wetextprocessing-1.3.0/itn/german/rules/date.py +63 -0
- wetextprocessing-1.3.0/itn/german/rules/decimal.py +67 -0
- wetextprocessing-1.3.0/itn/german/rules/electronic.py +58 -0
- wetextprocessing-1.3.0/itn/german/rules/fraction.py +75 -0
- wetextprocessing-1.3.0/itn/german/rules/measure.py +80 -0
- wetextprocessing-1.3.0/itn/german/rules/money.py +87 -0
- wetextprocessing-1.3.0/itn/german/rules/ordinal.py +31 -0
- wetextprocessing-1.3.0/itn/german/rules/punctuation.py +29 -0
- wetextprocessing-1.3.0/itn/german/rules/telephone.py +45 -0
- wetextprocessing-1.3.0/itn/german/rules/time.py +102 -0
- wetextprocessing-1.3.0/itn/german/rules/whitelist.py +30 -0
- wetextprocessing-1.3.0/itn/german/rules/word.py +33 -0
- wetextprocessing-1.3.0/itn/german/test/calendar_test.py +147 -0
- wetextprocessing-1.3.0/itn/german/test/data/test_cases_cardinal.txt +62 -0
- wetextprocessing-1.3.0/itn/german/test/data/test_cases_date.txt +22 -0
- wetextprocessing-1.3.0/itn/german/test/data/test_cases_decimal.txt +10 -0
- wetextprocessing-1.3.0/itn/german/test/data/test_cases_electronic.txt +9 -0
- wetextprocessing-1.3.0/itn/german/test/data/test_cases_fraction.txt +34 -0
- wetextprocessing-1.3.0/itn/german/test/data/test_cases_measure.txt +28 -0
- wetextprocessing-1.3.0/itn/german/test/data/test_cases_money.txt +23 -0
- wetextprocessing-1.3.0/itn/german/test/data/test_cases_ordinal.txt +20 -0
- wetextprocessing-1.3.0/itn/german/test/data/test_cases_telephone.txt +1 -0
- wetextprocessing-1.3.0/itn/german/test/data/test_cases_time.txt +24 -0
- wetextprocessing-1.3.0/itn/german/test/data/test_cases_whitelist.txt +6 -0
- wetextprocessing-1.3.0/itn/german/test/data/test_cases_word.txt +49 -0
- wetextprocessing-1.3.0/itn/german/test/financial_test.py +284 -0
- wetextprocessing-1.3.0/itn/german/test/inverse_normalizer_test.py +92 -0
- wetextprocessing-1.3.0/itn/german/test/numeric_test.py +219 -0
- wetextprocessing-1.3.0/itn/german/test/text_test.py +281 -0
- wetextprocessing-1.3.0/itn/german/test/time_test.py +70 -0
- wetextprocessing-1.3.0/itn/hebrew/data/decimals/minutes_exception.tsv +2 -0
- wetextprocessing-1.3.0/itn/hebrew/data/measurements.tsv +45 -0
- wetextprocessing-1.3.0/itn/hebrew/data/months.tsv +13 -0
- wetextprocessing-1.3.0/itn/hebrew/data/months_name2number.tsv +12 -0
- wetextprocessing-1.3.0/itn/hebrew/data/months_ordinal2number.tsv +12 -0
- wetextprocessing-1.3.0/itn/hebrew/data/numbers/decimal_fractions.tsv +6 -0
- wetextprocessing-1.3.0/itn/hebrew/data/numbers/digit.tsv +20 -0
- wetextprocessing-1.3.0/itn/hebrew/data/numbers/hundreds_exception.tsv +2 -0
- wetextprocessing-1.3.0/itn/hebrew/data/numbers/millions_exception.tsv +1 -0
- wetextprocessing-1.3.0/itn/hebrew/data/numbers/teen.tsv +21 -0
- wetextprocessing-1.3.0/itn/hebrew/data/numbers/thousands.tsv +8 -0
- wetextprocessing-1.3.0/itn/hebrew/data/numbers/thousands_exception.tsv +2 -0
- wetextprocessing-1.3.0/itn/hebrew/data/numbers/ties.tsv +8 -0
- wetextprocessing-1.3.0/itn/hebrew/data/numbers/viable_hours.tsv +15 -0
- wetextprocessing-1.3.0/itn/hebrew/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/itn/hebrew/data/ordinals/digit.tsv +10 -0
- wetextprocessing-1.3.0/itn/hebrew/data/prefix.tsv +17 -0
- wetextprocessing-1.3.0/itn/hebrew/data/spaced_measurements.tsv +17 -0
- wetextprocessing-1.3.0/itn/hebrew/data/time/day_suffix.tsv +2 -0
- wetextprocessing-1.3.0/itn/hebrew/data/time/evening_suffix.tsv +2 -0
- wetextprocessing-1.3.0/itn/hebrew/data/time/hour_to_evening.tsv +7 -0
- wetextprocessing-1.3.0/itn/hebrew/data/time/hour_to_night.tsv +9 -0
- wetextprocessing-1.3.0/itn/hebrew/data/time/hour_to_noon.tsv +7 -0
- wetextprocessing-1.3.0/itn/hebrew/data/time/midnight_to_hour.tsv +1 -0
- wetextprocessing-1.3.0/itn/hebrew/data/time/minute_to.tsv +58 -0
- wetextprocessing-1.3.0/itn/hebrew/data/time/minute_to_verbose.tsv +6 -0
- wetextprocessing-1.3.0/itn/hebrew/data/time/minute_verbose.tsv +8 -0
- wetextprocessing-1.3.0/itn/hebrew/data/time/night_suffix.tsv +1 -0
- wetextprocessing-1.3.0/itn/hebrew/data/time/noon_suffix.tsv +3 -0
- wetextprocessing-1.3.0/itn/hebrew/data/time/to_hour.tsv +13 -0
- wetextprocessing-1.3.0/itn/hebrew/data/whitelist.tsv +20 -0
- wetextprocessing-1.3.0/itn/hebrew/inverse_normalizer.py +64 -0
- wetextprocessing-1.3.0/itn/hebrew/rules/cardinal.py +94 -0
- wetextprocessing-1.3.0/itn/hebrew/rules/date.py +62 -0
- wetextprocessing-1.3.0/itn/hebrew/rules/decimal.py +74 -0
- wetextprocessing-1.3.0/itn/hebrew/rules/measure.py +53 -0
- wetextprocessing-1.3.0/itn/hebrew/rules/punctuation.py +28 -0
- wetextprocessing-1.3.0/itn/hebrew/rules/time.py +101 -0
- wetextprocessing-1.3.0/itn/hebrew/rules/utils.py +78 -0
- wetextprocessing-1.3.0/itn/hebrew/rules/whitelist.py +30 -0
- wetextprocessing-1.3.0/itn/hebrew/rules/word.py +28 -0
- wetextprocessing-1.3.0/itn/hebrew/test/data/test_cases_cardinal.txt +138 -0
- wetextprocessing-1.3.0/itn/hebrew/test/data/test_cases_date.txt +29 -0
- wetextprocessing-1.3.0/itn/hebrew/test/data/test_cases_decimal.txt +66 -0
- wetextprocessing-1.3.0/itn/hebrew/test/data/test_cases_measure.txt +10 -0
- wetextprocessing-1.3.0/itn/hebrew/test/data/test_cases_time.txt +34 -0
- wetextprocessing-1.3.0/itn/hebrew/test/data/test_cases_whitelist.txt +4 -0
- wetextprocessing-1.3.0/itn/hebrew/test/data/test_full_sentences.txt +56 -0
- wetextprocessing-1.3.0/itn/hebrew/test/normalizer_test.py +178 -0
- wetextprocessing-1.3.0/itn/hindi/data/address/cities.tsv +36 -0
- wetextprocessing-1.3.0/itn/hindi/data/address/context_cues.tsv +85 -0
- wetextprocessing-1.3.0/itn/hindi/data/address/digit_passthrough.tsv +10 -0
- wetextprocessing-1.3.0/itn/hindi/data/address/ordinals.tsv +167 -0
- wetextprocessing-1.3.0/itn/hindi/data/address/special_characters.tsv +2 -0
- wetextprocessing-1.3.0/itn/hindi/data/address/states.tsv +36 -0
- wetextprocessing-1.3.0/itn/hindi/data/date/century.tsv +5 -0
- wetextprocessing-1.3.0/itn/hindi/data/date/months.tsv +14 -0
- wetextprocessing-1.3.0/itn/hindi/data/measure/measurements.tsv +295 -0
- wetextprocessing-1.3.0/itn/hindi/data/money/currency.tsv +181 -0
- wetextprocessing-1.3.0/itn/hindi/data/numbers/digit.tsv +13 -0
- wetextprocessing-1.3.0/itn/hindi/data/numbers/hundred.tsv +1 -0
- wetextprocessing-1.3.0/itn/hindi/data/numbers/labels_exception.tsv +4 -0
- wetextprocessing-1.3.0/itn/hindi/data/numbers/paune.tsv +231 -0
- wetextprocessing-1.3.0/itn/hindi/data/numbers/teens_and_ties.tsv +220 -0
- wetextprocessing-1.3.0/itn/hindi/data/numbers/thousands.tsv +8 -0
- wetextprocessing-1.3.0/itn/hindi/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/itn/hindi/data/ordinals/digit.tsv +6 -0
- wetextprocessing-1.3.0/itn/hindi/data/ordinals/digit_fem.tsv +6 -0
- wetextprocessing-1.3.0/itn/hindi/data/ordinals/hundred_digit.tsv +12 -0
- wetextprocessing-1.3.0/itn/hindi/data/ordinals/hundred_digit_fem.tsv +12 -0
- wetextprocessing-1.3.0/itn/hindi/data/ordinals/morph_features.tsv +6 -0
- wetextprocessing-1.3.0/itn/hindi/data/ordinals/suffix.tsv +4 -0
- wetextprocessing-1.3.0/itn/hindi/data/ordinals/suffix_fem.tsv +2 -0
- wetextprocessing-1.3.0/itn/hindi/data/ordinals/teens_and_ties.tsv +1 -0
- wetextprocessing-1.3.0/itn/hindi/data/ordinals/teens_and_ties_fem.tsv +1 -0
- wetextprocessing-1.3.0/itn/hindi/data/telephone/context_cues.tsv +15 -0
- wetextprocessing-1.3.0/itn/hindi/data/telephone/country_codes.tsv +1 -0
- wetextprocessing-1.3.0/itn/hindi/data/telephone/eng_digit.tsv +9 -0
- wetextprocessing-1.3.0/itn/hindi/data/telephone/eng_zero.tsv +1 -0
- wetextprocessing-1.3.0/itn/hindi/data/telephone/teens_and_ties_eng_to_hin.tsv +90 -0
- wetextprocessing-1.3.0/itn/hindi/data/time/hour_for_paune.tsv +15 -0
- wetextprocessing-1.3.0/itn/hindi/data/whitelist/whitelist.tsv +7 -0
- wetextprocessing-1.3.0/itn/hindi/inverse_normalizer.py +78 -0
- wetextprocessing-1.3.0/itn/hindi/rules/cardinal.py +93 -0
- wetextprocessing-1.3.0/itn/hindi/rules/date.py +64 -0
- wetextprocessing-1.3.0/itn/hindi/rules/decimal.py +57 -0
- wetextprocessing-1.3.0/itn/hindi/rules/fraction.py +55 -0
- wetextprocessing-1.3.0/itn/hindi/rules/measure.py +83 -0
- wetextprocessing-1.3.0/itn/hindi/rules/money.py +81 -0
- wetextprocessing-1.3.0/itn/hindi/rules/ordinal.py +40 -0
- wetextprocessing-1.3.0/itn/hindi/rules/punctuation.py +31 -0
- wetextprocessing-1.3.0/itn/hindi/rules/telephone.py +65 -0
- wetextprocessing-1.3.0/itn/hindi/rules/time.py +53 -0
- wetextprocessing-1.3.0/itn/hindi/rules/utils.py +52 -0
- wetextprocessing-1.3.0/itn/hindi/rules/whitelist.py +32 -0
- wetextprocessing-1.3.0/itn/hindi/rules/word.py +30 -0
- wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_address.txt +25 -0
- wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_cardinal.txt +54 -0
- wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_date.txt +42 -0
- wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_decimal.txt +13 -0
- wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_fraction.txt +31 -0
- wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_measure.txt +48 -0
- wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_money.txt +50 -0
- wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_ordinal.txt +13 -0
- wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_telephone.txt +28 -0
- wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_time.txt +25 -0
- wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_whitelist.txt +8 -0
- wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_word.txt +15 -0
- wetextprocessing-1.3.0/itn/hindi/test/normalizer_test.py +127 -0
- wetextprocessing-1.3.0/itn/hindi_english/data/measurements.tsv +2 -0
- wetextprocessing-1.3.0/itn/hindi_english/inverse_normalizer.py +93 -0
- wetextprocessing-1.3.0/itn/hindi_english/rules/english.py +175 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_address.txt +41 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_cardinal.txt +61 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_date.txt +37 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_date_cased.txt +70 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_decimal.txt +25 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_electronic.txt +17 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_fraction.txt +39 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_measure.txt +25 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_money.txt +41 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_ordinal.txt +26 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_telephone.txt +39 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_time.txt +29 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_whitelist.txt +17 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_word.txt +19 -0
- wetextprocessing-1.3.0/itn/hindi_english/test/normalizer_test.py +211 -0
- wetextprocessing-1.3.0/itn/hungarian/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/hungarian/data/dates/days.tsv +31 -0
- wetextprocessing-1.3.0/itn/hungarian/data/dates/months.tsv +12 -0
- wetextprocessing-1.3.0/itn/hungarian/data/electronic/domain.tsv +15 -0
- wetextprocessing-1.3.0/itn/hungarian/data/electronic/server_name.tsv +15 -0
- wetextprocessing-1.3.0/itn/hungarian/data/electronic/symbols.tsv +23 -0
- wetextprocessing-1.3.0/itn/hungarian/data/inflection/endings.tsv +51 -0
- wetextprocessing-1.3.0/itn/hungarian/data/inflection/plural_endings.tsv +39 -0
- wetextprocessing-1.3.0/itn/hungarian/data/inflection/word_endings.tsv +1 -0
- wetextprocessing-1.3.0/itn/hungarian/data/measures/measurements.tsv +52 -0
- wetextprocessing-1.3.0/itn/hungarian/data/money/currency.tsv +11 -0
- wetextprocessing-1.3.0/itn/hungarian/data/money/currency_minor.tsv +11 -0
- wetextprocessing-1.3.0/itn/hungarian/data/number/digit.tsv +9 -0
- wetextprocessing-1.3.0/itn/hungarian/data/number/digit_inline.tsv +9 -0
- wetextprocessing-1.3.0/itn/hungarian/data/number/quantities.tsv +21 -0
- wetextprocessing-1.3.0/itn/hungarian/data/number/tens.tsv +9 -0
- wetextprocessing-1.3.0/itn/hungarian/data/number/tens_inline.tsv +9 -0
- wetextprocessing-1.3.0/itn/hungarian/data/number/zero.tsv +1 -0
- wetextprocessing-1.3.0/itn/hungarian/data/ordinals/endings.tsv +23 -0
- wetextprocessing-1.3.0/itn/hungarian/data/ordinals/exceptional.tsv +4 -0
- wetextprocessing-1.3.0/itn/hungarian/data/telephone/area_codes.tsv +71 -0
- wetextprocessing-1.3.0/itn/hungarian/data/telephone/country_codes.tsv +213 -0
- wetextprocessing-1.3.0/itn/hungarian/data/time/time_zone.tsv +9 -0
- wetextprocessing-1.3.0/itn/hungarian/data/whitelist.tsv +24 -0
- wetextprocessing-1.3.0/itn/hungarian/data/whitelist_inflect.tsv +39 -0
- wetextprocessing-1.3.0/itn/hungarian/data/whitelist_inflect_sg.tsv +14 -0
- wetextprocessing-1.3.0/itn/hungarian/inverse_normalizer.py +93 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/cardinal.py +99 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/date.py +66 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/decimal.py +80 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/electronic.py +71 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/fraction.py +41 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/measure.py +74 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/money.py +82 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/ordinal.py +42 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/punctuation.py +44 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/telephone.py +68 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/time.py +99 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/utils.py +143 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/whitelist.py +50 -0
- wetextprocessing-1.3.0/itn/hungarian/rules/word.py +38 -0
- wetextprocessing-1.3.0/itn/hungarian/test/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/hungarian/test/normalizer_test.py +214 -0
- wetextprocessing-1.3.0/itn/italian/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/italian/data/electronic/domain.tsv +14 -0
- wetextprocessing-1.3.0/itn/italian/data/electronic/server_name.tsv +11 -0
- wetextprocessing-1.3.0/itn/italian/data/electronic/symbols.tsv +22 -0
- wetextprocessing-1.3.0/itn/italian/data/measure/denominators.tsv +4 -0
- wetextprocessing-1.3.0/itn/italian/data/measure/units.tsv +51 -0
- wetextprocessing-1.3.0/itn/italian/data/money/currency.tsv +11 -0
- wetextprocessing-1.3.0/itn/italian/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/itn/italian/data/numbers/hundreds.tsv +9 -0
- wetextprocessing-1.3.0/itn/italian/data/numbers/quantities.tsv +4 -0
- wetextprocessing-1.3.0/itn/italian/data/numbers/teen.tsv +10 -0
- wetextprocessing-1.3.0/itn/italian/data/numbers/tens.tsv +8 -0
- wetextprocessing-1.3.0/itn/italian/data/whitelist.tsv +19 -0
- wetextprocessing-1.3.0/itn/italian/inverse_normalizer.py +77 -0
- wetextprocessing-1.3.0/itn/italian/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/italian/rules/cardinal.py +100 -0
- wetextprocessing-1.3.0/itn/italian/rules/decimal.py +59 -0
- wetextprocessing-1.3.0/itn/italian/rules/electronic.py +70 -0
- wetextprocessing-1.3.0/itn/italian/rules/measure.py +52 -0
- wetextprocessing-1.3.0/itn/italian/rules/money.py +78 -0
- wetextprocessing-1.3.0/itn/italian/rules/punctuation.py +27 -0
- wetextprocessing-1.3.0/itn/italian/rules/time.py +84 -0
- wetextprocessing-1.3.0/itn/italian/rules/utils.py +49 -0
- wetextprocessing-1.3.0/itn/italian/rules/whitelist.py +31 -0
- wetextprocessing-1.3.0/itn/italian/rules/word.py +27 -0
- wetextprocessing-1.3.0/itn/italian/test/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/italian/test/normalizer_test.py +277 -0
- wetextprocessing-1.3.0/itn/japanese/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/japanese/data/default/blacklist.tsv +0 -0
- wetextprocessing-1.3.0/itn/japanese/inverse_normalizer.py +111 -0
- wetextprocessing-1.3.0/itn/japanese/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/japanese/rules/cardinal.py +143 -0
- wetextprocessing-1.3.0/itn/japanese/rules/char.py +31 -0
- wetextprocessing-1.3.0/itn/japanese/rules/date.py +68 -0
- wetextprocessing-1.3.0/itn/japanese/rules/fraction.py +66 -0
- wetextprocessing-1.3.0/itn/japanese/rules/math.py +40 -0
- wetextprocessing-1.3.0/itn/japanese/rules/measure.py +49 -0
- wetextprocessing-1.3.0/itn/japanese/rules/money.py +48 -0
- wetextprocessing-1.3.0/itn/japanese/rules/ordinal.py +36 -0
- wetextprocessing-1.3.0/itn/japanese/rules/postprocessor.py +46 -0
- wetextprocessing-1.3.0/itn/japanese/rules/preprocessor.py +31 -0
- wetextprocessing-1.3.0/itn/japanese/rules/time.py +52 -0
- wetextprocessing-1.3.0/itn/japanese/rules/whitelist.py +35 -0
- wetextprocessing-1.3.0/itn/japanese/test/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/japanese/test/normalizer_test.py +215 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/data/numbers/digit_bi.tsv +9 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/data/numbers/digit_e.tsv +9 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/data/numbers/hundreds.tsv +9 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/data/numbers/tens.tsv +8 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/data/time/hours.tsv +12 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/data/time/minutes.tsv +59 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/data/whitelist.tsv +18 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/inverse_normalizer.py +53 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/rules/cardinal.py +131 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/rules/punctuation.py +28 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/rules/time.py +39 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/rules/utils.py +39 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/rules/whitelist.py +32 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/rules/word.py +28 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/test/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/kinyarwanda/test/normalizer_test.py +194 -0
- wetextprocessing-1.3.0/itn/korean/__init__.py +3 -0
- wetextprocessing-1.3.0/itn/korean/data/__init__.py +13 -0
- wetextprocessing-1.3.0/itn/korean/data/currency.tsv +9 -0
- wetextprocessing-1.3.0/itn/korean/data/measure_units.tsv +61 -0
- wetextprocessing-1.3.0/itn/korean/data/months.tsv +12 -0
- wetextprocessing-1.3.0/itn/korean/data/numbers/__init__.py +13 -0
- wetextprocessing-1.3.0/itn/korean/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/itn/korean/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/itn/korean/data/ordinals/__init__.py +13 -0
- wetextprocessing-1.3.0/itn/korean/data/ordinals/counter_suffix.tsv +17 -0
- wetextprocessing-1.3.0/itn/korean/data/ordinals/digit.tsv +9 -0
- wetextprocessing-1.3.0/itn/korean/data/time/__init__.py +13 -0
- wetextprocessing-1.3.0/itn/korean/data/time/ten_prefix.tsv +4 -0
- wetextprocessing-1.3.0/itn/korean/data/time/time_hours.tsv +23 -0
- wetextprocessing-1.3.0/itn/korean/data/whitelist.tsv +35 -0
- wetextprocessing-1.3.0/itn/korean/inverse_normalizer.py +76 -0
- wetextprocessing-1.3.0/itn/korean/rules/__init__.py +3 -0
- wetextprocessing-1.3.0/itn/korean/rules/cardinal.py +44 -0
- wetextprocessing-1.3.0/itn/korean/rules/date.py +42 -0
- wetextprocessing-1.3.0/itn/korean/rules/decimal.py +43 -0
- wetextprocessing-1.3.0/itn/korean/rules/fraction.py +48 -0
- wetextprocessing-1.3.0/itn/korean/rules/measure.py +52 -0
- wetextprocessing-1.3.0/itn/korean/rules/money.py +34 -0
- wetextprocessing-1.3.0/itn/korean/rules/ordinal.py +49 -0
- wetextprocessing-1.3.0/itn/korean/rules/telephone.py +30 -0
- wetextprocessing-1.3.0/itn/korean/rules/time.py +61 -0
- wetextprocessing-1.3.0/itn/korean/rules/utils.py +22 -0
- wetextprocessing-1.3.0/itn/korean/rules/whitelist.py +28 -0
- wetextprocessing-1.3.0/itn/korean/rules/word.py +27 -0
- wetextprocessing-1.3.0/itn/korean/test/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/korean/test/data/test_cases_cardinal.txt +37 -0
- wetextprocessing-1.3.0/itn/korean/test/data/test_cases_date.txt +15 -0
- wetextprocessing-1.3.0/itn/korean/test/data/test_cases_decimal.txt +14 -0
- wetextprocessing-1.3.0/itn/korean/test/data/test_cases_fraction.txt +29 -0
- wetextprocessing-1.3.0/itn/korean/test/data/test_cases_measure.txt +46 -0
- wetextprocessing-1.3.0/itn/korean/test/data/test_cases_money.txt +29 -0
- wetextprocessing-1.3.0/itn/korean/test/data/test_cases_ordinal.txt +17 -0
- wetextprocessing-1.3.0/itn/korean/test/data/test_cases_telephone.txt +7 -0
- wetextprocessing-1.3.0/itn/korean/test/data/test_cases_time.txt +15 -0
- wetextprocessing-1.3.0/itn/korean/test/test_inverse_normalizer.py +184 -0
- wetextprocessing-1.3.0/itn/main.py +23 -0
- wetextprocessing-1.3.0/itn/marathi/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/marathi/data/date/eras.tsv +2 -0
- wetextprocessing-1.3.0/itn/marathi/data/date/months.tsv +12 -0
- wetextprocessing-1.3.0/itn/marathi/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/itn/marathi/data/numbers/tens.tsv +90 -0
- wetextprocessing-1.3.0/itn/marathi/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/itn/marathi/inverse_normalizer.py +56 -0
- wetextprocessing-1.3.0/itn/marathi/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/marathi/rules/cardinal.py +76 -0
- wetextprocessing-1.3.0/itn/marathi/rules/date.py +63 -0
- wetextprocessing-1.3.0/itn/marathi/rules/decimal.py +62 -0
- wetextprocessing-1.3.0/itn/marathi/rules/punctuation.py +29 -0
- wetextprocessing-1.3.0/itn/marathi/rules/time.py +69 -0
- wetextprocessing-1.3.0/itn/marathi/rules/utils.py +62 -0
- wetextprocessing-1.3.0/itn/marathi/rules/word.py +28 -0
- wetextprocessing-1.3.0/itn/marathi/test/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/marathi/test/data/test_cases_cardinal.txt +33 -0
- wetextprocessing-1.3.0/itn/marathi/test/data/test_cases_date.txt +29 -0
- wetextprocessing-1.3.0/itn/marathi/test/data/test_cases_decimal.txt +29 -0
- wetextprocessing-1.3.0/itn/marathi/test/data/test_cases_time.txt +29 -0
- wetextprocessing-1.3.0/itn/marathi/test/data/test_cases_word.txt +11 -0
- wetextprocessing-1.3.0/itn/marathi/test/normalizer_test.py +164 -0
- wetextprocessing-1.3.0/itn/portuguese/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/portuguese/data/currency_plural.tsv +5 -0
- wetextprocessing-1.3.0/itn/portuguese/data/currency_singular.tsv +5 -0
- wetextprocessing-1.3.0/itn/portuguese/data/electronic/domain.tsv +26 -0
- wetextprocessing-1.3.0/itn/portuguese/data/electronic/server_name.tsv +11 -0
- wetextprocessing-1.3.0/itn/portuguese/data/electronic/symbols.tsv +6 -0
- wetextprocessing-1.3.0/itn/portuguese/data/measurements_plural.tsv +56 -0
- wetextprocessing-1.3.0/itn/portuguese/data/measurements_singular.tsv +55 -0
- wetextprocessing-1.3.0/itn/portuguese/data/months.tsv +12 -0
- wetextprocessing-1.3.0/itn/portuguese/data/numbers/digit.tsv +11 -0
- wetextprocessing-1.3.0/itn/portuguese/data/numbers/hundreds.tsv +17 -0
- wetextprocessing-1.3.0/itn/portuguese/data/numbers/onehundred.tsv +1 -0
- wetextprocessing-1.3.0/itn/portuguese/data/numbers/teen.tsv +11 -0
- wetextprocessing-1.3.0/itn/portuguese/data/numbers/ties.tsv +8 -0
- wetextprocessing-1.3.0/itn/portuguese/data/numbers/twenties.tsv +9 -0
- wetextprocessing-1.3.0/itn/portuguese/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/itn/portuguese/data/ordinals/digit.tsv +18 -0
- wetextprocessing-1.3.0/itn/portuguese/data/ordinals/hundreds.tsv +28 -0
- wetextprocessing-1.3.0/itn/portuguese/data/ordinals/ties.tsv +20 -0
- wetextprocessing-1.3.0/itn/portuguese/data/time/hours_to.tsv +23 -0
- wetextprocessing-1.3.0/itn/portuguese/data/time/minutes_to.tsv +59 -0
- wetextprocessing-1.3.0/itn/portuguese/data/time/time_suffix_am.tsv +2 -0
- wetextprocessing-1.3.0/itn/portuguese/data/time/time_suffix_pm.tsv +2 -0
- wetextprocessing-1.3.0/itn/portuguese/data/whitelist.tsv +5 -0
- wetextprocessing-1.3.0/itn/portuguese/inverse_normalizer.py +69 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/cardinal.py +117 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/date.py +64 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/decimal.py +76 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/electronic.py +69 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/measure.py +61 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/money.py +89 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/ordinal.py +56 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/punctuation.py +32 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/telephone.py +56 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/time.py +107 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/whitelist.py +32 -0
- wetextprocessing-1.3.0/itn/portuguese/rules/word.py +31 -0
- wetextprocessing-1.3.0/itn/portuguese/test/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_cardinal.txt +70 -0
- wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_date.txt +15 -0
- wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_decimal.txt +26 -0
- wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_electronic.txt +13 -0
- wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_measure.txt +12 -0
- wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_money.txt +25 -0
- wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_ordinal.txt +19 -0
- wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_telephone.txt +27 -0
- wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_time.txt +19 -0
- wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_whitelist.txt +6 -0
- wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_word.txt +49 -0
- wetextprocessing-1.3.0/itn/portuguese/test/normalizer_test.py +156 -0
- wetextprocessing-1.3.0/itn/russian/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/russian/data/currency/currency_plural.tsv +53 -0
- wetextprocessing-1.3.0/itn/russian/data/currency/currency_singular.tsv +48 -0
- wetextprocessing-1.3.0/itn/russian/data/electronic/domain.tsv +9 -0
- wetextprocessing-1.3.0/itn/russian/data/electronic/server_name.tsv +17 -0
- wetextprocessing-1.3.0/itn/russian/data/electronic/symbols.tsv +28 -0
- wetextprocessing-1.3.0/itn/russian/data/latin_to_cyrillic.tsv +38 -0
- wetextprocessing-1.3.0/itn/russian/data/measurements.tsv +321 -0
- wetextprocessing-1.3.0/itn/russian/data/months/abbr.tsv +12 -0
- wetextprocessing-1.3.0/itn/russian/data/months/abbr_to_name.tsv +62 -0
- wetextprocessing-1.3.0/itn/russian/data/months/numbers.tsv +21 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/1_cardinals_nominative.tsv +50 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/2_cardinals_genitive.tsv +45 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/3_cardinals_dative.tsv +45 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/4_cardinals_accusative.tsv +48 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/5_cardinals_instrumental.tsv +47 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/6_cardinals_prepositional.tsv +46 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/cardinal_endings.tsv +6 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/cardinals_alternatives.tsv +16 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/cardinals_nominative_case.tsv +37 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/decimal_delimiter.tsv +15 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/decimal_endings.tsv +37 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/digits_nominative_case.tsv +10 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/ordinal_endings.tsv +40 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/ordinals.tsv +752 -0
- wetextprocessing-1.3.0/itn/russian/data/numbers/quantity.tsv +16 -0
- wetextprocessing-1.3.0/itn/russian/data/time/increment_hour_cardinal.tsv +24 -0
- wetextprocessing-1.3.0/itn/russian/data/time/increment_hour_ordinal.tsv +24 -0
- wetextprocessing-1.3.0/itn/russian/data/time/minutes_to_hour.tsv +25 -0
- wetextprocessing-1.3.0/itn/russian/data/time/time_convert.tsv +12 -0
- wetextprocessing-1.3.0/itn/russian/data/whitelist.tsv +37 -0
- wetextprocessing-1.3.0/itn/russian/inverse_normalizer.py +71 -0
- wetextprocessing-1.3.0/itn/russian/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/russian/rules/cardinal.py +57 -0
- wetextprocessing-1.3.0/itn/russian/rules/date.py +78 -0
- wetextprocessing-1.3.0/itn/russian/rules/decimal.py +55 -0
- wetextprocessing-1.3.0/itn/russian/rules/electronic.py +67 -0
- wetextprocessing-1.3.0/itn/russian/rules/measure.py +45 -0
- wetextprocessing-1.3.0/itn/russian/rules/money.py +53 -0
- wetextprocessing-1.3.0/itn/russian/rules/ordinal.py +40 -0
- wetextprocessing-1.3.0/itn/russian/rules/punctuation.py +40 -0
- wetextprocessing-1.3.0/itn/russian/rules/telephone.py +45 -0
- wetextprocessing-1.3.0/itn/russian/rules/time.py +83 -0
- wetextprocessing-1.3.0/itn/russian/rules/utils.py +151 -0
- wetextprocessing-1.3.0/itn/russian/rules/whitelist.py +29 -0
- wetextprocessing-1.3.0/itn/russian/rules/word.py +30 -0
- wetextprocessing-1.3.0/itn/russian/test/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/russian/test/data/test_cases_cardinal.txt +30 -0
- wetextprocessing-1.3.0/itn/russian/test/data/test_cases_date.txt +15 -0
- wetextprocessing-1.3.0/itn/russian/test/data/test_cases_decimal.txt +16 -0
- wetextprocessing-1.3.0/itn/russian/test/data/test_cases_electronic.txt +4 -0
- wetextprocessing-1.3.0/itn/russian/test/data/test_cases_measure.txt +16 -0
- wetextprocessing-1.3.0/itn/russian/test/data/test_cases_money.txt +9 -0
- wetextprocessing-1.3.0/itn/russian/test/data/test_cases_ordinal.txt +15 -0
- wetextprocessing-1.3.0/itn/russian/test/data/test_cases_ordinal_hard.txt +57 -0
- wetextprocessing-1.3.0/itn/russian/test/data/test_cases_telephone.txt +1 -0
- wetextprocessing-1.3.0/itn/russian/test/data/test_cases_time.txt +9 -0
- wetextprocessing-1.3.0/itn/russian/test/data/test_cases_whitelist.txt +4 -0
- wetextprocessing-1.3.0/itn/russian/test/data/test_cases_word.txt +50 -0
- wetextprocessing-1.3.0/itn/russian/test/normalizer_test.py +193 -0
- wetextprocessing-1.3.0/itn/spanish/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/spanish/data/dates/months.tsv +12 -0
- wetextprocessing-1.3.0/itn/spanish/data/dates/months_cased.tsv +12 -0
- wetextprocessing-1.3.0/itn/spanish/data/dates/year_suffix.tsv +11 -0
- wetextprocessing-1.3.0/itn/spanish/data/dates/year_suffix_cased.tsv +11 -0
- wetextprocessing-1.3.0/itn/spanish/data/electronic/domain.tsv +27 -0
- wetextprocessing-1.3.0/itn/spanish/data/electronic/server_name.tsv +17 -0
- wetextprocessing-1.3.0/itn/spanish/data/electronic/symbols.tsv +22 -0
- wetextprocessing-1.3.0/itn/spanish/data/fractions/numbers_read_as_ordinals.tsv +28 -0
- wetextprocessing-1.3.0/itn/spanish/data/fractions/ordinal_exceptions.tsv +2 -0
- wetextprocessing-1.3.0/itn/spanish/data/measures/math_symbols.tsv +4 -0
- wetextprocessing-1.3.0/itn/spanish/data/measures/measurements_plural.tsv +34 -0
- wetextprocessing-1.3.0/itn/spanish/data/measures/measurements_singular.tsv +35 -0
- wetextprocessing-1.3.0/itn/spanish/data/money/currency_major_plural.tsv +43 -0
- wetextprocessing-1.3.0/itn/spanish/data/money/currency_major_plural_capitalized.tsv +75 -0
- wetextprocessing-1.3.0/itn/spanish/data/money/currency_major_plural_ext.tsv +196 -0
- wetextprocessing-1.3.0/itn/spanish/data/money/currency_major_singular.tsv +43 -0
- wetextprocessing-1.3.0/itn/spanish/data/money/currency_major_singular_capitalized.tsv +76 -0
- wetextprocessing-1.3.0/itn/spanish/data/money/currency_major_singular_ext.tsv +196 -0
- wetextprocessing-1.3.0/itn/spanish/data/money/currency_minor_plural.tsv +5 -0
- wetextprocessing-1.3.0/itn/spanish/data/money/currency_minor_plural_ext.tsv +62 -0
- wetextprocessing-1.3.0/itn/spanish/data/money/currency_minor_singular.tsv +5 -0
- wetextprocessing-1.3.0/itn/spanish/data/money/currency_minor_singular_ext.tsv +64 -0
- wetextprocessing-1.3.0/itn/spanish/data/months.tsv +12 -0
- wetextprocessing-1.3.0/itn/spanish/data/numbers/digit.tsv +11 -0
- wetextprocessing-1.3.0/itn/spanish/data/numbers/hundreds.tsv +18 -0
- wetextprocessing-1.3.0/itn/spanish/data/numbers/teen.tsv +10 -0
- wetextprocessing-1.3.0/itn/spanish/data/numbers/ties.tsv +8 -0
- wetextprocessing-1.3.0/itn/spanish/data/numbers/twenties.tsv +11 -0
- wetextprocessing-1.3.0/itn/spanish/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/itn/spanish/data/ordinals/digit.tsv +22 -0
- wetextprocessing-1.3.0/itn/spanish/data/ordinals/digit_capitalized.tsv +22 -0
- wetextprocessing-1.3.0/itn/spanish/data/ordinals/hundreds.tsv +18 -0
- wetextprocessing-1.3.0/itn/spanish/data/ordinals/hundreds_capitalized.tsv +18 -0
- wetextprocessing-1.3.0/itn/spanish/data/ordinals/teen.tsv +60 -0
- wetextprocessing-1.3.0/itn/spanish/data/ordinals/teen_capitalized.tsv +60 -0
- wetextprocessing-1.3.0/itn/spanish/data/ordinals/ties.tsv +15 -0
- wetextprocessing-1.3.0/itn/spanish/data/ordinals/ties_capitalized.tsv +15 -0
- wetextprocessing-1.3.0/itn/spanish/data/ordinals/twenties.tsv +50 -0
- wetextprocessing-1.3.0/itn/spanish/data/ordinals/twenties_capitalized.tsv +50 -0
- wetextprocessing-1.3.0/itn/spanish/data/roman/digit.tsv +9 -0
- wetextprocessing-1.3.0/itn/spanish/data/roman/hundreds.tsv +9 -0
- wetextprocessing-1.3.0/itn/spanish/data/roman/thousands.tsv +3 -0
- wetextprocessing-1.3.0/itn/spanish/data/roman/ties.tsv +9 -0
- wetextprocessing-1.3.0/itn/spanish/data/time/minutes_to.tsv +6 -0
- wetextprocessing-1.3.0/itn/spanish/data/time/time_suffix.tsv +15 -0
- wetextprocessing-1.3.0/itn/spanish/data/time/time_suffix_cased.tsv +15 -0
- wetextprocessing-1.3.0/itn/spanish/data/time/time_to.tsv +25 -0
- wetextprocessing-1.3.0/itn/spanish/data/time/time_zone.tsv +42 -0
- wetextprocessing-1.3.0/itn/spanish/data/time/time_zone_cased.tsv +42 -0
- wetextprocessing-1.3.0/itn/spanish/data/time/time_zone_ext.tsv +105 -0
- wetextprocessing-1.3.0/itn/spanish/data/whitelist.tsv +47 -0
- wetextprocessing-1.3.0/itn/spanish/inverse_normalizer.py +77 -0
- wetextprocessing-1.3.0/itn/spanish/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/spanish/rules/cardinal.py +98 -0
- wetextprocessing-1.3.0/itn/spanish/rules/date.py +83 -0
- wetextprocessing-1.3.0/itn/spanish/rules/decimal.py +64 -0
- wetextprocessing-1.3.0/itn/spanish/rules/electronic.py +94 -0
- wetextprocessing-1.3.0/itn/spanish/rules/fraction.py +77 -0
- wetextprocessing-1.3.0/itn/spanish/rules/measure.py +84 -0
- wetextprocessing-1.3.0/itn/spanish/rules/money.py +95 -0
- wetextprocessing-1.3.0/itn/spanish/rules/ordinal.py +53 -0
- wetextprocessing-1.3.0/itn/spanish/rules/punctuation.py +30 -0
- wetextprocessing-1.3.0/itn/spanish/rules/telephone.py +70 -0
- wetextprocessing-1.3.0/itn/spanish/rules/time.py +112 -0
- wetextprocessing-1.3.0/itn/spanish/rules/utils.py +102 -0
- wetextprocessing-1.3.0/itn/spanish/rules/whitelist.py +47 -0
- wetextprocessing-1.3.0/itn/spanish/rules/word.py +32 -0
- wetextprocessing-1.3.0/itn/spanish/test/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_cardinal.txt +51 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_cardinal_cased.txt +30 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_date.txt +8 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_date_cased.txt +8 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_decimal.txt +29 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_decimal_cased.txt +6 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_electronic.txt +16 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_electronic_cased.txt +5 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_fraction.txt +12 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_measure.txt +20 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_measure_cased.txt +11 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_money.txt +24 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_money_cased.txt +6 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_ordinal.txt +30 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_ordinal_cased.txt +11 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_telephone.txt +9 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_telephone_cased.txt +6 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_time.txt +25 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_time_cased.txt +9 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_whitelist.txt +5 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_word.txt +49 -0
- wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_word_cased.txt +11 -0
- wetextprocessing-1.3.0/itn/spanish/test/normalizer_test.py +143 -0
- wetextprocessing-1.3.0/itn/spanish_english/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/spanish_english/data/en_whitelist.tsv +416 -0
- wetextprocessing-1.3.0/itn/spanish_english/data/es_whitelist.tsv +16 -0
- wetextprocessing-1.3.0/itn/spanish_english/data/measurements.tsv +3 -0
- wetextprocessing-1.3.0/itn/spanish_english/inverse_normalizer.py +99 -0
- wetextprocessing-1.3.0/itn/spanish_english/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/spanish_english/rules/english.py +230 -0
- wetextprocessing-1.3.0/itn/spanish_english/rules/prefixed.py +34 -0
- wetextprocessing-1.3.0/itn/spanish_english/rules/time.py +39 -0
- wetextprocessing-1.3.0/itn/spanish_english/rules/whitelist.py +38 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_cardinal.txt +80 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_cardinal_cased.txt +60 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_date.txt +44 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_date_cased.txt +20 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_decimal.txt +92 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_decimal_cased.txt +19 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_electronic.txt +41 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_electronic_cased.txt +14 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_fraction.txt +12 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_measure.txt +132 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_measure_cased.txt +22 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_money.txt +76 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_money_cased.txt +16 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_ordinal.txt +64 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_ordinal_cased.txt +21 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_telephone.txt +32 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_telephone_cased.txt +19 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_time.txt +54 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_time_cased.txt +19 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_whitelist.txt +18 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_word.txt +104 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_word_cased.txt +17 -0
- wetextprocessing-1.3.0/itn/spanish_english/test/normalizer_test.py +207 -0
- wetextprocessing-1.3.0/itn/swedish/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/swedish/data/dates/era_words.tsv +4 -0
- wetextprocessing-1.3.0/itn/swedish/data/dates/month_abbr.tsv +17 -0
- wetextprocessing-1.3.0/itn/swedish/data/dates/months.tsv +12 -0
- wetextprocessing-1.3.0/itn/swedish/data/electronic/domain.tsv +10 -0
- wetextprocessing-1.3.0/itn/swedish/data/electronic/server_name.tsv +29 -0
- wetextprocessing-1.3.0/itn/swedish/data/electronic/symbols.tsv +20 -0
- wetextprocessing-1.3.0/itn/swedish/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/itn/swedish/data/numbers/millions.tsv +88 -0
- wetextprocessing-1.3.0/itn/swedish/data/numbers/teen.tsv +10 -0
- wetextprocessing-1.3.0/itn/swedish/data/numbers/ties.tsv +8 -0
- wetextprocessing-1.3.0/itn/swedish/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/itn/swedish/data/ordinals/digit.tsv +9 -0
- wetextprocessing-1.3.0/itn/swedish/data/ordinals/teen.tsv +10 -0
- wetextprocessing-1.3.0/itn/swedish/data/ordinals/ties.tsv +8 -0
- wetextprocessing-1.3.0/itn/swedish/data/time/suffix.tsv +2 -0
- wetextprocessing-1.3.0/itn/swedish/data/time/time_zone.tsv +7 -0
- wetextprocessing-1.3.0/itn/swedish/data/whitelist.tsv +4 -0
- wetextprocessing-1.3.0/itn/swedish/inverse_normalizer.py +67 -0
- wetextprocessing-1.3.0/itn/swedish/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/swedish/rules/cardinal.py +88 -0
- wetextprocessing-1.3.0/itn/swedish/rules/date.py +91 -0
- wetextprocessing-1.3.0/itn/swedish/rules/decimal.py +68 -0
- wetextprocessing-1.3.0/itn/swedish/rules/electronic.py +69 -0
- wetextprocessing-1.3.0/itn/swedish/rules/fraction.py +90 -0
- wetextprocessing-1.3.0/itn/swedish/rules/ordinal.py +85 -0
- wetextprocessing-1.3.0/itn/swedish/rules/punctuation.py +29 -0
- wetextprocessing-1.3.0/itn/swedish/rules/telephone.py +59 -0
- wetextprocessing-1.3.0/itn/swedish/rules/time.py +99 -0
- wetextprocessing-1.3.0/itn/swedish/rules/utils.py +55 -0
- wetextprocessing-1.3.0/itn/swedish/rules/whitelist.py +29 -0
- wetextprocessing-1.3.0/itn/swedish/rules/word.py +29 -0
- wetextprocessing-1.3.0/itn/swedish/test/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_cardinal.txt +24 -0
- wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_date.txt +9 -0
- wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_decimal.txt +15 -0
- wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_electronic.txt +12 -0
- wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_fraction.txt +3 -0
- wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_ordinal.txt +7 -0
- wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_telephone.txt +2 -0
- wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_time.txt +14 -0
- wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_whitelist.txt +1 -0
- wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_word.txt +48 -0
- wetextprocessing-1.3.0/itn/swedish/test/normalizer_test.py +122 -0
- wetextprocessing-1.3.0/itn/test/utils_test.py +17 -0
- wetextprocessing-1.3.0/itn/utils.py +40 -0
- wetextprocessing-1.3.0/itn/vietnamese/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/currency.tsv +11 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/electronic/domain.tsv +11 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/electronic/server_name.tsv +16 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/electronic/symbols.tsv +21 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/electronic/url_symbols.tsv +8 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/magnitudes.tsv +5 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/math/symbols.tsv +10 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/measurements.tsv +185 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/months.tsv +14 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/numbers/digit.tsv +10 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/numbers/digit_special.tsv +3 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/numbers/hundred.tsv +1 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/numbers/teen.tsv +11 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/numbers/thousands.tsv +6 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/numbers/ties.tsv +9 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/ordinals/digit.tsv +12 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/time/hours.tsv +29 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/time/hours_to.tsv +25 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/time/hours_to_night.tsv +12 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/time/minutes.tsv +118 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/time/minutes_to.tsv +59 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/time/time_suffix.tsv +8 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/time/time_zone.tsv +7 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/time/zero_prefix.tsv +3 -0
- wetextprocessing-1.3.0/itn/vietnamese/data/whitelist.tsv +0 -0
- wetextprocessing-1.3.0/itn/vietnamese/inverse_normalizer.py +73 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/cardinal.py +99 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/date.py +76 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/decimal.py +72 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/electronic.py +60 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/fraction.py +41 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/measure.py +60 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/money.py +54 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/ordinal.py +32 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/punctuation.py +29 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/telephone.py +57 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/time.py +83 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/whitelist.py +30 -0
- wetextprocessing-1.3.0/itn/vietnamese/rules/word.py +28 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/__init__.py +0 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_cardinal.txt +142 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_date.txt +69 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_decimal.txt +74 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_electronic.txt +18 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_fraction.txt +14 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_measure.txt +35 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_money.txt +15 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_ordinal.txt +12 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_telephone.txt +25 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_time.txt +31 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_whitelist.txt +1 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_word.txt +17 -0
- wetextprocessing-1.3.0/itn/vietnamese/test/normalizer_test.py +144 -0
- wetextprocessing-1.3.0/pyproject.toml +125 -0
- wetextprocessing-1.3.0/runtime/CMakeLists.txt +51 -0
- wetextprocessing-1.3.0/runtime/CMakePresets.json +69 -0
- wetextprocessing-1.3.0/runtime/android/.gitignore +15 -0
- wetextprocessing-1.3.0/runtime/android/app/build.gradle.kts +75 -0
- wetextprocessing-1.3.0/runtime/android/app/src/main/AndroidManifest.xml +21 -0
- wetextprocessing-1.3.0/runtime/android/app/src/main/assets/README.md +29 -0
- wetextprocessing-1.3.0/runtime/android/app/src/main/java/com/wenet/WeTextProcessing/MainActivity.kt +140 -0
- wetextprocessing-1.3.0/runtime/android/app/src/main/java/com/wenet/WeTextProcessing/Theme.kt +46 -0
- wetextprocessing-1.3.0/runtime/android/app/src/main/java/com/wenet/WeTextProcessing/WeTextProcessing.kt +17 -0
- wetextprocessing-1.3.0/runtime/android/app/src/main/jniLibs/.gitignore +2 -0
- wetextprocessing-1.3.0/runtime/android/app/src/main/res/drawable/ic_launcher_background.xml +21 -0
- wetextprocessing-1.3.0/runtime/android/app/src/main/res/drawable/ic_launcher_foreground.xml +30 -0
- wetextprocessing-1.3.0/runtime/android/app/src/main/res/values/themes.xml +5 -0
- wetextprocessing-1.3.0/runtime/android/app/src/main/res/values-night/themes.xml +3 -0
- wetextprocessing-1.3.0/runtime/android/build.gradle.kts +4 -0
- wetextprocessing-1.3.0/runtime/android/gradle/gradle-daemon-jvm.properties +12 -0
- wetextprocessing-1.3.0/runtime/android/gradle/libs.versions.toml +21 -0
- wetextprocessing-1.3.0/runtime/android/gradle/wrapper/gradle-wrapper.properties +5 -0
- wetextprocessing-1.3.0/runtime/android/gradle.properties +17 -0
- wetextprocessing-1.3.0/runtime/android/settings.gradle.kts +26 -0
- wetextprocessing-1.3.0/runtime/bin/CMakeLists.txt +3 -0
- wetextprocessing-1.3.0/runtime/bindings/android/CMakeLists.txt +17 -0
- wetextprocessing-1.3.0/runtime/bindings/android/wetextprocessing.cc +80 -0
- wetextprocessing-1.3.0/runtime/cmake/glog.cmake +15 -0
- wetextprocessing-1.3.0/runtime/cmake/openfst.cmake +27 -0
- wetextprocessing-1.3.0/runtime/processor/CMakeLists.txt +15 -0
- wetextprocessing-1.3.0/runtime/processor/wetext_processor.cc +87 -0
- wetextprocessing-1.3.0/runtime/processor/wetext_token_parser.cc +201 -0
- wetextprocessing-1.3.0/runtime/processor/wetext_token_parser.h +98 -0
- wetextprocessing-1.3.0/runtime/test/CMakeLists.txt +22 -0
- wetextprocessing-1.3.0/runtime/test/processor_test.cc +78 -0
- wetextprocessing-1.3.0/runtime/test/token_parser_test.cc +122 -0
- wetextprocessing-1.3.0/runtime/utils/wetext_flags.h +20 -0
- wetextprocessing-1.3.0/runtime/utils/wetext_log.h +20 -0
- wetextprocessing-1.3.0/tn/README.md +81 -0
- wetextprocessing-1.3.0/tn/__init__.py +3 -0
- wetextprocessing-1.3.0/tn/_version.py +24 -0
- wetextprocessing-1.3.0/tn/alignment.py +254 -0
- wetextprocessing-1.3.0/tn/arabic/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/arabic/data/measure/measurements.tsv +14 -0
- wetextprocessing-1.3.0/tn/arabic/data/money/currency_major.tsv +24 -0
- wetextprocessing-1.3.0/tn/arabic/data/money/currency_minor_plural.tsv +4 -0
- wetextprocessing-1.3.0/tn/arabic/data/money/currency_minor_singular.tsv +3 -0
- wetextprocessing-1.3.0/tn/arabic/data/money/local_currency.tsv +20 -0
- wetextprocessing-1.3.0/tn/arabic/data/number/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/arabic/data/number/digit_100.tsv +7 -0
- wetextprocessing-1.3.0/tn/arabic/data/number/digit_1000.tsv +7 -0
- wetextprocessing-1.3.0/tn/arabic/data/number/flops.tsv +72 -0
- wetextprocessing-1.3.0/tn/arabic/data/number/fraction_dual.tsv +9 -0
- wetextprocessing-1.3.0/tn/arabic/data/number/fraction_plural.tsv +8 -0
- wetextprocessing-1.3.0/tn/arabic/data/number/fraction_singular.tsv +9 -0
- wetextprocessing-1.3.0/tn/arabic/data/number/quantities.tsv +6 -0
- wetextprocessing-1.3.0/tn/arabic/data/number/teens.tsv +9 -0
- wetextprocessing-1.3.0/tn/arabic/data/number/tens.tsv +10 -0
- wetextprocessing-1.3.0/tn/arabic/normalizer.py +61 -0
- wetextprocessing-1.3.0/tn/arabic/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/arabic/rules/cardinal.py +86 -0
- wetextprocessing-1.3.0/tn/arabic/rules/decimal.py +69 -0
- wetextprocessing-1.3.0/tn/arabic/rules/fraction.py +79 -0
- wetextprocessing-1.3.0/tn/arabic/rules/measure.py +80 -0
- wetextprocessing-1.3.0/tn/arabic/rules/money.py +131 -0
- wetextprocessing-1.3.0/tn/arabic/rules/punctuation.py +44 -0
- wetextprocessing-1.3.0/tn/arabic/rules/word.py +32 -0
- wetextprocessing-1.3.0/tn/arabic/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/arabic/test/data/test_cases_cardinal.txt +27 -0
- wetextprocessing-1.3.0/tn/arabic/test/data/test_cases_decimal.txt +30 -0
- wetextprocessing-1.3.0/tn/arabic/test/data/test_cases_fraction.txt +20 -0
- wetextprocessing-1.3.0/tn/arabic/test/data/test_cases_measure.txt +14 -0
- wetextprocessing-1.3.0/tn/arabic/test/data/test_cases_money.txt +27 -0
- wetextprocessing-1.3.0/tn/arabic/test/data/test_cases_whitelist.txt +6 -0
- wetextprocessing-1.3.0/tn/arabic/test/normalizer_test.py +162 -0
- wetextprocessing-1.3.0/tn/armenian/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/armenian/data/currency.tsv +52 -0
- wetextprocessing-1.3.0/tn/armenian/data/measurement_dates.tsv +8 -0
- wetextprocessing-1.3.0/tn/armenian/data/measurements.tsv +44 -0
- wetextprocessing-1.3.0/tn/armenian/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/armenian/data/numbers/quantities.tsv +3 -0
- wetextprocessing-1.3.0/tn/armenian/data/numbers/ties.tsv +8 -0
- wetextprocessing-1.3.0/tn/armenian/data/time/hours.tsv +26 -0
- wetextprocessing-1.3.0/tn/armenian/data/time/minutes.tsv +61 -0
- wetextprocessing-1.3.0/tn/armenian/data/whitelist.tsv +14 -0
- wetextprocessing-1.3.0/tn/armenian/normalizer.py +65 -0
- wetextprocessing-1.3.0/tn/armenian/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/armenian/rules/cardinal.py +75 -0
- wetextprocessing-1.3.0/tn/armenian/rules/decimal.py +63 -0
- wetextprocessing-1.3.0/tn/armenian/rules/fraction.py +36 -0
- wetextprocessing-1.3.0/tn/armenian/rules/measure.py +49 -0
- wetextprocessing-1.3.0/tn/armenian/rules/money.py +38 -0
- wetextprocessing-1.3.0/tn/armenian/rules/ordinal.py +59 -0
- wetextprocessing-1.3.0/tn/armenian/rules/punctuation.py +31 -0
- wetextprocessing-1.3.0/tn/armenian/rules/time.py +34 -0
- wetextprocessing-1.3.0/tn/armenian/rules/utils.py +60 -0
- wetextprocessing-1.3.0/tn/armenian/rules/whitelist.py +31 -0
- wetextprocessing-1.3.0/tn/armenian/rules/word.py +28 -0
- wetextprocessing-1.3.0/tn/armenian/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_cardinal.txt +12 -0
- wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_decimal.txt +9 -0
- wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_fraction.txt +10 -0
- wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_measure.txt +9 -0
- wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_money.txt +10 -0
- wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_ordinal.txt +11 -0
- wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_time.txt +9 -0
- wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_whitelist.txt +14 -0
- wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_word.txt +50 -0
- wetextprocessing-1.3.0/tn/armenian/test/normalizer_test.py +156 -0
- wetextprocessing-1.3.0/tn/cache.py +822 -0
- wetextprocessing-1.3.0/tn/chinese/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/chinese/normalizer.py +112 -0
- wetextprocessing-1.3.0/tn/chinese/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/chinese/rules/cardinal.py +94 -0
- wetextprocessing-1.3.0/tn/chinese/rules/date.py +72 -0
- wetextprocessing-1.3.0/tn/chinese/rules/fraction.py +41 -0
- wetextprocessing-1.3.0/tn/chinese/rules/math.py +44 -0
- wetextprocessing-1.3.0/tn/chinese/rules/measure.py +77 -0
- wetextprocessing-1.3.0/tn/chinese/rules/money.py +45 -0
- wetextprocessing-1.3.0/tn/chinese/rules/postprocessor.py +63 -0
- wetextprocessing-1.3.0/tn/chinese/rules/range.py +29 -0
- wetextprocessing-1.3.0/tn/chinese/rules/sport.py +49 -0
- wetextprocessing-1.3.0/tn/chinese/rules/time.py +62 -0
- wetextprocessing-1.3.0/tn/chinese/rules/whitelist.py +46 -0
- wetextprocessing-1.3.0/tn/chinese/test/alignment_test.py +154 -0
- wetextprocessing-1.3.0/tn/chinese/test/cache_safety_test.py +505 -0
- wetextprocessing-1.3.0/tn/chinese/test/cache_test.py +192 -0
- wetextprocessing-1.3.0/tn/chinese/test/data/normalizer.txt +46 -0
- wetextprocessing-1.3.0/tn/chinese/test/data/normalizer_tag_oov.txt +2 -0
- wetextprocessing-1.3.0/tn/chinese/test/normalizer_test.py +94 -0
- wetextprocessing-1.3.0/tn/chinese/test/postprocessor_test.py +45 -0
- wetextprocessing-1.3.0/tn/chinese/test/processor_test.py +496 -0
- wetextprocessing-1.3.0/tn/chinese/test/token_parser_test.py +110 -0
- wetextprocessing-1.3.0/tn/cli.py +278 -0
- wetextprocessing-1.3.0/tn/english/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/english/data/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/english/data/date/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/english/data/electronic/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/english/data/measure/unit_alternatives.tsv +42 -0
- wetextprocessing-1.3.0/tn/english/data/money/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/english/data/number/fraction.tsv +18 -0
- wetextprocessing-1.3.0/tn/english/data/ordinal/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/english/data/roman/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/english/data/telephone/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/english/data/whitelist/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/english/data/whitelist/symbol.tsv +23 -0
- wetextprocessing-1.3.0/tn/english/normalizer.py +98 -0
- wetextprocessing-1.3.0/tn/english/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/english/rules/cardinal.py +149 -0
- wetextprocessing-1.3.0/tn/english/rules/date.py +270 -0
- wetextprocessing-1.3.0/tn/english/rules/decimal.py +145 -0
- wetextprocessing-1.3.0/tn/english/rules/electronic.py +189 -0
- wetextprocessing-1.3.0/tn/english/rules/fraction.py +132 -0
- wetextprocessing-1.3.0/tn/english/rules/measure.py +313 -0
- wetextprocessing-1.3.0/tn/english/rules/money.py +118 -0
- wetextprocessing-1.3.0/tn/english/rules/ordinal.py +75 -0
- wetextprocessing-1.3.0/tn/english/rules/punctuation.py +75 -0
- wetextprocessing-1.3.0/tn/english/rules/range.py +117 -0
- wetextprocessing-1.3.0/tn/english/rules/roman.py +141 -0
- wetextprocessing-1.3.0/tn/english/rules/serial.py +88 -0
- wetextprocessing-1.3.0/tn/english/rules/telephone.py +129 -0
- wetextprocessing-1.3.0/tn/english/rules/time.py +140 -0
- wetextprocessing-1.3.0/tn/english/rules/whitelist.py +130 -0
- wetextprocessing-1.3.0/tn/english/rules/word.py +61 -0
- wetextprocessing-1.3.0/tn/english/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/english/test/data/date.txt +14 -0
- wetextprocessing-1.3.0/tn/english/test/data/normalizer.txt +11 -0
- wetextprocessing-1.3.0/tn/english/test/electronic_test.py +55 -0
- wetextprocessing-1.3.0/tn/english/test/nbest_test.py +87 -0
- wetextprocessing-1.3.0/tn/english/test/normalizer_test.py +206 -0
- wetextprocessing-1.3.0/tn/french/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/french/data/dates/eras.tsv +8 -0
- wetextprocessing-1.3.0/tn/french/data/dates/months.tsv +12 -0
- wetextprocessing-1.3.0/tn/french/data/fractions/irregular_denominators.tsv +7 -0
- wetextprocessing-1.3.0/tn/french/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/french/data/numbers/eighty.tsv +1 -0
- wetextprocessing-1.3.0/tn/french/data/numbers/ninety.tsv +1 -0
- wetextprocessing-1.3.0/tn/french/data/numbers/quantities.tsv +10 -0
- wetextprocessing-1.3.0/tn/french/data/numbers/seventy.tsv +1 -0
- wetextprocessing-1.3.0/tn/french/data/numbers/teens.tsv +9 -0
- wetextprocessing-1.3.0/tn/french/data/numbers/ten.tsv +1 -0
- wetextprocessing-1.3.0/tn/french/data/numbers/tens_simple.tsv +5 -0
- wetextprocessing-1.3.0/tn/french/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/tn/french/data/ordinals/irregular_numbers.tsv +30 -0
- wetextprocessing-1.3.0/tn/french/data/ordinals/suffixes.tsv +6 -0
- wetextprocessing-1.3.0/tn/french/data/punctuation/symbol.tsv +23 -0
- wetextprocessing-1.3.0/tn/french/data/whitelist.tsv +13 -0
- wetextprocessing-1.3.0/tn/french/normalizer.py +66 -0
- wetextprocessing-1.3.0/tn/french/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/french/rules/cardinal.py +110 -0
- wetextprocessing-1.3.0/tn/french/rules/date.py +68 -0
- wetextprocessing-1.3.0/tn/french/rules/decimal.py +62 -0
- wetextprocessing-1.3.0/tn/french/rules/fraction.py +60 -0
- wetextprocessing-1.3.0/tn/french/rules/ordinal.py +56 -0
- wetextprocessing-1.3.0/tn/french/rules/punctuation.py +43 -0
- wetextprocessing-1.3.0/tn/french/rules/whitelist.py +29 -0
- wetextprocessing-1.3.0/tn/french/rules/word.py +29 -0
- wetextprocessing-1.3.0/tn/french/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/french/test/data/test_cases_cardinal.txt +14 -0
- wetextprocessing-1.3.0/tn/french/test/data/test_cases_date.txt +13 -0
- wetextprocessing-1.3.0/tn/french/test/data/test_cases_decimal.txt +7 -0
- wetextprocessing-1.3.0/tn/french/test/data/test_cases_fraction.txt +12 -0
- wetextprocessing-1.3.0/tn/french/test/data/test_cases_ordinal.txt +15 -0
- wetextprocessing-1.3.0/tn/french/test/data/test_cases_whitelist.txt +7 -0
- wetextprocessing-1.3.0/tn/french/test/data/test_cases_word.txt +49 -0
- wetextprocessing-1.3.0/tn/french/test/normalizer_test.py +157 -0
- wetextprocessing-1.3.0/tn/german/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/german/data/__init__.py +13 -0
- wetextprocessing-1.3.0/tn/german/data/electronic/__init__.py +13 -0
- wetextprocessing-1.3.0/tn/german/data/electronic/domain.tsv +844 -0
- wetextprocessing-1.3.0/tn/german/data/electronic/server_name.tsv +17 -0
- wetextprocessing-1.3.0/tn/german/data/electronic/symbols.tsv +20 -0
- wetextprocessing-1.3.0/tn/german/data/fractions.tsv +30 -0
- wetextprocessing-1.3.0/tn/german/data/measure/__init__.py +13 -0
- wetextprocessing-1.3.0/tn/german/data/measure/measurements.tsv +82 -0
- wetextprocessing-1.3.0/tn/german/data/measure/suppletive.tsv +10 -0
- wetextprocessing-1.3.0/tn/german/data/money/__init__.py +13 -0
- wetextprocessing-1.3.0/tn/german/data/money/currency.tsv +25 -0
- wetextprocessing-1.3.0/tn/german/data/money/currency_minor_plural.tsv +3 -0
- wetextprocessing-1.3.0/tn/german/data/money/currency_minor_singular.tsv +3 -0
- wetextprocessing-1.3.0/tn/german/data/months/__init__.py +13 -0
- wetextprocessing-1.3.0/tn/german/data/months/abbr_to_name.tsv +13 -0
- wetextprocessing-1.3.0/tn/german/data/months/numbers.tsv +24 -0
- wetextprocessing-1.3.0/tn/german/data/numbers/__init__.py +13 -0
- wetextprocessing-1.3.0/tn/german/data/numbers/digit.tsv +8 -0
- wetextprocessing-1.3.0/tn/german/data/numbers/ones.tsv +3 -0
- wetextprocessing-1.3.0/tn/german/data/numbers/quantities.tsv +12 -0
- wetextprocessing-1.3.0/tn/german/data/numbers/teen.tsv +10 -0
- wetextprocessing-1.3.0/tn/german/data/numbers/ties.tsv +8 -0
- wetextprocessing-1.3.0/tn/german/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/tn/german/data/ordinals/__init__.py +13 -0
- wetextprocessing-1.3.0/tn/german/data/ordinals/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/german/data/ordinals/thousands.tsv +4 -0
- wetextprocessing-1.3.0/tn/german/data/ordinals/ties.tsv +8 -0
- wetextprocessing-1.3.0/tn/german/data/time/__init__.py +13 -0
- wetextprocessing-1.3.0/tn/german/data/time/hour_to.tsv +12 -0
- wetextprocessing-1.3.0/tn/german/data/time/hour_to_night.tsv +13 -0
- wetextprocessing-1.3.0/tn/german/data/time/minute_to.tsv +59 -0
- wetextprocessing-1.3.0/tn/german/data/time/time_zone.tsv +7 -0
- wetextprocessing-1.3.0/tn/german/data/whitelist.tsv +21 -0
- wetextprocessing-1.3.0/tn/german/normalizer.py +77 -0
- wetextprocessing-1.3.0/tn/german/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/german/rules/cardinal.py +120 -0
- wetextprocessing-1.3.0/tn/german/rules/date.py +98 -0
- wetextprocessing-1.3.0/tn/german/rules/decimal.py +71 -0
- wetextprocessing-1.3.0/tn/german/rules/electronic.py +67 -0
- wetextprocessing-1.3.0/tn/german/rules/fraction.py +65 -0
- wetextprocessing-1.3.0/tn/german/rules/measure.py +117 -0
- wetextprocessing-1.3.0/tn/german/rules/money.py +144 -0
- wetextprocessing-1.3.0/tn/german/rules/ordinal.py +46 -0
- wetextprocessing-1.3.0/tn/german/rules/punctuation.py +39 -0
- wetextprocessing-1.3.0/tn/german/rules/telephone.py +48 -0
- wetextprocessing-1.3.0/tn/german/rules/time.py +123 -0
- wetextprocessing-1.3.0/tn/german/rules/whitelist.py +34 -0
- wetextprocessing-1.3.0/tn/german/rules/word.py +34 -0
- wetextprocessing-1.3.0/tn/german/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/german/test/calendar_test.py +168 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_cardinal.txt +50 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_date.txt +15 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_decimal.txt +10 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_electronic.txt +19 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_fraction.txt +34 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_measure.txt +29 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_money.txt +21 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_normalize_with_audio.txt +131 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_ordinal.txt +19 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_telephone.txt +4 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_time.txt +28 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_whitelist.txt +6 -0
- wetextprocessing-1.3.0/tn/german/test/data/test_cases_word.txt +49 -0
- wetextprocessing-1.3.0/tn/german/test/financial_test.py +167 -0
- wetextprocessing-1.3.0/tn/german/test/normalizer_test.py +136 -0
- wetextprocessing-1.3.0/tn/german/test/numeric_test.py +185 -0
- wetextprocessing-1.3.0/tn/german/test/text_test.py +235 -0
- wetextprocessing-1.3.0/tn/hebrew/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/hebrew/data/measurements.tsv +29 -0
- wetextprocessing-1.3.0/tn/hebrew/data/prefix.tsv +17 -0
- wetextprocessing-1.3.0/tn/hebrew/data/spaced_measurements.tsv +16 -0
- wetextprocessing-1.3.0/tn/hebrew/data/whitelist.tsv +9 -0
- wetextprocessing-1.3.0/tn/hebrew/normalizer.py +58 -0
- wetextprocessing-1.3.0/tn/hebrew/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/hebrew/rules/cardinal.py +43 -0
- wetextprocessing-1.3.0/tn/hebrew/rules/date.py +84 -0
- wetextprocessing-1.3.0/tn/hebrew/rules/decimal.py +71 -0
- wetextprocessing-1.3.0/tn/hebrew/rules/measure.py +56 -0
- wetextprocessing-1.3.0/tn/hebrew/rules/numbers.py +135 -0
- wetextprocessing-1.3.0/tn/hebrew/rules/punctuation.py +29 -0
- wetextprocessing-1.3.0/tn/hebrew/rules/time.py +94 -0
- wetextprocessing-1.3.0/tn/hebrew/rules/utils.py +59 -0
- wetextprocessing-1.3.0/tn/hebrew/rules/whitelist.py +26 -0
- wetextprocessing-1.3.0/tn/hebrew/rules/word.py +27 -0
- wetextprocessing-1.3.0/tn/hebrew/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/hebrew/test/normalizer_test.py +245 -0
- wetextprocessing-1.3.0/tn/hindi/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/hindi/data/address/cities.tsv +36 -0
- wetextprocessing-1.3.0/tn/hindi/data/address/context.tsv +47 -0
- wetextprocessing-1.3.0/tn/hindi/data/address/letters.tsv +26 -0
- wetextprocessing-1.3.0/tn/hindi/data/address/special_characters.tsv +2 -0
- wetextprocessing-1.3.0/tn/hindi/data/address/states.tsv +36 -0
- wetextprocessing-1.3.0/tn/hindi/data/date/days.tsv +62 -0
- wetextprocessing-1.3.0/tn/hindi/data/date/months.tsv +24 -0
- wetextprocessing-1.3.0/tn/hindi/data/date/prefixes.tsv +4 -0
- wetextprocessing-1.3.0/tn/hindi/data/date/suffixes.tsv +10 -0
- wetextprocessing-1.3.0/tn/hindi/data/date/year_suffix.tsv +2 -0
- wetextprocessing-1.3.0/tn/hindi/data/electronic/domain.tsv +24 -0
- wetextprocessing-1.3.0/tn/hindi/data/electronic/elements.tsv +132 -0
- wetextprocessing-1.3.0/tn/hindi/data/electronic/file_extensions.tsv +34 -0
- wetextprocessing-1.3.0/tn/hindi/data/electronic/protocols.tsv +5 -0
- wetextprocessing-1.3.0/tn/hindi/data/electronic/subscript_digit.tsv +10 -0
- wetextprocessing-1.3.0/tn/hindi/data/electronic/symbol_classes.tsv +16 -0
- wetextprocessing-1.3.0/tn/hindi/data/electronic/symbols.tsv +36 -0
- wetextprocessing-1.3.0/tn/hindi/data/fraction/common_fractions.tsv +10 -0
- wetextprocessing-1.3.0/tn/hindi/data/measure/quarterly_units_list.tsv +5 -0
- wetextprocessing-1.3.0/tn/hindi/data/measure/quarterly_units_map.tsv +10 -0
- wetextprocessing-1.3.0/tn/hindi/data/measure/unit.tsv +146 -0
- wetextprocessing-1.3.0/tn/hindi/data/measure/unit_year_formal.tsv +1 -0
- wetextprocessing-1.3.0/tn/hindi/data/money/currency.tsv +9 -0
- wetextprocessing-1.3.0/tn/hindi/data/money/currency_singular.tsv +9 -0
- wetextprocessing-1.3.0/tn/hindi/data/money/major_minor_currencies.tsv +10 -0
- wetextprocessing-1.3.0/tn/hindi/data/numbers/digit.tsv +18 -0
- wetextprocessing-1.3.0/tn/hindi/data/numbers/hundred.tsv +1 -0
- wetextprocessing-1.3.0/tn/hindi/data/numbers/teens_and_ties.tsv +90 -0
- wetextprocessing-1.3.0/tn/hindi/data/numbers/teens_and_ties_en.tsv +90 -0
- wetextprocessing-1.3.0/tn/hindi/data/numbers/thousands.tsv +8 -0
- wetextprocessing-1.3.0/tn/hindi/data/numbers/zero.tsv +2 -0
- wetextprocessing-1.3.0/tn/hindi/data/ordinal/en_to_hi_digit.tsv +10 -0
- wetextprocessing-1.3.0/tn/hindi/data/ordinal/exceptions.tsv +50 -0
- wetextprocessing-1.3.0/tn/hindi/data/ordinal/suffixes.tsv +3 -0
- wetextprocessing-1.3.0/tn/hindi/data/ordinal/suffixes_map.tsv +1 -0
- wetextprocessing-1.3.0/tn/hindi/data/roman/roman_ordinal_exceptions.tsv +10 -0
- wetextprocessing-1.3.0/tn/hindi/data/roman/roman_to_spoken.tsv +100 -0
- wetextprocessing-1.3.0/tn/hindi/data/serial/chars.tsv +68 -0
- wetextprocessing-1.3.0/tn/hindi/data/serial/power_special.tsv +4 -0
- wetextprocessing-1.3.0/tn/hindi/data/serial/special_symbols.tsv +4 -0
- wetextprocessing-1.3.0/tn/hindi/data/telephone/credit_context.tsv +3 -0
- wetextprocessing-1.3.0/tn/hindi/data/telephone/landline_context.tsv +5 -0
- wetextprocessing-1.3.0/tn/hindi/data/telephone/mobile_context.tsv +4 -0
- wetextprocessing-1.3.0/tn/hindi/data/telephone/number.tsv +10 -0
- wetextprocessing-1.3.0/tn/hindi/data/telephone/pincode_context.tsv +4 -0
- wetextprocessing-1.3.0/tn/hindi/data/time/hours.tsv +50 -0
- wetextprocessing-1.3.0/tn/hindi/data/time/minutes.tsv +120 -0
- wetextprocessing-1.3.0/tn/hindi/data/time/seconds.tsv +120 -0
- wetextprocessing-1.3.0/tn/hindi/data/whitelist/abbreviations.tsv +7 -0
- wetextprocessing-1.3.0/tn/hindi/data/whitelist/paune_mappings.tsv +100 -0
- wetextprocessing-1.3.0/tn/hindi/normalizer.py +95 -0
- wetextprocessing-1.3.0/tn/hindi/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/hindi/rules/address.py +97 -0
- wetextprocessing-1.3.0/tn/hindi/rules/cardinal.py +117 -0
- wetextprocessing-1.3.0/tn/hindi/rules/date.py +135 -0
- wetextprocessing-1.3.0/tn/hindi/rules/decimal.py +42 -0
- wetextprocessing-1.3.0/tn/hindi/rules/electronic.py +145 -0
- wetextprocessing-1.3.0/tn/hindi/rules/fraction.py +62 -0
- wetextprocessing-1.3.0/tn/hindi/rules/measure.py +110 -0
- wetextprocessing-1.3.0/tn/hindi/rules/money.py +104 -0
- wetextprocessing-1.3.0/tn/hindi/rules/ordinal.py +37 -0
- wetextprocessing-1.3.0/tn/hindi/rules/punctuation.py +50 -0
- wetextprocessing-1.3.0/tn/hindi/rules/roman.py +70 -0
- wetextprocessing-1.3.0/tn/hindi/rules/serial.py +82 -0
- wetextprocessing-1.3.0/tn/hindi/rules/telephone.py +98 -0
- wetextprocessing-1.3.0/tn/hindi/rules/time.py +62 -0
- wetextprocessing-1.3.0/tn/hindi/rules/utils.py +54 -0
- wetextprocessing-1.3.0/tn/hindi/rules/whitelist.py +37 -0
- wetextprocessing-1.3.0/tn/hindi/rules/word.py +31 -0
- wetextprocessing-1.3.0/tn/hindi/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_address.txt +52 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_cardinal.txt +161 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_date.txt +39 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_decimal.txt +24 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_electronic.txt +65 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_fraction.txt +27 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_measure.txt +69 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_money.txt +144 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_ordinal.txt +72 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_roman.txt +23 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_serial.txt +31 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_telephone.txt +25 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_time.txt +18 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_whitelist.txt +14 -0
- wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_word.txt +20 -0
- wetextprocessing-1.3.0/tn/hindi/test/normalizer_test.py +137 -0
- wetextprocessing-1.3.0/tn/hungarian/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/hungarian/data/dates/days.tsv +31 -0
- wetextprocessing-1.3.0/tn/hungarian/data/dates/month_abbr.tsv +12 -0
- wetextprocessing-1.3.0/tn/hungarian/data/dates/months.tsv +12 -0
- wetextprocessing-1.3.0/tn/hungarian/data/dates/months_roman.tsv +12 -0
- wetextprocessing-1.3.0/tn/hungarian/data/electronic/domain.tsv +15 -0
- wetextprocessing-1.3.0/tn/hungarian/data/electronic/server_name.tsv +15 -0
- wetextprocessing-1.3.0/tn/hungarian/data/electronic/symbols.tsv +23 -0
- wetextprocessing-1.3.0/tn/hungarian/data/fractions/fraction_symbols.tsv +19 -0
- wetextprocessing-1.3.0/tn/hungarian/data/inflection/endings.tsv +51 -0
- wetextprocessing-1.3.0/tn/hungarian/data/inflection/plural_endings.tsv +39 -0
- wetextprocessing-1.3.0/tn/hungarian/data/inflection/word_endings.tsv +1 -0
- wetextprocessing-1.3.0/tn/hungarian/data/measures/measurements.tsv +52 -0
- wetextprocessing-1.3.0/tn/hungarian/data/money/alphabet.tsv +44 -0
- wetextprocessing-1.3.0/tn/hungarian/data/money/currency.tsv +11 -0
- wetextprocessing-1.3.0/tn/hungarian/data/money/currency_minor.tsv +11 -0
- wetextprocessing-1.3.0/tn/hungarian/data/number/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/hungarian/data/number/digit_inline.tsv +9 -0
- wetextprocessing-1.3.0/tn/hungarian/data/number/quantities.tsv +21 -0
- wetextprocessing-1.3.0/tn/hungarian/data/number/tens.tsv +9 -0
- wetextprocessing-1.3.0/tn/hungarian/data/number/tens_inline.tsv +9 -0
- wetextprocessing-1.3.0/tn/hungarian/data/number/zero.tsv +1 -0
- wetextprocessing-1.3.0/tn/hungarian/data/ordinals/endings.tsv +23 -0
- wetextprocessing-1.3.0/tn/hungarian/data/ordinals/exceptional.tsv +4 -0
- wetextprocessing-1.3.0/tn/hungarian/data/telephone/area_codes.tsv +71 -0
- wetextprocessing-1.3.0/tn/hungarian/data/telephone/country_codes.tsv +213 -0
- wetextprocessing-1.3.0/tn/hungarian/data/telephone/telephone_abbr.tsv +3 -0
- wetextprocessing-1.3.0/tn/hungarian/data/time/time_zone.tsv +9 -0
- wetextprocessing-1.3.0/tn/hungarian/data/whitelist.tsv +24 -0
- wetextprocessing-1.3.0/tn/hungarian/data/whitelist_inflect.tsv +39 -0
- wetextprocessing-1.3.0/tn/hungarian/data/whitelist_inflect_sg.tsv +14 -0
- wetextprocessing-1.3.0/tn/hungarian/normalizer.py +80 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/cardinal.py +98 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/date.py +104 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/decimal.py +85 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/electronic.py +69 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/fraction.py +55 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/measure.py +58 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/money.py +81 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/ordinal.py +41 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/punctuation.py +46 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/telephone.py +71 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/time.py +106 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/utils.py +156 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/whitelist.py +32 -0
- wetextprocessing-1.3.0/tn/hungarian/rules/word.py +38 -0
- wetextprocessing-1.3.0/tn/hungarian/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_cardinal.txt +69 -0
- wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_date.txt +17 -0
- wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_decimal.txt +6 -0
- wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_electronic.txt +8 -0
- wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_fraction.txt +4 -0
- wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_measure.txt +6 -0
- wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_money.txt +9 -0
- wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_ordinal.txt +5 -0
- wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_telephone.txt +9 -0
- wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_time.txt +10 -0
- wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_whitelist.txt +10 -0
- wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_word.txt +52 -0
- wetextprocessing-1.3.0/tn/hungarian/test/normalizer_test.py +210 -0
- wetextprocessing-1.3.0/tn/italian/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/italian/data/electronic/domain.tsv +14 -0
- wetextprocessing-1.3.0/tn/italian/data/electronic/server_name.tsv +11 -0
- wetextprocessing-1.3.0/tn/italian/data/electronic/symbols.tsv +22 -0
- wetextprocessing-1.3.0/tn/italian/data/measure/measurements.tsv +64 -0
- wetextprocessing-1.3.0/tn/italian/data/measure/suppletive.tsv +11 -0
- wetextprocessing-1.3.0/tn/italian/data/money/currency_major.tsv +11 -0
- wetextprocessing-1.3.0/tn/italian/data/money/currency_minor.tsv +11 -0
- wetextprocessing-1.3.0/tn/italian/data/money/currency_plural_fem.tsv +3 -0
- wetextprocessing-1.3.0/tn/italian/data/money/currency_plural_masc.tsv +11 -0
- wetextprocessing-1.3.0/tn/italian/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/italian/data/numbers/hundreds.tsv +8 -0
- wetextprocessing-1.3.0/tn/italian/data/numbers/quantities.tsv +2 -0
- wetextprocessing-1.3.0/tn/italian/data/numbers/teen.tsv +10 -0
- wetextprocessing-1.3.0/tn/italian/data/numbers/tens.tsv +8 -0
- wetextprocessing-1.3.0/tn/italian/data/numbers/tens_eight.tsv +8 -0
- wetextprocessing-1.3.0/tn/italian/data/numbers/tens_one.tsv +8 -0
- wetextprocessing-1.3.0/tn/italian/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/tn/italian/data/whitelist/whitelist.tsv +65 -0
- wetextprocessing-1.3.0/tn/italian/normalizer.py +60 -0
- wetextprocessing-1.3.0/tn/italian/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/italian/rules/cardinal.py +99 -0
- wetextprocessing-1.3.0/tn/italian/rules/decimal.py +82 -0
- wetextprocessing-1.3.0/tn/italian/rules/electronic.py +80 -0
- wetextprocessing-1.3.0/tn/italian/rules/measure.py +73 -0
- wetextprocessing-1.3.0/tn/italian/rules/money.py +92 -0
- wetextprocessing-1.3.0/tn/italian/rules/punctuation.py +30 -0
- wetextprocessing-1.3.0/tn/italian/rules/time.py +61 -0
- wetextprocessing-1.3.0/tn/italian/rules/utils.py +74 -0
- wetextprocessing-1.3.0/tn/italian/rules/whitelist.py +27 -0
- wetextprocessing-1.3.0/tn/italian/rules/word.py +28 -0
- wetextprocessing-1.3.0/tn/italian/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/italian/test/data/test_cases_cardinal.txt +13 -0
- wetextprocessing-1.3.0/tn/italian/test/data/test_cases_decimal.txt +4 -0
- wetextprocessing-1.3.0/tn/italian/test/data/test_cases_electronic.txt +13 -0
- wetextprocessing-1.3.0/tn/italian/test/data/test_cases_measure.txt +7 -0
- wetextprocessing-1.3.0/tn/italian/test/data/test_cases_money.txt +5 -0
- wetextprocessing-1.3.0/tn/italian/test/data/test_cases_time.txt +6 -0
- wetextprocessing-1.3.0/tn/italian/test/data/test_cases_whitelist.txt +4 -0
- wetextprocessing-1.3.0/tn/italian/test/normalizer_test.py +209 -0
- wetextprocessing-1.3.0/tn/japanese/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/japanese/data/default/blacklist.tsv +0 -0
- wetextprocessing-1.3.0/tn/japanese/normalizer.py +114 -0
- wetextprocessing-1.3.0/tn/japanese/rules/cardinal.py +113 -0
- wetextprocessing-1.3.0/tn/japanese/rules/date.py +82 -0
- wetextprocessing-1.3.0/tn/japanese/rules/fraction.py +43 -0
- wetextprocessing-1.3.0/tn/japanese/rules/math.py +45 -0
- wetextprocessing-1.3.0/tn/japanese/rules/measure.py +54 -0
- wetextprocessing-1.3.0/tn/japanese/rules/money.py +46 -0
- wetextprocessing-1.3.0/tn/japanese/rules/postprocessor.py +50 -0
- wetextprocessing-1.3.0/tn/japanese/rules/preprocessor.py +31 -0
- wetextprocessing-1.3.0/tn/japanese/rules/range.py +32 -0
- wetextprocessing-1.3.0/tn/japanese/rules/sport.py +58 -0
- wetextprocessing-1.3.0/tn/japanese/rules/time.py +65 -0
- wetextprocessing-1.3.0/tn/japanese/rules/transliteration.py +40 -0
- wetextprocessing-1.3.0/tn/japanese/rules/whitelist.py +40 -0
- wetextprocessing-1.3.0/tn/japanese/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/japanese/test/normalizer_test.py +145 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/data/numbers/digit_bi.tsv +9 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/data/numbers/digit_e.tsv +9 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/data/numbers/hundreds.tsv +9 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/data/numbers/tens.tsv +8 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/data/time/hours.tsv +12 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/data/time/minutes.tsv +60 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/data/whitelist.tsv +175 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/normalizer.py +56 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/rules/cardinal.py +117 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/rules/punctuation.py +31 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/rules/time.py +40 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/rules/utils.py +43 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/rules/whitelist.py +28 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/rules/word.py +29 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/test/data/test_cases_cardinal.txt +57 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/test/data/test_cases_time.txt +14 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/test/data/test_cases_whitelist.txt +3 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/test/data/test_cases_word.txt +26 -0
- wetextprocessing-1.3.0/tn/kinyarwanda/test/normalizer_test.py +198 -0
- wetextprocessing-1.3.0/tn/korean/__init__.py +16 -0
- wetextprocessing-1.3.0/tn/korean/data/date/exceptions.tsv +2 -0
- wetextprocessing-1.3.0/tn/korean/data/date/week.tsv +8 -0
- wetextprocessing-1.3.0/tn/korean/data/electronic/cc_cues.tsv +11 -0
- wetextprocessing-1.3.0/tn/korean/data/electronic/domain.tsv +28 -0
- wetextprocessing-1.3.0/tn/korean/data/electronic/extensions.tsv +6 -0
- wetextprocessing-1.3.0/tn/korean/data/electronic/symbol.tsv +12 -0
- wetextprocessing-1.3.0/tn/korean/data/measure/unit.tsv +18 -0
- wetextprocessing-1.3.0/tn/korean/data/money/currency_major.tsv +22 -0
- wetextprocessing-1.3.0/tn/korean/data/number/counter_suffix.tsv +16 -0
- wetextprocessing-1.3.0/tn/korean/data/number/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/korean/data/number/native_ones.tsv +9 -0
- wetextprocessing-1.3.0/tn/korean/data/number/tens.tsv +9 -0
- wetextprocessing-1.3.0/tn/korean/data/number/zero.tsv +1 -0
- wetextprocessing-1.3.0/tn/korean/data/ordinal/digit.tsv +8 -0
- wetextprocessing-1.3.0/tn/korean/data/ordinal/exceptions.tsv +5 -0
- wetextprocessing-1.3.0/tn/korean/data/ordinal/tens.tsv +2 -0
- wetextprocessing-1.3.0/tn/korean/data/ordinal/tens_prefix.tsv +3 -0
- wetextprocessing-1.3.0/tn/korean/data/time/division.tsv +7 -0
- wetextprocessing-1.3.0/tn/korean/data/time/hour.tsv +12 -0
- wetextprocessing-1.3.0/tn/korean/data/whitelist.tsv +39 -0
- wetextprocessing-1.3.0/tn/korean/normalizer.py +79 -0
- wetextprocessing-1.3.0/tn/korean/rules/__init__.py +3 -0
- wetextprocessing-1.3.0/tn/korean/rules/cardinal.py +91 -0
- wetextprocessing-1.3.0/tn/korean/rules/date.py +129 -0
- wetextprocessing-1.3.0/tn/korean/rules/decimal.py +53 -0
- wetextprocessing-1.3.0/tn/korean/rules/electronic.py +84 -0
- wetextprocessing-1.3.0/tn/korean/rules/fraction.py +81 -0
- wetextprocessing-1.3.0/tn/korean/rules/measure.py +82 -0
- wetextprocessing-1.3.0/tn/korean/rules/money.py +60 -0
- wetextprocessing-1.3.0/tn/korean/rules/ordinal.py +62 -0
- wetextprocessing-1.3.0/tn/korean/rules/punctuation.py +16 -0
- wetextprocessing-1.3.0/tn/korean/rules/telephone.py +40 -0
- wetextprocessing-1.3.0/tn/korean/rules/time.py +81 -0
- wetextprocessing-1.3.0/tn/korean/rules/whitelist.py +17 -0
- wetextprocessing-1.3.0/tn/korean/rules/word.py +18 -0
- wetextprocessing-1.3.0/tn/korean/test/__init__.py +3 -0
- wetextprocessing-1.3.0/tn/korean/test/data/test_cases_cardinal.txt +68 -0
- wetextprocessing-1.3.0/tn/korean/test/data/test_cases_date.txt +29 -0
- wetextprocessing-1.3.0/tn/korean/test/data/test_cases_decimal.txt +28 -0
- wetextprocessing-1.3.0/tn/korean/test/data/test_cases_electronic.txt +20 -0
- wetextprocessing-1.3.0/tn/korean/test/data/test_cases_fraction.txt +28 -0
- wetextprocessing-1.3.0/tn/korean/test/data/test_cases_measure.txt +40 -0
- wetextprocessing-1.3.0/tn/korean/test/data/test_cases_money.txt +64 -0
- wetextprocessing-1.3.0/tn/korean/test/data/test_cases_ordinal.txt +54 -0
- wetextprocessing-1.3.0/tn/korean/test/data/test_cases_telephone.txt +39 -0
- wetextprocessing-1.3.0/tn/korean/test/data/test_cases_time.txt +25 -0
- wetextprocessing-1.3.0/tn/korean/test/date_test.py +99 -0
- wetextprocessing-1.3.0/tn/korean/test/decimal_test.py +67 -0
- wetextprocessing-1.3.0/tn/korean/test/electronic_test.py +58 -0
- wetextprocessing-1.3.0/tn/korean/test/fraction_test.py +89 -0
- wetextprocessing-1.3.0/tn/korean/test/measure_test.py +90 -0
- wetextprocessing-1.3.0/tn/korean/test/money_test.py +92 -0
- wetextprocessing-1.3.0/tn/korean/test/normalizer_test.py +146 -0
- wetextprocessing-1.3.0/tn/korean/test/ordinal_test.py +78 -0
- wetextprocessing-1.3.0/tn/korean/test/telephone_test.py +40 -0
- wetextprocessing-1.3.0/tn/korean/test/time_test.py +51 -0
- wetextprocessing-1.3.0/tn/main.py +23 -0
- wetextprocessing-1.3.0/tn/marathi/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/marathi/data/date/eras.tsv +2 -0
- wetextprocessing-1.3.0/tn/marathi/data/date/months.tsv +12 -0
- wetextprocessing-1.3.0/tn/marathi/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/marathi/data/numbers/tens.tsv +90 -0
- wetextprocessing-1.3.0/tn/marathi/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/tn/marathi/normalizer.py +53 -0
- wetextprocessing-1.3.0/tn/marathi/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/marathi/rules/cardinal.py +76 -0
- wetextprocessing-1.3.0/tn/marathi/rules/date.py +70 -0
- wetextprocessing-1.3.0/tn/marathi/rules/decimal.py +59 -0
- wetextprocessing-1.3.0/tn/marathi/rules/punctuation.py +30 -0
- wetextprocessing-1.3.0/tn/marathi/rules/time.py +86 -0
- wetextprocessing-1.3.0/tn/marathi/rules/utils.py +76 -0
- wetextprocessing-1.3.0/tn/marathi/rules/word.py +28 -0
- wetextprocessing-1.3.0/tn/marathi/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/marathi/test/normalizer_test.py +221 -0
- wetextprocessing-1.3.0/tn/portuguese/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/portuguese/data/date/months.tsv +21 -0
- wetextprocessing-1.3.0/tn/portuguese/data/date/numeric_separators.tsv +3 -0
- wetextprocessing-1.3.0/tn/portuguese/data/date/verbal_phrases.tsv +1 -0
- wetextprocessing-1.3.0/tn/portuguese/data/electronic/domain.tsv +7 -0
- wetextprocessing-1.3.0/tn/portuguese/data/electronic/electronic_spoken_unit.tsv +16 -0
- wetextprocessing-1.3.0/tn/portuguese/data/electronic/server_name.tsv +6 -0
- wetextprocessing-1.3.0/tn/portuguese/data/electronic/symbols.tsv +22 -0
- wetextprocessing-1.3.0/tn/portuguese/data/fractions/ordinal_exceptions.tsv +2 -0
- wetextprocessing-1.3.0/tn/portuguese/data/fractions/powers_of_ten.tsv +1 -0
- wetextprocessing-1.3.0/tn/portuguese/data/fractions/specials.tsv +5 -0
- wetextprocessing-1.3.0/tn/portuguese/data/measure/measurements_plural.tsv +64 -0
- wetextprocessing-1.3.0/tn/portuguese/data/measure/measurements_singular.tsv +65 -0
- wetextprocessing-1.3.0/tn/portuguese/data/money/currency_major.tsv +5 -0
- wetextprocessing-1.3.0/tn/portuguese/data/money/currency_major_plural.tsv +5 -0
- wetextprocessing-1.3.0/tn/portuguese/data/money/currency_minor.tsv +5 -0
- wetextprocessing-1.3.0/tn/portuguese/data/money/currency_minor_plural.tsv +1 -0
- wetextprocessing-1.3.0/tn/portuguese/data/numbers/cardinal_specials.tsv +4 -0
- wetextprocessing-1.3.0/tn/portuguese/data/numbers/decimal_specials.tsv +2 -0
- wetextprocessing-1.3.0/tn/portuguese/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/portuguese/data/numbers/hundreds.tsv +8 -0
- wetextprocessing-1.3.0/tn/portuguese/data/numbers/quantity_words.tsv +9 -0
- wetextprocessing-1.3.0/tn/portuguese/data/numbers/scales.tsv +4 -0
- wetextprocessing-1.3.0/tn/portuguese/data/numbers/teens.tsv +10 -0
- wetextprocessing-1.3.0/tn/portuguese/data/numbers/tens.tsv +8 -0
- wetextprocessing-1.3.0/tn/portuguese/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/tn/portuguese/data/ordinals/digit.tsv +10 -0
- wetextprocessing-1.3.0/tn/portuguese/data/ordinals/feminine.tsv +11 -0
- wetextprocessing-1.3.0/tn/portuguese/data/ordinals/hundreds.tsv +10 -0
- wetextprocessing-1.3.0/tn/portuguese/data/ordinals/ties.tsv +8 -0
- wetextprocessing-1.3.0/tn/portuguese/data/telephone/ip_prompt.tsv +2 -0
- wetextprocessing-1.3.0/tn/portuguese/data/telephone/telephone_prompt.tsv +4 -0
- wetextprocessing-1.3.0/tn/portuguese/data/time/day_period_suffix.tsv +4 -0
- wetextprocessing-1.3.0/tn/portuguese/data/whitelist/symbol.tsv +23 -0
- wetextprocessing-1.3.0/tn/portuguese/normalizer.py +74 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/cardinal.py +170 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/date.py +69 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/decimal.py +67 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/electronic.py +82 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/fraction.py +79 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/measure.py +66 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/money.py +117 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/ordinal.py +82 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/punctuation.py +47 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/telephone.py +71 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/time.py +101 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/whitelist.py +37 -0
- wetextprocessing-1.3.0/tn/portuguese/rules/word.py +35 -0
- wetextprocessing-1.3.0/tn/portuguese/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_cardinal.txt +121 -0
- wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_date.txt +26 -0
- wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_decimal.txt +58 -0
- wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_electronic.txt +13 -0
- wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_fraction.txt +22 -0
- wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_measure.txt +18 -0
- wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_money.txt +25 -0
- wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_ordinal.txt +39 -0
- wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_telephone.txt +7 -0
- wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_time.txt +30 -0
- wetextprocessing-1.3.0/tn/portuguese/test/normalizer_test.py +166 -0
- wetextprocessing-1.3.0/tn/processor.py +518 -0
- wetextprocessing-1.3.0/tn/russian/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/russian/data/currency/currency_agreement.tsv +10 -0
- wetextprocessing-1.3.0/tn/russian/data/currency/currency_plural.tsv +53 -0
- wetextprocessing-1.3.0/tn/russian/data/currency/currency_singular.tsv +48 -0
- wetextprocessing-1.3.0/tn/russian/data/electronic/domain.tsv +9 -0
- wetextprocessing-1.3.0/tn/russian/data/electronic/server_name.tsv +17 -0
- wetextprocessing-1.3.0/tn/russian/data/electronic/symbols.tsv +28 -0
- wetextprocessing-1.3.0/tn/russian/data/latin_to_cyrillic.tsv +38 -0
- wetextprocessing-1.3.0/tn/russian/data/measurements.tsv +321 -0
- wetextprocessing-1.3.0/tn/russian/data/months/abbr.tsv +12 -0
- wetextprocessing-1.3.0/tn/russian/data/months/abbr_to_name.tsv +62 -0
- wetextprocessing-1.3.0/tn/russian/data/months/numbers.tsv +21 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/1_cardinals_nominative.tsv +50 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/2_cardinals_genitive.tsv +45 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/3_cardinals_dative.tsv +45 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/4_cardinals_accusative.tsv +48 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/5_cardinals_instrumental.tsv +47 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/6_cardinals_prepositional.tsv +46 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/cardinals_alternatives.tsv +16 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/cardinals_nominative_case.tsv +37 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/decimal_delimiter.tsv +15 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/decimal_endings.tsv +37 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/digits_nominative_case.tsv +10 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/ordinal_endings.tsv +40 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/ordinals.tsv +752 -0
- wetextprocessing-1.3.0/tn/russian/data/numbers/quantity.tsv +16 -0
- wetextprocessing-1.3.0/tn/russian/data/time/increment_hour_cardinal.tsv +24 -0
- wetextprocessing-1.3.0/tn/russian/data/time/increment_hour_ordinal.tsv +24 -0
- wetextprocessing-1.3.0/tn/russian/data/time/minutes_to_hour.tsv +25 -0
- wetextprocessing-1.3.0/tn/russian/data/time/time_convert.tsv +12 -0
- wetextprocessing-1.3.0/tn/russian/data/whitelist.tsv +37 -0
- wetextprocessing-1.3.0/tn/russian/normalizer.py +75 -0
- wetextprocessing-1.3.0/tn/russian/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/russian/rules/cardinal.py +92 -0
- wetextprocessing-1.3.0/tn/russian/rules/date.py +93 -0
- wetextprocessing-1.3.0/tn/russian/rules/decimal.py +88 -0
- wetextprocessing-1.3.0/tn/russian/rules/electronic.py +69 -0
- wetextprocessing-1.3.0/tn/russian/rules/measure.py +70 -0
- wetextprocessing-1.3.0/tn/russian/rules/money.py +73 -0
- wetextprocessing-1.3.0/tn/russian/rules/ordinal.py +45 -0
- wetextprocessing-1.3.0/tn/russian/rules/punctuation.py +46 -0
- wetextprocessing-1.3.0/tn/russian/rules/telephone.py +44 -0
- wetextprocessing-1.3.0/tn/russian/rules/time.py +80 -0
- wetextprocessing-1.3.0/tn/russian/rules/utils.py +293 -0
- wetextprocessing-1.3.0/tn/russian/rules/whitelist.py +41 -0
- wetextprocessing-1.3.0/tn/russian/rules/word.py +34 -0
- wetextprocessing-1.3.0/tn/russian/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/russian/test/data/test_cases_cardinal.txt +28 -0
- wetextprocessing-1.3.0/tn/russian/test/data/test_cases_cardinal_normalize_with_audio.txt +51 -0
- wetextprocessing-1.3.0/tn/russian/test/data/test_cases_date.txt +6 -0
- wetextprocessing-1.3.0/tn/russian/test/data/test_cases_decimal.txt +17 -0
- wetextprocessing-1.3.0/tn/russian/test/data/test_cases_electronic.txt +4 -0
- wetextprocessing-1.3.0/tn/russian/test/data/test_cases_measure.txt +13 -0
- wetextprocessing-1.3.0/tn/russian/test/data/test_cases_money.txt +9 -0
- wetextprocessing-1.3.0/tn/russian/test/data/test_cases_ordinal.txt +75 -0
- wetextprocessing-1.3.0/tn/russian/test/data/test_cases_telephone.txt +1 -0
- wetextprocessing-1.3.0/tn/russian/test/data/test_cases_time.txt +9 -0
- wetextprocessing-1.3.0/tn/russian/test/data/test_cases_whitelist.txt +4 -0
- wetextprocessing-1.3.0/tn/russian/test/data/test_cases_word.txt +3 -0
- wetextprocessing-1.3.0/tn/russian/test/normalizer_test.py +214 -0
- wetextprocessing-1.3.0/tn/spanish/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/spanish/data/dates/days_abbr.tsv +27 -0
- wetextprocessing-1.3.0/tn/spanish/data/dates/months.tsv +12 -0
- wetextprocessing-1.3.0/tn/spanish/data/dates/months_abbr.tsv +44 -0
- wetextprocessing-1.3.0/tn/spanish/data/dates/year_suffix.tsv +51 -0
- wetextprocessing-1.3.0/tn/spanish/data/electronic/domain.tsv +6 -0
- wetextprocessing-1.3.0/tn/spanish/data/electronic/server_name.tsv +11 -0
- wetextprocessing-1.3.0/tn/spanish/data/electronic/symbols.tsv +22 -0
- wetextprocessing-1.3.0/tn/spanish/data/fractions/fraction_symbols.tsv +15 -0
- wetextprocessing-1.3.0/tn/spanish/data/fractions/ordinal_exceptions.tsv +2 -0
- wetextprocessing-1.3.0/tn/spanish/data/fractions/powers_of_ten.tsv +9 -0
- wetextprocessing-1.3.0/tn/spanish/data/fractions/powers_of_ten_fem.tsv +6 -0
- wetextprocessing-1.3.0/tn/spanish/data/measures/math_symbols.tsv +7 -0
- wetextprocessing-1.3.0/tn/spanish/data/measures/measurements.tsv +18 -0
- wetextprocessing-1.3.0/tn/spanish/data/measures/measurements_complex.tsv +5 -0
- wetextprocessing-1.3.0/tn/spanish/data/measures/measurements_plural_fem.tsv +3 -0
- wetextprocessing-1.3.0/tn/spanish/data/measures/measurements_plural_masc.tsv +21 -0
- wetextprocessing-1.3.0/tn/spanish/data/money/currency_major.tsv +64 -0
- wetextprocessing-1.3.0/tn/spanish/data/money/currency_major_ext.tsv +215 -0
- wetextprocessing-1.3.0/tn/spanish/data/money/currency_minor.tsv +64 -0
- wetextprocessing-1.3.0/tn/spanish/data/money/currency_minor_ext.tsv +181 -0
- wetextprocessing-1.3.0/tn/spanish/data/money/currency_plural_fem.tsv +8 -0
- wetextprocessing-1.3.0/tn/spanish/data/money/currency_plural_fem_ext.tsv +56 -0
- wetextprocessing-1.3.0/tn/spanish/data/money/currency_plural_masc.tsv +51 -0
- wetextprocessing-1.3.0/tn/spanish/data/money/currency_plural_masc_ext.tsv +199 -0
- wetextprocessing-1.3.0/tn/spanish/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/spanish/data/numbers/hundreds.tsv +8 -0
- wetextprocessing-1.3.0/tn/spanish/data/numbers/quantities.tsv +6 -0
- wetextprocessing-1.3.0/tn/spanish/data/numbers/teen.tsv +10 -0
- wetextprocessing-1.3.0/tn/spanish/data/numbers/ties.tsv +7 -0
- wetextprocessing-1.3.0/tn/spanish/data/numbers/twenties.tsv +10 -0
- wetextprocessing-1.3.0/tn/spanish/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/tn/spanish/data/ordinals/digit.tsv +11 -0
- wetextprocessing-1.3.0/tn/spanish/data/ordinals/gender_suffix.tsv +20 -0
- wetextprocessing-1.3.0/tn/spanish/data/ordinals/hundreds.tsv +18 -0
- wetextprocessing-1.3.0/tn/spanish/data/ordinals/roman_exceptions.tsv +10 -0
- wetextprocessing-1.3.0/tn/spanish/data/ordinals/teen.tsv +10 -0
- wetextprocessing-1.3.0/tn/spanish/data/ordinals/ties.tsv +8 -0
- wetextprocessing-1.3.0/tn/spanish/data/ordinals/twenties.tsv +11 -0
- wetextprocessing-1.3.0/tn/spanish/data/roman/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/spanish/data/roman/hundreds.tsv +9 -0
- wetextprocessing-1.3.0/tn/spanish/data/roman/thousands.tsv +3 -0
- wetextprocessing-1.3.0/tn/spanish/data/roman/ties.tsv +9 -0
- wetextprocessing-1.3.0/tn/spanish/data/telephone/ip_prompt.tsv +2 -0
- wetextprocessing-1.3.0/tn/spanish/data/telephone/telephone_prompt.tsv +10 -0
- wetextprocessing-1.3.0/tn/spanish/data/time/afternoon_times.tsv +6 -0
- wetextprocessing-1.3.0/tn/spanish/data/time/alt_minutes.tsv +2 -0
- wetextprocessing-1.3.0/tn/spanish/data/time/evening_times.tsv +5 -0
- wetextprocessing-1.3.0/tn/spanish/data/time/hour_to_12.tsv +12 -0
- wetextprocessing-1.3.0/tn/spanish/data/time/hour_to_24.tsv +25 -0
- wetextprocessing-1.3.0/tn/spanish/data/time/hour_to_night.tsv +13 -0
- wetextprocessing-1.3.0/tn/spanish/data/time/minute_to.tsv +59 -0
- wetextprocessing-1.3.0/tn/spanish/data/time/morning_times.tsv +12 -0
- wetextprocessing-1.3.0/tn/spanish/data/time/time_suffix.tsv +20 -0
- wetextprocessing-1.3.0/tn/spanish/data/time/time_zone.tsv +42 -0
- wetextprocessing-1.3.0/tn/spanish/data/time/time_zone_ext.tsv +105 -0
- wetextprocessing-1.3.0/tn/spanish/data/whitelist.tsv +62 -0
- wetextprocessing-1.3.0/tn/spanish/normalizer.py +79 -0
- wetextprocessing-1.3.0/tn/spanish/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/spanish/rules/cardinal.py +125 -0
- wetextprocessing-1.3.0/tn/spanish/rules/date.py +96 -0
- wetextprocessing-1.3.0/tn/spanish/rules/decimal.py +122 -0
- wetextprocessing-1.3.0/tn/spanish/rules/electronic.py +69 -0
- wetextprocessing-1.3.0/tn/spanish/rules/fraction.py +189 -0
- wetextprocessing-1.3.0/tn/spanish/rules/measure.py +177 -0
- wetextprocessing-1.3.0/tn/spanish/rules/money.py +186 -0
- wetextprocessing-1.3.0/tn/spanish/rules/ordinal.py +119 -0
- wetextprocessing-1.3.0/tn/spanish/rules/punctuation.py +48 -0
- wetextprocessing-1.3.0/tn/spanish/rules/telephone.py +93 -0
- wetextprocessing-1.3.0/tn/spanish/rules/time.py +171 -0
- wetextprocessing-1.3.0/tn/spanish/rules/utils.py +170 -0
- wetextprocessing-1.3.0/tn/spanish/rules/whitelist.py +39 -0
- wetextprocessing-1.3.0/tn/spanish/rules/word.py +39 -0
- wetextprocessing-1.3.0/tn/spanish/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_cardinal.txt +69 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_date.txt +21 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_decimal.txt +24 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_electronic.txt +22 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_fraction.txt +75 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_measure.txt +32 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_money.txt +69 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_normalize_with_audio.txt +162 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_ordinal.txt +120 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_telephone.txt +15 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_time.txt +35 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_whitelist.txt +7 -0
- wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_word.txt +48 -0
- wetextprocessing-1.3.0/tn/spanish/test/normalizer_test.py +166 -0
- wetextprocessing-1.3.0/tn/swedish/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/swedish/data/abbreviations_nondet.tsv +27 -0
- wetextprocessing-1.3.0/tn/swedish/data/dates/era_suffix.tsv +18 -0
- wetextprocessing-1.3.0/tn/swedish/data/dates/era_words.tsv +4 -0
- wetextprocessing-1.3.0/tn/swedish/data/dates/month_abbr.tsv +17 -0
- wetextprocessing-1.3.0/tn/swedish/data/dates/months.tsv +12 -0
- wetextprocessing-1.3.0/tn/swedish/data/electronic/domain.tsv +10 -0
- wetextprocessing-1.3.0/tn/swedish/data/electronic/server_name.tsv +29 -0
- wetextprocessing-1.3.0/tn/swedish/data/electronic/symbols.tsv +20 -0
- wetextprocessing-1.3.0/tn/swedish/data/math_operations.tsv +8 -0
- wetextprocessing-1.3.0/tn/swedish/data/measure/greek_lower.tsv +25 -0
- wetextprocessing-1.3.0/tn/swedish/data/measure/greek_upper.tsv +25 -0
- wetextprocessing-1.3.0/tn/swedish/data/measure/unit.tsv +147 -0
- wetextprocessing-1.3.0/tn/swedish/data/measure/unit_neuter.tsv +13 -0
- wetextprocessing-1.3.0/tn/swedish/data/measure/unit_plural.tsv +121 -0
- wetextprocessing-1.3.0/tn/swedish/data/money/currency_major.tsv +75 -0
- wetextprocessing-1.3.0/tn/swedish/data/money/currency_major_nt.tsv +2 -0
- wetextprocessing-1.3.0/tn/swedish/data/money/currency_minor_plural.tsv +79 -0
- wetextprocessing-1.3.0/tn/swedish/data/money/currency_minor_singular.tsv +78 -0
- wetextprocessing-1.3.0/tn/swedish/data/money/currency_plurals.tsv +44 -0
- wetextprocessing-1.3.0/tn/swedish/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/swedish/data/numbers/fraction.tsv +18 -0
- wetextprocessing-1.3.0/tn/swedish/data/numbers/millions.tsv +88 -0
- wetextprocessing-1.3.0/tn/swedish/data/numbers/millions_abbr.tsv +4 -0
- wetextprocessing-1.3.0/tn/swedish/data/numbers/teen.tsv +10 -0
- wetextprocessing-1.3.0/tn/swedish/data/numbers/ties.tsv +8 -0
- wetextprocessing-1.3.0/tn/swedish/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/tn/swedish/data/ordinals/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/swedish/data/ordinals/teen.tsv +10 -0
- wetextprocessing-1.3.0/tn/swedish/data/ordinals/ties.tsv +8 -0
- wetextprocessing-1.3.0/tn/swedish/data/ordinals/zero.tsv +1 -0
- wetextprocessing-1.3.0/tn/swedish/data/telephone/ip_prompt.tsv +2 -0
- wetextprocessing-1.3.0/tn/swedish/data/telephone/special_numbers.tsv +4 -0
- wetextprocessing-1.3.0/tn/swedish/data/telephone/ssn_prompt.tsv +3 -0
- wetextprocessing-1.3.0/tn/swedish/data/telephone/telephone_abbr.tsv +24 -0
- wetextprocessing-1.3.0/tn/swedish/data/telephone/telephone_prompt.tsv +5 -0
- wetextprocessing-1.3.0/tn/swedish/data/time/suffix.tsv +16 -0
- wetextprocessing-1.3.0/tn/swedish/data/time/time_zone.tsv +7 -0
- wetextprocessing-1.3.0/tn/swedish/data/whitelist.tsv +58 -0
- wetextprocessing-1.3.0/tn/swedish/normalizer.py +75 -0
- wetextprocessing-1.3.0/tn/swedish/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/swedish/rules/cardinal.py +172 -0
- wetextprocessing-1.3.0/tn/swedish/rules/date.py +122 -0
- wetextprocessing-1.3.0/tn/swedish/rules/decimal.py +110 -0
- wetextprocessing-1.3.0/tn/swedish/rules/electronic.py +75 -0
- wetextprocessing-1.3.0/tn/swedish/rules/fraction.py +85 -0
- wetextprocessing-1.3.0/tn/swedish/rules/measure.py +143 -0
- wetextprocessing-1.3.0/tn/swedish/rules/money.py +127 -0
- wetextprocessing-1.3.0/tn/swedish/rules/ordinal.py +132 -0
- wetextprocessing-1.3.0/tn/swedish/rules/punctuation.py +47 -0
- wetextprocessing-1.3.0/tn/swedish/rules/telephone.py +89 -0
- wetextprocessing-1.3.0/tn/swedish/rules/time.py +98 -0
- wetextprocessing-1.3.0/tn/swedish/rules/utils.py +63 -0
- wetextprocessing-1.3.0/tn/swedish/rules/whitelist.py +38 -0
- wetextprocessing-1.3.0/tn/swedish/rules/word.py +33 -0
- wetextprocessing-1.3.0/tn/swedish/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_cardinal.txt +68 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_date.txt +17 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_decimal.txt +7 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_electronic.txt +12 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_fraction.txt +2 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_measure.txt +7 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_money.txt +65 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_normalize_with_audio.txt +14 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_ordinal.txt +7 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_telephone.txt +7 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_time.txt +12 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_whitelist.txt +1 -0
- wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_word.txt +48 -0
- wetextprocessing-1.3.0/tn/swedish/test/normalizer_test.py +195 -0
- wetextprocessing-1.3.0/tn/test/cli_test.py +337 -0
- wetextprocessing-1.3.0/tn/test/punctuation_spacing_test.py +22 -0
- wetextprocessing-1.3.0/tn/test/robustness_test.py +32 -0
- wetextprocessing-1.3.0/tn/token_parser.py +217 -0
- wetextprocessing-1.3.0/tn/vietnamese/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/date/days.tsv +40 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/date/months.tsv +21 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/date/year_suffix.tsv +4 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/fraction/denominator_exceptions.tsv +1 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/measure/base_units.tsv +20 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/measure/measurements_minimal.tsv +33 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/measure/prefixes.tsv +17 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/money/currency.tsv +51 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/money/currency_minor.tsv +7 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/money/per_unit_bases.tsv +8 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/money/per_unit_non_metric.tsv +28 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/money/per_unit_prefixes.tsv +6 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/numbers/digit.tsv +9 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/numbers/digit_special.tsv +3 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/numbers/magnitudes.tsv +8 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/numbers/quantity_abbr.tsv +7 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/numbers/teen.tsv +10 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/numbers/ties.tsv +8 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/numbers/zero.tsv +1 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/ordinal/ordinal_exceptions.tsv +2 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/roman/key_word.tsv +16 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/roman/roman_numerals.tsv +13 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/time/time_units.tsv +4 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/time/time_zones.tsv +18 -0
- wetextprocessing-1.3.0/tn/vietnamese/data/whitelist.tsv +0 -0
- wetextprocessing-1.3.0/tn/vietnamese/normalizer.py +106 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/cardinal.py +163 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/date.py +81 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/decimal.py +85 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/fraction.py +50 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/measure.py +57 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/money.py +87 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/ordinal.py +34 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/punctuation.py +31 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/range.py +36 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/roman.py +59 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/time.py +86 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/whitelist.py +29 -0
- wetextprocessing-1.3.0/tn/vietnamese/rules/word.py +36 -0
- wetextprocessing-1.3.0/tn/vietnamese/test/__init__.py +0 -0
- wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_cardinal.txt +109 -0
- wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_date.txt +13 -0
- wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_decimal.txt +29 -0
- wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_fraction.txt +13 -0
- wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_measure.txt +63 -0
- wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_money.txt +30 -0
- wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_ordinal.txt +30 -0
- wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_range.txt +11 -0
- wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_roman.txt +57 -0
- wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_time.txt +117 -0
- wetextprocessing-1.3.0/tn/vietnamese/test/normalizer_test.py +146 -0
- wetextprocessing-1.2.0/.github/workflows/runtime.yml +0 -22
- wetextprocessing-1.2.0/.github/workflows/unittest.yml +0 -83
- wetextprocessing-1.2.0/.github/workflows/wheels.yml +0 -71
- wetextprocessing-1.2.0/.gitignore +0 -47
- wetextprocessing-1.2.0/.pre-commit-config.yaml +0 -23
- wetextprocessing-1.2.0/PKG-INFO +0 -160
- wetextprocessing-1.2.0/README.md +0 -137
- wetextprocessing-1.2.0/WeTextProcessing.egg-info/PKG-INFO +0 -160
- wetextprocessing-1.2.0/WeTextProcessing.egg-info/SOURCES.txt +0 -527
- wetextprocessing-1.2.0/itn/README.md +0 -60
- wetextprocessing-1.2.0/itn/chinese/inverse_normalizer.py +0 -98
- wetextprocessing-1.2.0/itn/chinese/rules/cardinal.py +0 -178
- wetextprocessing-1.2.0/itn/chinese/rules/date.py +0 -56
- wetextprocessing-1.2.0/itn/chinese/rules/fraction.py +0 -56
- wetextprocessing-1.2.0/itn/chinese/rules/license_plate.py +0 -37
- wetextprocessing-1.2.0/itn/chinese/rules/math.py +0 -37
- wetextprocessing-1.2.0/itn/chinese/rules/measure.py +0 -116
- wetextprocessing-1.2.0/itn/chinese/rules/money.py +0 -59
- wetextprocessing-1.2.0/itn/chinese/rules/postprocessor.py +0 -32
- wetextprocessing-1.2.0/itn/chinese/rules/time.py +0 -60
- wetextprocessing-1.2.0/itn/chinese/rules/train_number.py +0 -36
- wetextprocessing-1.2.0/itn/chinese/rules/whitelist.py +0 -33
- wetextprocessing-1.2.0/itn/chinese/test/normalizer_test.py +0 -117
- wetextprocessing-1.2.0/itn/english/inverse_normalizer.py +0 -96
- wetextprocessing-1.2.0/itn/english/rules/cardinal.py +0 -136
- wetextprocessing-1.2.0/itn/english/rules/date.py +0 -220
- wetextprocessing-1.2.0/itn/english/rules/decimal.py +0 -85
- wetextprocessing-1.2.0/itn/english/rules/electronic.py +0 -76
- wetextprocessing-1.2.0/itn/english/rules/measure.py +0 -115
- wetextprocessing-1.2.0/itn/english/rules/money.py +0 -145
- wetextprocessing-1.2.0/itn/english/rules/ordinal.py +0 -52
- wetextprocessing-1.2.0/itn/english/rules/telephone.py +0 -123
- wetextprocessing-1.2.0/itn/english/rules/time.py +0 -123
- wetextprocessing-1.2.0/itn/english/rules/whitelist.py +0 -32
- wetextprocessing-1.2.0/itn/english/rules/word.py +0 -31
- wetextprocessing-1.2.0/itn/english/test/normalizer_test.py +0 -43
- wetextprocessing-1.2.0/itn/japanese/inverse_normalizer.py +0 -96
- wetextprocessing-1.2.0/itn/japanese/rules/cardinal.py +0 -158
- wetextprocessing-1.2.0/itn/japanese/rules/date.py +0 -71
- wetextprocessing-1.2.0/itn/japanese/rules/fraction.py +0 -64
- wetextprocessing-1.2.0/itn/japanese/rules/math.py +0 -39
- wetextprocessing-1.2.0/itn/japanese/rules/measure.py +0 -48
- wetextprocessing-1.2.0/itn/japanese/rules/money.py +0 -45
- wetextprocessing-1.2.0/itn/japanese/rules/ordinal.py +0 -34
- wetextprocessing-1.2.0/itn/japanese/rules/postprocessor.py +0 -46
- wetextprocessing-1.2.0/itn/japanese/rules/preprocessor.py +0 -31
- wetextprocessing-1.2.0/itn/japanese/rules/time.py +0 -53
- wetextprocessing-1.2.0/itn/japanese/rules/whitelist.py +0 -33
- wetextprocessing-1.2.0/itn/japanese/test/normalizer_test.py +0 -113
- wetextprocessing-1.2.0/itn/main.py +0 -62
- wetextprocessing-1.2.0/itn/zh_itn_tagger.fst +0 -0
- wetextprocessing-1.2.0/itn/zh_itn_verbalizer.fst +0 -0
- wetextprocessing-1.2.0/pyproject.toml +0 -57
- wetextprocessing-1.2.0/runtime/CMakeLists.txt +0 -38
- wetextprocessing-1.2.0/runtime/android/.gitignore +0 -15
- wetextprocessing-1.2.0/runtime/android/app/build.gradle +0 -105
- wetextprocessing-1.2.0/runtime/android/app/src/androidTest/java/com/mobvoi/WeTextProcessing/ExampleInstrumentedTest.java +0 -26
- wetextprocessing-1.2.0/runtime/android/app/src/main/AndroidManifest.xml +0 -22
- wetextprocessing-1.2.0/runtime/android/app/src/main/assets/README.md +0 -1
- wetextprocessing-1.2.0/runtime/android/app/src/main/cpp/CMakeLists.txt +0 -22
- wetextprocessing-1.2.0/runtime/android/app/src/main/cpp/wetextprocessing.cc +0 -80
- wetextprocessing-1.2.0/runtime/android/app/src/main/java/com/mobvoi/WeTextProcessing/MainActivity.java +0 -78
- wetextprocessing-1.2.0/runtime/android/app/src/main/java/com/mobvoi/WeTextProcessing/WeTextProcessing.java +0 -12
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/drawable/ic_launcher_background.xml +0 -170
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/drawable-v24/ic_launcher_foreground.xml +0 -30
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/layout/activity_main.xml +0 -59
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/mipmap-hdpi/ic_launcher.png +0 -0
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/mipmap-hdpi/ic_launcher_round.png +0 -0
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/mipmap-mdpi/ic_launcher.png +0 -0
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/mipmap-mdpi/ic_launcher_round.png +0 -0
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/mipmap-xhdpi/ic_launcher.png +0 -0
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/mipmap-xhdpi/ic_launcher_round.png +0 -0
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png +0 -0
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.png +0 -0
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png +0 -0
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.png +0 -0
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/values/attrs.xml +0 -17
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/values/colors.xml +0 -16
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/values/themes.xml +0 -16
- wetextprocessing-1.2.0/runtime/android/app/src/main/res/values-night/themes.xml +0 -16
- wetextprocessing-1.2.0/runtime/android/app/src/test/java/com/mobvoi/WeTextProcessing/ExampleUnitTest.java +0 -17
- wetextprocessing-1.2.0/runtime/android/build.gradle +0 -21
- wetextprocessing-1.2.0/runtime/android/gradle/wrapper/gradle-wrapper.properties +0 -5
- wetextprocessing-1.2.0/runtime/android/gradle.properties +0 -19
- wetextprocessing-1.2.0/runtime/android/settings.gradle +0 -2
- wetextprocessing-1.2.0/runtime/bin/CMakeLists.txt +0 -3
- wetextprocessing-1.2.0/runtime/cmake/glog.cmake +0 -6
- wetextprocessing-1.2.0/runtime/cmake/openfst.cmake +0 -43
- wetextprocessing-1.2.0/runtime/patch/CPPLINT.cfg +0 -1
- wetextprocessing-1.2.0/runtime/patch/openfst/src/CMakeLists.txt +0 -23
- wetextprocessing-1.2.0/runtime/patch/openfst/src/extensions/special/CMakeLists.txt +0 -67
- wetextprocessing-1.2.0/runtime/patch/openfst/src/include/fst/flags.h +0 -219
- wetextprocessing-1.2.0/runtime/patch/openfst/src/include/fst/log.h +0 -78
- wetextprocessing-1.2.0/runtime/patch/openfst/src/lib/flags.cc +0 -160
- wetextprocessing-1.2.0/runtime/patch/openfst/src/test/CMakeLists.txt +0 -55
- wetextprocessing-1.2.0/runtime/processor/CMakeLists.txt +0 -22
- wetextprocessing-1.2.0/runtime/processor/wetext_processor.cc +0 -81
- wetextprocessing-1.2.0/runtime/processor/wetext_token_parser.cc +0 -167
- wetextprocessing-1.2.0/runtime/processor/wetext_token_parser.h +0 -96
- wetextprocessing-1.2.0/runtime/test/CMakeLists.txt +0 -18
- wetextprocessing-1.2.0/runtime/test/processor_test.cc +0 -72
- wetextprocessing-1.2.0/runtime/test/token_parser_test.cc +0 -103
- wetextprocessing-1.2.0/runtime/utils/wetext_flags.h +0 -23
- wetextprocessing-1.2.0/runtime/utils/wetext_log.h +0 -23
- wetextprocessing-1.2.0/tn/README.md +0 -69
- wetextprocessing-1.2.0/tn/_version.py +0 -34
- wetextprocessing-1.2.0/tn/chinese/normalizer.py +0 -104
- wetextprocessing-1.2.0/tn/chinese/rules/cardinal.py +0 -102
- wetextprocessing-1.2.0/tn/chinese/rules/date.py +0 -64
- wetextprocessing-1.2.0/tn/chinese/rules/fraction.py +0 -49
- wetextprocessing-1.2.0/tn/chinese/rules/math.py +0 -40
- wetextprocessing-1.2.0/tn/chinese/rules/measure.py +0 -69
- wetextprocessing-1.2.0/tn/chinese/rules/money.py +0 -51
- wetextprocessing-1.2.0/tn/chinese/rules/postprocessor.py +0 -48
- wetextprocessing-1.2.0/tn/chinese/rules/preprocessor.py +0 -31
- wetextprocessing-1.2.0/tn/chinese/rules/sport.py +0 -47
- wetextprocessing-1.2.0/tn/chinese/rules/time.py +0 -59
- wetextprocessing-1.2.0/tn/chinese/rules/whitelist.py +0 -45
- wetextprocessing-1.2.0/tn/chinese/test/data/normalizer.txt +0 -48
- wetextprocessing-1.2.0/tn/chinese/test/normalizer_test.py +0 -41
- wetextprocessing-1.2.0/tn/chinese/test/postprocessor_test.py +0 -29
- wetextprocessing-1.2.0/tn/chinese/test/preprocessor_test.py +0 -29
- wetextprocessing-1.2.0/tn/chinese/test/token_parser_test.py +0 -76
- wetextprocessing-1.2.0/tn/en_tn_tagger.fst +0 -0
- wetextprocessing-1.2.0/tn/en_tn_verbalizer.fst +0 -0
- wetextprocessing-1.2.0/tn/english/data/measure/unit_alternatives.tsv +0 -43
- wetextprocessing-1.2.0/tn/english/normalizer.py +0 -98
- wetextprocessing-1.2.0/tn/english/rules/cardinal.py +0 -173
- wetextprocessing-1.2.0/tn/english/rules/date.py +0 -352
- wetextprocessing-1.2.0/tn/english/rules/decimal.py +0 -192
- wetextprocessing-1.2.0/tn/english/rules/electronic.py +0 -199
- wetextprocessing-1.2.0/tn/english/rules/fraction.py +0 -122
- wetextprocessing-1.2.0/tn/english/rules/measure.py +0 -354
- wetextprocessing-1.2.0/tn/english/rules/money.py +0 -114
- wetextprocessing-1.2.0/tn/english/rules/ordinal.py +0 -91
- wetextprocessing-1.2.0/tn/english/rules/punctuation.py +0 -81
- wetextprocessing-1.2.0/tn/english/rules/range.py +0 -131
- wetextprocessing-1.2.0/tn/english/rules/roman.py +0 -159
- wetextprocessing-1.2.0/tn/english/rules/serial.py +0 -106
- wetextprocessing-1.2.0/tn/english/rules/telephone.py +0 -165
- wetextprocessing-1.2.0/tn/english/rules/time.py +0 -188
- wetextprocessing-1.2.0/tn/english/rules/whitelist.py +0 -140
- wetextprocessing-1.2.0/tn/english/rules/word.py +0 -52
- wetextprocessing-1.2.0/tn/english/test/data/date.txt +0 -14
- wetextprocessing-1.2.0/tn/english/test/data/normalizer.txt +0 -8
- wetextprocessing-1.2.0/tn/english/test/electronic_test.py +0 -28
- wetextprocessing-1.2.0/tn/english/test/normalizer_test.py +0 -28
- wetextprocessing-1.2.0/tn/japanese/normalizer.py +0 -105
- wetextprocessing-1.2.0/tn/japanese/rules/cardinal.py +0 -132
- wetextprocessing-1.2.0/tn/japanese/rules/char.py +0 -29
- wetextprocessing-1.2.0/tn/japanese/rules/date.py +0 -67
- wetextprocessing-1.2.0/tn/japanese/rules/fraction.py +0 -49
- wetextprocessing-1.2.0/tn/japanese/rules/math.py +0 -37
- wetextprocessing-1.2.0/tn/japanese/rules/measure.py +0 -48
- wetextprocessing-1.2.0/tn/japanese/rules/money.py +0 -51
- wetextprocessing-1.2.0/tn/japanese/rules/postprocessor.py +0 -46
- wetextprocessing-1.2.0/tn/japanese/rules/preprocessor.py +0 -31
- wetextprocessing-1.2.0/tn/japanese/rules/sport.py +0 -50
- wetextprocessing-1.2.0/tn/japanese/rules/time.py +0 -54
- wetextprocessing-1.2.0/tn/japanese/rules/transliteration.py +0 -33
- wetextprocessing-1.2.0/tn/japanese/rules/whitelist.py +0 -36
- wetextprocessing-1.2.0/tn/japanese/test/normalizer_test.py +0 -42
- wetextprocessing-1.2.0/tn/main.py +0 -73
- wetextprocessing-1.2.0/tn/processor.py +0 -132
- wetextprocessing-1.2.0/tn/token_parser.py +0 -157
- wetextprocessing-1.2.0/tn/zh_tn_tagger.fst +0 -0
- wetextprocessing-1.2.0/tn/zh_tn_verbalizer.fst +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/.clang-format +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/.flake8 +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/CPPLINT.cfg +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/LICENSE +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/WeTextProcessing.egg-info/dependency_links.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/WeTextProcessing.egg-info/entry_points.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/WeTextProcessing.egg-info/requires.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/WeTextProcessing.egg-info/top_level.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/__main__.py +0 -0
- {wetextprocessing-1.2.0/itn/chinese → wetextprocessing-1.3.0/itn/arabic}/__init__.py +0 -0
- {wetextprocessing-1.2.0/itn/chinese → wetextprocessing-1.3.0/itn/arabic}/rules/__init__.py +0 -0
- {wetextprocessing-1.2.0/itn/english → wetextprocessing-1.3.0/itn/arabic/test}/__init__.py +0 -0
- {wetextprocessing-1.2.0/itn/english/rules → wetextprocessing-1.3.0/itn/armenian}/__init__.py +0 -0
- {wetextprocessing-1.2.0/itn/english/test → wetextprocessing-1.3.0/itn/armenian/rules}/__init__.py +0 -0
- {wetextprocessing-1.2.0/itn/japanese → wetextprocessing-1.3.0/itn/armenian/test}/__init__.py +0 -0
- {wetextprocessing-1.2.0/itn/japanese/rules → wetextprocessing-1.3.0/itn/chinese}/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/date/dd.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/date/mm.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/default/blacklist.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/default/whitelist.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/license_plate/province.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/math/operator.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/measure/units_en.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/measure/units_zh.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/money/code.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/money/symbol.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/number/digit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/number/digit_zh.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/number/dot.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/number/repeat_digit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/number/sign.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/number/special_dash.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/number/special_tilde.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/number/zero.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/time/hour.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/time/minute.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/time/minute_zero.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/time/noon.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/time/second.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/data/train_number/prefix.tsv +0 -0
- {wetextprocessing-1.2.0/itn/japanese/test → wetextprocessing-1.3.0/itn/chinese/rules}/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/rules/char.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/cardinal.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/char.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/date.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/fraction.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/license_plate.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/math.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/measure.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/money.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/normalizer.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/normalizer_disable_standalone_number_disable_0_to_9.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/normalizer_disable_standalone_number_enable_0_to_9.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/normalizer_enable_standalone_number_disable_0_to_9.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/number.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/time.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/train_number.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/data/whitelist.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/chinese/test/utils.py +0 -0
- {wetextprocessing-1.2.0/tn → wetextprocessing-1.3.0/itn/english}/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/currency.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/date_period.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/electronic/domain.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/electronic/server_name.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/electronic/symbols.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/electronic/url_symbols.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/magnitudes.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/measurements.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/months.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/months_cased.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/numbers/digit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/numbers/hundred.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/numbers/teen.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/numbers/thousands.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/numbers/ties.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/numbers/zero.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/ordinals/digit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/ordinals/teen.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/time/minute_to.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/time/time_suffix.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/time/time_suffix_cased.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/time/time_zone.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/time/time_zone_cased.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/time/to_hour.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/whitelist.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/whitelist_tech.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/data/year_suffix.tsv +0 -0
- {wetextprocessing-1.2.0/tn/chinese → wetextprocessing-1.3.0/itn/english/rules}/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/rules/char.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/rules/punctuation.py +0 -0
- {wetextprocessing-1.2.0/tn/chinese/rules → wetextprocessing-1.3.0/itn/english/test}/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/test/data/en_cardinal.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/test/data/en_date.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/test/data/en_decimal.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/test/data/en_electronic.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/test/data/en_measure.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/test/data/en_money.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/test/data/en_ordinal.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/test/data/en_telephone.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/test/data/en_time.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/test/data/en_whitelist.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/test/data/en_word.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/english/test/utils.py +0 -0
- {wetextprocessing-1.2.0/tn/english → wetextprocessing-1.3.0/itn/french}/__init__.py +0 -0
- {wetextprocessing-1.2.0/tn/english/data → wetextprocessing-1.3.0/itn/french/rules}/__init__.py +0 -0
- {wetextprocessing-1.2.0/tn/english/data/date → wetextprocessing-1.3.0/itn/french/test}/__init__.py +0 -0
- {wetextprocessing-1.2.0/tn/english/data/electronic → wetextprocessing-1.3.0/itn/german}/__init__.py +0 -0
- {wetextprocessing-1.2.0/tn/english/data/money → wetextprocessing-1.3.0/itn/german/rules}/__init__.py +0 -0
- {wetextprocessing-1.2.0/tn/english/data/ordinal → wetextprocessing-1.3.0/itn/german/test}/__init__.py +0 -0
- {wetextprocessing-1.2.0/tn/english/data/roman → wetextprocessing-1.3.0/itn/hebrew}/__init__.py +0 -0
- {wetextprocessing-1.2.0/tn/english/data/telephone → wetextprocessing-1.3.0/itn/hebrew/rules}/__init__.py +0 -0
- {wetextprocessing-1.2.0/tn/english/data/whitelist → wetextprocessing-1.3.0/itn/hebrew/test}/__init__.py +0 -0
- {wetextprocessing-1.2.0/tn/english/rules → wetextprocessing-1.3.0/itn/hindi}/__init__.py +0 -0
- {wetextprocessing-1.2.0/tn/english/test → wetextprocessing-1.3.0/itn/hindi/rules}/__init__.py +0 -0
- {wetextprocessing-1.2.0/tn/japanese → wetextprocessing-1.3.0/itn/hindi/test}/__init__.py +0 -0
- {wetextprocessing-1.2.0/tn/japanese/test → wetextprocessing-1.3.0/itn/hindi_english}/__init__.py +0 -0
- /wetextprocessing-1.2.0/itn/japanese/data/default/blacklist.tsv → /wetextprocessing-1.3.0/itn/hindi_english/rules/__init__.py +0 -0
- /wetextprocessing-1.2.0/tn/japanese/data/default/blacklist.tsv → /wetextprocessing-1.3.0/itn/hindi_english/test/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/char/common_chinese_char.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/char/fullwidth_to_halfwidth.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/char/hiragana_and_katakana.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/char/oov_tags.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/char/punctuations_ja.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/date/day.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/date/month.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/date/week.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/default/whitelist.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/math/operator.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/measure/unit_en.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/measure/unit_ja.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/money/symbol.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/number/digit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/number/dot.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/number/hundred.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/number/hundred_digit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/number/sign.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/number/teen.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/number/thousands.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/number/ties.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/number/zero.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/time/hour.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/time/minute.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/data/time/second.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/cardinal.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/char.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/date.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/fraction.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/math.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/measure.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/money.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/normalizer_disable_standalone_number_disable_0_to_9.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/normalizer_disable_standalone_number_enable_0_to_9.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/normalizer_enable_standalone_number_disable_0_to_9.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/number.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/time.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/data/whitelist.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/itn/japanese/test/utils.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/README.md +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/android/app/.gitignore +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/android/app/proguard-rules.pro +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/android/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/android/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/android/app/src/main/res/values/strings.xml +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/android/app/wenet.keystore +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/android/gradle/wrapper/gradle-wrapper.jar +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/android/gradlew +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/android/gradlew.bat +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/bin/processor_main.cc +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/cmake/gflags.cmake +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/cmake/gtest.cmake +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/processor/wetext_processor.h +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/processor/wetext_processor_c_api.cc +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/processor/wetext_processor_c_api.h +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/test/string_test.cc +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/utils/CMakeLists.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/utils/wetext_string.cc +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/runtime/utils/wetext_string.h +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/setup.cfg +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/__main__.py +0 -0
- {wetextprocessing-1.2.0/tn/english → wetextprocessing-1.3.0/tn/arabic}/data/number/fraction.tsv +0 -0
- {wetextprocessing-1.2.0/tn/english/data/whitelist → wetextprocessing-1.3.0/tn/arabic/data/punctuation}/symbol.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/cardinal/phone.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/char/charset_extension.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/char/charset_national_standard_2013_8105.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/char/fullwidth_to_halfwidth.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/char/punctuations_zh.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/char/traditional_to_simple.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/date/d.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/date/dd.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/date/m.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/date/mm.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/default/blacklist.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/default/whitelist.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/erhua/whitelist.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/math/operator.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/math/operator_unambiguous.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/measure/units_en.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/measure/units_zh.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/money/code.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/money/symbol.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/number/digit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/number/dot.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/number/sign.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/number/teen.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/number/zero.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/sport/club.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/sport/country.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/time/hour.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/time/minute.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/time/noon.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/data/time/second.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/rules/char.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/cardinal_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/char_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/cardinal.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/char.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/date.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/fraction.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/math.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/measure.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/money.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/number.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/postprocessor.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/preprocessor.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/sport.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/time.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/data/whitelist.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/date_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/fraction_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/math_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/measure_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/money_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/sport_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/time_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/utils.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/chinese/test/whitelist_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/address/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/address/address_word.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/address/state.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/date/day.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/date/month_abbr.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/date/month_name.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/date/month_number.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/date/week.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/date/year_suffix.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/electronic/domain.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/electronic/symbol.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/electronic/words.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/measure/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/measure/math_operation.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/measure/unit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/money/currency_major.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/money/currency_minor_plural.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/money/currency_minor_singular.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/money/per_unit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/number/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/number/cardinal_number_name.far +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/number/cardinal_number_name_au.far +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/number/digit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/number/hundred.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/number/quantity_abbr.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/number/teen.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/number/thousand.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/number/ty.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/number/zero.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/ordinal/digit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/ordinal/teen.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/roman/README.md +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/roman/female.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/roman/key_word.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/roman/male.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/roman/roman_to_spoken.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/suppletive.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/telephone/ip_prompt.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/telephone/ssn_prompt.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/telephone/telephone_prompt.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/time/__init__.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/time/suffix.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/time/zone.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/whitelist/alternatives.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/whitelist/alternatives_all_format.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/whitelist/asr.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/whitelist/asr_with_pc.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/whitelist/ipa_symbols.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/whitelist/lj_speech.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/data/whitelist/tts.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/cardinal_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/cardinal.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/decimal.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/electronic.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/fraction.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/measure.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/money.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/ordinal.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/range.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/roman.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/telephone.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/time.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/whitelist.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/data/word.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/date_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/decimal_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/fraction_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/measure_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/money_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/ordinal_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/range_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/roman_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/telephone_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/time_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/utils.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/whitelist_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/english/test/word_test.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/char/common_chinese_char.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/char/fullwidth_to_halfwidth.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/char/hiragana_and_katakana.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/char/oov_tags.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/char/punctuations_ja.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/date/d.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/date/date.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/date/dd.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/date/m.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/date/mm.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/default/whitelist.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/math/operator.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/measure/units_en.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/measure/units_ja.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/money/code.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/money/symbol.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/number/digit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/number/dot.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/number/en_digit.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/number/sign.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/number/teen.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/number/zero.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/pyopenjtalk/transliteration.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/sport/club.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/sport/country.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/time/hour.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/time/minute.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/time/noon.tsv +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/data/time/second.tsv +0 -0
- {wetextprocessing-1.2.0/itn → wetextprocessing-1.3.0/tn}/japanese/rules/char.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/test/data/cardinal.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/test/data/char.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/test/data/date.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/test/data/fraction.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/test/data/math.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/test/data/measure.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/test/data/money.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/test/data/sport.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/test/data/time.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/test/data/whitelist.txt +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/japanese/test/utils.py +0 -0
- {wetextprocessing-1.2.0 → wetextprocessing-1.3.0}/tn/utils.py +0 -0
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
name: Android Build
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
workflow_dispatch:
|
|
5
|
+
pull_request:
|
|
6
|
+
paths:
|
|
7
|
+
# Android project itself, plus the shared C++ runtime it compiles from source.
|
|
8
|
+
- "runtime/android/**"
|
|
9
|
+
- "runtime/bindings/android/**"
|
|
10
|
+
- "runtime/CMakeLists.txt"
|
|
11
|
+
- "runtime/CMakePresets.json"
|
|
12
|
+
- "runtime/processor/**"
|
|
13
|
+
- "runtime/utils/**"
|
|
14
|
+
- "runtime/cmake/**"
|
|
15
|
+
- ".github/workflows/android.yml"
|
|
16
|
+
|
|
17
|
+
jobs:
|
|
18
|
+
build-android:
|
|
19
|
+
name: build (android)
|
|
20
|
+
runs-on: ubuntu-latest
|
|
21
|
+
steps:
|
|
22
|
+
- uses: actions/checkout@v4
|
|
23
|
+
|
|
24
|
+
- uses: actions/setup-java@v4
|
|
25
|
+
with:
|
|
26
|
+
distribution: temurin
|
|
27
|
+
java-version: '21'
|
|
28
|
+
|
|
29
|
+
# Caches downloaded Gradle distribution and dependencies across runs.
|
|
30
|
+
- name: Setup Gradle
|
|
31
|
+
uses: gradle/actions/setup-gradle@v4
|
|
32
|
+
with:
|
|
33
|
+
build-root-directory: runtime/android
|
|
34
|
+
|
|
35
|
+
# Puts sdkmanager on PATH, accepts licenses, installs platform + build-tools.
|
|
36
|
+
- name: Setup Android SDK
|
|
37
|
+
uses: android-actions/setup-android@v3
|
|
38
|
+
with:
|
|
39
|
+
packages: 'platforms;android-36 build-tools;36.0.0'
|
|
40
|
+
|
|
41
|
+
# sdkmanager has no bare "ndk" package — must install "ndk;X.Y.Z". Pick the
|
|
42
|
+
# newest side-by-side NDK from the package list.
|
|
43
|
+
- name: Install NDK (latest)
|
|
44
|
+
run: |
|
|
45
|
+
NDK_PKG=$(sdkmanager --list 2>/dev/null \
|
|
46
|
+
| grep -oE 'ndk;[0-9]+\.[0-9]+\.[0-9]+' \
|
|
47
|
+
| sort -t';' -k2 -V \
|
|
48
|
+
| tail -1)
|
|
49
|
+
if [ -z "$NDK_PKG" ]; then
|
|
50
|
+
echo "ERROR: no ndk;* package found in sdkmanager --list" >&2
|
|
51
|
+
sdkmanager --list 2>/dev/null | grep -i ndk | head -20 >&2 || true
|
|
52
|
+
exit 1
|
|
53
|
+
fi
|
|
54
|
+
echo "Installing $NDK_PKG"
|
|
55
|
+
sdkmanager --install "$NDK_PKG"
|
|
56
|
+
NDK_VER="${NDK_PKG#ndk;}"
|
|
57
|
+
echo "Using NDK $NDK_VER"
|
|
58
|
+
echo "ANDROID_NDK_HOME=$ANDROID_HOME/ndk/$NDK_VER" >> "$GITHUB_ENV"
|
|
59
|
+
|
|
60
|
+
# Build the native lib with CMake presets and install it into jniLibs/<abi>/.
|
|
61
|
+
# Gradle does NOT drive CMake; it only packages the prebuilt .so.
|
|
62
|
+
- name: Build native lib
|
|
63
|
+
working-directory: runtime
|
|
64
|
+
run: |
|
|
65
|
+
cmake --preset android-arm64-v8a
|
|
66
|
+
cmake --build --preset android-arm64-v8a
|
|
67
|
+
cmake --install build/aarch64-linux-android --component jni
|
|
68
|
+
|
|
69
|
+
- uses: actions/setup-python@v5
|
|
70
|
+
with:
|
|
71
|
+
python-version: '3.x'
|
|
72
|
+
|
|
73
|
+
# Generate the four FST models straight into assets/ so the APK ships with
|
|
74
|
+
# them (the app loads zh_tn_* and zh_itn_* at runtime).
|
|
75
|
+
- name: Generate model assets
|
|
76
|
+
run: |
|
|
77
|
+
pip install pynini importlib_resources
|
|
78
|
+
assets=runtime/android/app/src/main/assets
|
|
79
|
+
python -m tn --language zh --overwrite_cache --cache_dir "$assets"
|
|
80
|
+
python -m itn --language zh --overwrite_cache --cache_dir "$assets"
|
|
81
|
+
|
|
82
|
+
- name: Build release APK
|
|
83
|
+
working-directory: runtime/android
|
|
84
|
+
run: ./gradlew :app:assembleRelease -PabiFilters=arm64-v8a --no-daemon
|
|
85
|
+
|
|
86
|
+
- name: Upload release APK
|
|
87
|
+
uses: actions/upload-artifact@v4
|
|
88
|
+
with:
|
|
89
|
+
name: app-release-arm64-v8a
|
|
90
|
+
path: runtime/android/app/build/outputs/apk/release/app-release.apk
|
|
91
|
+
if-no-files-found: error
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
name: Runtime Build
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
workflow_dispatch:
|
|
5
|
+
pull_request:
|
|
6
|
+
paths:
|
|
7
|
+
- "runtime/**"
|
|
8
|
+
# Android has its own workflow; don't rebuild the desktop runtime for
|
|
9
|
+
# Android-only changes (shared processor/utils/cmake still match above).
|
|
10
|
+
- "!runtime/android/**"
|
|
11
|
+
- "!runtime/bindings/android/**"
|
|
12
|
+
- ".github/workflows/runtime.yml"
|
|
13
|
+
|
|
14
|
+
jobs:
|
|
15
|
+
build:
|
|
16
|
+
runs-on: ${{ matrix.os }}
|
|
17
|
+
strategy:
|
|
18
|
+
fail-fast: false
|
|
19
|
+
matrix:
|
|
20
|
+
os: [ubuntu-latest, macos-latest]
|
|
21
|
+
steps:
|
|
22
|
+
- uses: actions/checkout@v4
|
|
23
|
+
- name: Build runtime
|
|
24
|
+
run: |
|
|
25
|
+
cmake -B build -G Ninja -S runtime -DBUILD_TESTING=ON
|
|
26
|
+
cmake --build build --parallel
|
|
27
|
+
- uses: actions/setup-python@v5
|
|
28
|
+
if: matrix.os == 'ubuntu-latest'
|
|
29
|
+
with:
|
|
30
|
+
python-version: '3.x'
|
|
31
|
+
- name: Install dependencies & Generate TN models
|
|
32
|
+
if: matrix.os == 'ubuntu-latest'
|
|
33
|
+
run: |
|
|
34
|
+
pip install pynini importlib_resources
|
|
35
|
+
python -m tn --language zh --overwrite_cache
|
|
36
|
+
- name: Run tests
|
|
37
|
+
if: matrix.os == 'ubuntu-latest'
|
|
38
|
+
run: ctest --test-dir build/test --output-on-failure
|
|
39
|
+
|
|
40
|
+
build-windows:
|
|
41
|
+
name: build (windows-latest)
|
|
42
|
+
runs-on: windows-latest
|
|
43
|
+
defaults:
|
|
44
|
+
run:
|
|
45
|
+
shell: msys2 {0}
|
|
46
|
+
steps:
|
|
47
|
+
- uses: actions/checkout@v4
|
|
48
|
+
|
|
49
|
+
- uses: msys2/setup-msys2@v2
|
|
50
|
+
with:
|
|
51
|
+
msystem: MINGW64
|
|
52
|
+
update: false
|
|
53
|
+
cache: true
|
|
54
|
+
install: >-
|
|
55
|
+
mingw-w64-x86_64-cmake
|
|
56
|
+
mingw-w64-x86_64-gcc
|
|
57
|
+
mingw-w64-x86_64-ninja
|
|
58
|
+
|
|
59
|
+
- name: Build runtime
|
|
60
|
+
run: |
|
|
61
|
+
cmake --version
|
|
62
|
+
gcc --version
|
|
63
|
+
|
|
64
|
+
# Tests are skipped on Windows: processor_test needs zh_tn_*.fst, which
|
|
65
|
+
# are generated via pynini, and pynini has no Windows build.
|
|
66
|
+
cmake -S runtime -B build -G Ninja \
|
|
67
|
+
-DCMAKE_BUILD_TYPE=Release \
|
|
68
|
+
-DBUILD_TESTING=OFF
|
|
69
|
+
cmake --build build --parallel
|
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
name: UnitTest
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
workflow_dispatch:
|
|
5
|
+
pull_request:
|
|
6
|
+
paths:
|
|
7
|
+
- '**.py'
|
|
8
|
+
- '**.tsv'
|
|
9
|
+
- '**.toml'
|
|
10
|
+
- '**.yml'
|
|
11
|
+
- '**.yaml'
|
|
12
|
+
- '.flake8'
|
|
13
|
+
- '.style.yapf'
|
|
14
|
+
|
|
15
|
+
jobs:
|
|
16
|
+
unit-test:
|
|
17
|
+
runs-on: ${{ matrix.os }}
|
|
18
|
+
strategy:
|
|
19
|
+
fail-fast: false
|
|
20
|
+
matrix:
|
|
21
|
+
os: ${{ github.event_name == 'pull_request' && fromJSON('["ubuntu-latest"]') || fromJSON('["ubuntu-latest", "macos-latest", "windows-latest"]') }}
|
|
22
|
+
python-version: ['3.9']
|
|
23
|
+
steps:
|
|
24
|
+
- uses: actions/checkout@v4
|
|
25
|
+
with:
|
|
26
|
+
fetch-depth: 0
|
|
27
|
+
- name: Set up Python ${{ matrix.python-version }} (non-Windows)
|
|
28
|
+
if: runner.os != 'Windows'
|
|
29
|
+
uses: actions/setup-python@v4
|
|
30
|
+
with:
|
|
31
|
+
python-version: ${{ matrix.python-version }}
|
|
32
|
+
- name: Cache Homebrew (macOS)
|
|
33
|
+
if: runner.os == 'macOS'
|
|
34
|
+
uses: actions/cache@v4
|
|
35
|
+
with:
|
|
36
|
+
path: |
|
|
37
|
+
/opt/homebrew/Cellar/openfst
|
|
38
|
+
/opt/homebrew/include/fst
|
|
39
|
+
/opt/homebrew/lib/libfst*
|
|
40
|
+
key: brew-openfst-${{ runner.os }}
|
|
41
|
+
- name: Install openfst (macOS)
|
|
42
|
+
if: runner.os == 'macOS'
|
|
43
|
+
run: |
|
|
44
|
+
brew install openfst || true
|
|
45
|
+
echo "CPLUS_INCLUDE_PATH=/opt/homebrew/include:$CPLUS_INCLUDE_PATH" >> $GITHUB_ENV
|
|
46
|
+
echo "LIBRARY_PATH=/opt/homebrew/lib:$LIBRARY_PATH" >> $GITHUB_ENV
|
|
47
|
+
- name: Cache pip (non-Windows)
|
|
48
|
+
if: runner.os != 'Windows'
|
|
49
|
+
uses: actions/cache@v4
|
|
50
|
+
with:
|
|
51
|
+
path: ~/.cache/pip
|
|
52
|
+
key: pip-${{ runner.os }}-${{ hashFiles('pyproject.toml') }}
|
|
53
|
+
- name: Install dependencies (non-Windows)
|
|
54
|
+
if: runner.os != 'Windows'
|
|
55
|
+
run: |
|
|
56
|
+
python -m pip install --upgrade pip
|
|
57
|
+
pip install .[test]
|
|
58
|
+
- name: Set up Conda (Windows)
|
|
59
|
+
if: runner.os == 'Windows'
|
|
60
|
+
uses: conda-incubator/setup-miniconda@v3
|
|
61
|
+
with:
|
|
62
|
+
python-version: ${{ matrix.python-version }}
|
|
63
|
+
activate-environment: test
|
|
64
|
+
use-only-tar-bz2: true
|
|
65
|
+
- name: Cache Conda (Windows)
|
|
66
|
+
if: runner.os == 'Windows'
|
|
67
|
+
uses: actions/cache@v4
|
|
68
|
+
with:
|
|
69
|
+
path: C:\Miniconda\envs\test
|
|
70
|
+
key: conda-${{ runner.os }}-${{ hashFiles('pyproject.toml') }}
|
|
71
|
+
- name: Install dependencies (Windows)
|
|
72
|
+
if: runner.os == 'Windows'
|
|
73
|
+
shell: bash -el {0}
|
|
74
|
+
run: |
|
|
75
|
+
conda install -c conda-forge pynini
|
|
76
|
+
python -m pip install --upgrade pip
|
|
77
|
+
pip install .[test] --no-deps
|
|
78
|
+
pip install flake8 pytest importlib_resources
|
|
79
|
+
- name: Lint with flake8
|
|
80
|
+
shell: bash -el {0}
|
|
81
|
+
run: |
|
|
82
|
+
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
|
|
83
|
+
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
|
|
84
|
+
- name: Test with pytest
|
|
85
|
+
shell: bash -el {0}
|
|
86
|
+
run: pytest
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
name: Build Wheels
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
workflow_dispatch:
|
|
5
|
+
inputs:
|
|
6
|
+
version:
|
|
7
|
+
description: 'Release version (optional, will be determined by setuptools_scm if not provided)'
|
|
8
|
+
required: false
|
|
9
|
+
|
|
10
|
+
jobs:
|
|
11
|
+
build:
|
|
12
|
+
runs-on: ubuntu-latest
|
|
13
|
+
strategy:
|
|
14
|
+
matrix:
|
|
15
|
+
python-version: [3.9]
|
|
16
|
+
steps:
|
|
17
|
+
- uses: actions/checkout@v4
|
|
18
|
+
with:
|
|
19
|
+
fetch-depth: 0
|
|
20
|
+
- name: Set up Python ${{ matrix.python-version }}
|
|
21
|
+
uses: actions/setup-python@v5
|
|
22
|
+
with:
|
|
23
|
+
python-version: ${{ matrix.python-version }}
|
|
24
|
+
- name: Install dependencies
|
|
25
|
+
run: |
|
|
26
|
+
python -m pip install --upgrade pip
|
|
27
|
+
pip install .[test]
|
|
28
|
+
pip install build twine
|
|
29
|
+
|
|
30
|
+
- name: Build Graph
|
|
31
|
+
run: |
|
|
32
|
+
python -m tn --text "2.5平方电线" --overwrite_cache --language "zh"
|
|
33
|
+
python -m tn --text "2010-03-21" --overwrite_cache --language "en"
|
|
34
|
+
python -m itn --text "二点五平方电线" --overwrite_cache
|
|
35
|
+
python -m itn --text "one two three" --overwrite_cache --language "en"
|
|
36
|
+
|
|
37
|
+
- name: Prepare Graph
|
|
38
|
+
run: |
|
|
39
|
+
mkdir graph
|
|
40
|
+
cp tn/*.fst graph
|
|
41
|
+
cp itn/*.fst graph
|
|
42
|
+
cp tn/*_cache.json graph
|
|
43
|
+
cp itn/*_cache.json graph
|
|
44
|
+
|
|
45
|
+
- name: Get version from setuptools_scm
|
|
46
|
+
id: scm_version
|
|
47
|
+
run: |
|
|
48
|
+
# Check if version is provided as input
|
|
49
|
+
if [ -n "${{ github.event.inputs.version }}" ]; then
|
|
50
|
+
echo "VERSION=${{ github.event.inputs.version }}" >> $GITHUB_OUTPUT
|
|
51
|
+
else
|
|
52
|
+
# Get version from setuptools_scm
|
|
53
|
+
pip install setuptools_scm
|
|
54
|
+
VERSION=$(python -m setuptools_scm | tail -1)
|
|
55
|
+
echo "VERSION=$VERSION" >> $GITHUB_OUTPUT
|
|
56
|
+
fi
|
|
57
|
+
|
|
58
|
+
- name: Upload Graph
|
|
59
|
+
uses: actions/upload-artifact@v4
|
|
60
|
+
with:
|
|
61
|
+
name: release-graph-v${{ steps.scm_version.outputs.VERSION }}
|
|
62
|
+
path: graph
|
|
63
|
+
|
|
64
|
+
- name: Build package
|
|
65
|
+
run: |
|
|
66
|
+
python -m build
|
|
67
|
+
|
|
68
|
+
- name: Publish on pypi.org
|
|
69
|
+
if: github.ref == 'refs/heads/master' && github.repository == 'wenet-e2e/WeTextProcessing'
|
|
70
|
+
env:
|
|
71
|
+
TWINE_USERNAME: __token__
|
|
72
|
+
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
|
|
73
|
+
run: |
|
|
74
|
+
python -m twine upload dist/*
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# Byte-compiled / optimized / DLL files
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*$py.class
|
|
5
|
+
|
|
6
|
+
# Visual Studio Code files
|
|
7
|
+
.vscode
|
|
8
|
+
.vs
|
|
9
|
+
|
|
10
|
+
# PyCharm files
|
|
11
|
+
.idea
|
|
12
|
+
|
|
13
|
+
# Eclipse Project settings
|
|
14
|
+
*.*project
|
|
15
|
+
.settings
|
|
16
|
+
|
|
17
|
+
# Sublime Text settings
|
|
18
|
+
*.sublime-workspace
|
|
19
|
+
*.sublime-project
|
|
20
|
+
|
|
21
|
+
# Editor temporaries
|
|
22
|
+
*.swn
|
|
23
|
+
*.swo
|
|
24
|
+
*.swp
|
|
25
|
+
*.swm
|
|
26
|
+
*~
|
|
27
|
+
|
|
28
|
+
# IPython notebook checkpoints
|
|
29
|
+
.ipynb_checkpoints
|
|
30
|
+
|
|
31
|
+
# macOS dir files
|
|
32
|
+
.DS_Store
|
|
33
|
+
|
|
34
|
+
# Clangd files
|
|
35
|
+
.cache
|
|
36
|
+
.venv/
|
|
37
|
+
compile_commands.json
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
# Setup files
|
|
41
|
+
WeTextProcessing.egg-info/
|
|
42
|
+
build/
|
|
43
|
+
dist/
|
|
44
|
+
tn/*.far
|
|
45
|
+
itn/*.far
|
|
46
|
+
|
|
47
|
+
# Version file(setuptools_scm generated)
|
|
48
|
+
tn/_version.py
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
exclude: '.*\.(txt|tsv)$'
|
|
2
|
+
repos:
|
|
3
|
+
- repo: https://github.com/pre-commit/pre-commit-hooks
|
|
4
|
+
rev: v4.5.0
|
|
5
|
+
hooks:
|
|
6
|
+
- id: trailing-whitespace
|
|
7
|
+
- repo: https://github.com/google/yapf
|
|
8
|
+
rev: 'v0.43.0'
|
|
9
|
+
hooks:
|
|
10
|
+
- id: yapf
|
|
11
|
+
- repo: https://github.com/pycqa/flake8
|
|
12
|
+
rev: '7.3.0'
|
|
13
|
+
hooks:
|
|
14
|
+
- id: flake8
|
|
15
|
+
- repo: https://github.com/pre-commit/mirrors-clang-format
|
|
16
|
+
rev: 'v17.0.6'
|
|
17
|
+
hooks:
|
|
18
|
+
- id: clang-format
|
|
19
|
+
exclude: '.*\.(json|java|js|m|mm|proto)'
|
|
20
|
+
- repo: https://github.com/cpplint/cpplint
|
|
21
|
+
rev: '1.6.1'
|
|
22
|
+
hooks:
|
|
23
|
+
- id: cpplint
|
|
@@ -0,0 +1,290 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: WeTextProcessing
|
|
3
|
+
Version: 1.3.0
|
|
4
|
+
Summary: WeTextProcessing, including TN & ITN
|
|
5
|
+
Author-email: Zhendong Peng <pzd17@tsinghua.org.cn>, Xingchen Song <sxc19@tsinghua.org.cn>
|
|
6
|
+
Project-URL: Homepage, https://github.com/wenet-e2e/WeTextProcessing
|
|
7
|
+
Classifier: Programming Language :: Python :: 3
|
|
8
|
+
Classifier: Operating System :: OS Independent
|
|
9
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
10
|
+
Requires-Python: >=3.7
|
|
11
|
+
Description-Content-Type: text/markdown
|
|
12
|
+
License-File: LICENSE
|
|
13
|
+
Requires-Dist: pynini>=2.1.6
|
|
14
|
+
Requires-Dist: importlib_resources
|
|
15
|
+
Provides-Extra: test
|
|
16
|
+
Requires-Dist: pytest; extra == "test"
|
|
17
|
+
Requires-Dist: flake8; extra == "test"
|
|
18
|
+
Provides-Extra: dev
|
|
19
|
+
Requires-Dist: flake8; extra == "dev"
|
|
20
|
+
Requires-Dist: pre-commit==3.5.0; extra == "dev"
|
|
21
|
+
Requires-Dist: pytest; extra == "dev"
|
|
22
|
+
Dynamic: license-file
|
|
23
|
+
|
|
24
|
+
<div align="center">
|
|
25
|
+
|
|
26
|
+
# WeTextProcessing
|
|
27
|
+
|
|
28
|
+
**Rule-based text normalization and inverse text normalization for speech applications.**
|
|
29
|
+
|
|
30
|
+
[Quick start](#quick-start) · [Languages](#supported-languages) · [Python API](#python-api) · [Rule architecture](docs/python-rule-architecture.md) · [C++ runtime](runtime/README.md)
|
|
31
|
+
|
|
32
|
+
[](https://github.com/wenet-e2e/WeTextProcessing/actions/workflows/unittest.yml)
|
|
33
|
+
|
|
34
|
+
</div>
|
|
35
|
+
|
|
36
|
+
WeTextProcessing converts between written and spoken text using weighted finite-state transducers (WFSTs), built with [Pynini](https://www.openfst.org/twiki/bin/view/GRM/Pynini) and [OpenFst](https://www.openfst.org/twiki/bin/view/FST/WebHome).
|
|
37
|
+
|
|
38
|
+
- **TN and ITN** — normalize numbers, dates, times, currencies, units, and other structured expressions.
|
|
39
|
+
- **Multilingual rules** — language-specific grammars for every language in [Supported languages](#supported-languages).
|
|
40
|
+
- **Explainable output** — inspect raw tagged fields, trace input/output spans, and request unique n-best results.
|
|
41
|
+
- **Reusable graphs** — automatic, configuration-aware FST caching and a separate C++ deployment runtime.
|
|
42
|
+
|
|
43
|
+
For background and motivation, read [WeTextProcessing: Production First & Production Ready Text Processing Toolkit](https://mp.weixin.qq.com/s/q_11lck78qcjylHCi6wVsQ) (Chinese).
|
|
44
|
+
|
|
45
|
+
## What are TN and ITN?
|
|
46
|
+
|
|
47
|
+
### Text normalization
|
|
48
|
+
|
|
49
|
+
TN converts written expressions into their spoken forms, for example `12` → `十二`. It is commonly used before speech synthesis.
|
|
50
|
+
|
|
51
|
+
<div align=center><img src="https://user-images.githubusercontent.com/13466943/193439861-acfba531-13d1-4fca-b2f2-6e47fc10f195.png" alt="Cover" width="50%"/></div>
|
|
52
|
+
|
|
53
|
+
### Inverse text normalization
|
|
54
|
+
|
|
55
|
+
ITN converts spoken expressions into written forms, for example `十二` → `12`. It is commonly used after speech recognition.
|
|
56
|
+
|
|
57
|
+
<div align=center><img src="https://user-images.githubusercontent.com/13466943/193439870-634c44a3-bd62-4311-bcf2-1427758d5f62.png" alt="Cover" width="50%"/></div>
|
|
58
|
+
|
|
59
|
+
## Supported languages
|
|
60
|
+
|
|
61
|
+
| Language | CLI code | TN | ITN | Python package |
|
|
62
|
+
| --- | --- | --- | --- | --- |
|
|
63
|
+
| Arabic | `ar` | Yes | Yes | `arabic` |
|
|
64
|
+
| Armenian | `hy` | Yes | Yes | `armenian` |
|
|
65
|
+
| Chinese | `zh` | Yes | Yes | `chinese` |
|
|
66
|
+
| English | `en` | Yes | Yes | `english` |
|
|
67
|
+
| French | `fr` | Yes | Yes | `french` |
|
|
68
|
+
| German | `de` | Yes | Yes | `german` |
|
|
69
|
+
| Hebrew | `he` | Yes | Yes | `hebrew` |
|
|
70
|
+
| Hindi | `hi` | Yes | Yes | `hindi` |
|
|
71
|
+
| Hindi–English (code-mixed) | `hi_en` | No | Yes | `hindi_english` |
|
|
72
|
+
| Hungarian | `hu` | Yes | Yes | `hungarian` |
|
|
73
|
+
| Italian | `it` | Yes | Yes | `italian` |
|
|
74
|
+
| Japanese | `ja` | Yes | Yes | `japanese` |
|
|
75
|
+
| Kinyarwanda | `rw` | Yes | Yes | `kinyarwanda` |
|
|
76
|
+
| Korean | `ko` | Yes | Yes | `korean` |
|
|
77
|
+
| Marathi | `mr` | Yes | Yes | `marathi` |
|
|
78
|
+
| Portuguese | `pt` | Yes | Yes | `portuguese` |
|
|
79
|
+
| Russian | `ru` | Yes | Yes | `russian` |
|
|
80
|
+
| Spanish | `es` | Yes | Yes | `spanish` |
|
|
81
|
+
| Spanish–English (code-mixed) | `es_en` | No | Yes | `spanish_english` |
|
|
82
|
+
| Swedish | `sv` | Yes | Yes | `swedish` |
|
|
83
|
+
| Vietnamese | `vi` | Yes | Yes | `vietnamese` |
|
|
84
|
+
|
|
85
|
+
Chinese is the CLI default. Rule coverage and configurable options vary by language and direction.
|
|
86
|
+
|
|
87
|
+
The published package includes Chinese, English, and Japanese. The other languages are included in this source checkout; install from source to use them before the next package release.
|
|
88
|
+
|
|
89
|
+
## Quick start
|
|
90
|
+
|
|
91
|
+
### Install
|
|
92
|
+
|
|
93
|
+
Install the published package:
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
pip install WeTextProcessing
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
For the latest rules, including every language beyond Chinese, English, and Japanese, install from source:
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
git clone https://github.com/wenet-e2e/WeTextProcessing.git
|
|
103
|
+
cd WeTextProcessing
|
|
104
|
+
pip install -e .
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Pynini/OpenFst must be available in your Python environment. If a compatible wheel is unavailable, follow the [Pynini installation instructions](https://www.openfst.org/twiki/bin/view/GRM/Pynini). On Windows, the project's CI uses `conda install -c conda-forge pynini` before installing WeTextProcessing.
|
|
108
|
+
|
|
109
|
+
### Run the CLI
|
|
110
|
+
|
|
111
|
+
```bash
|
|
112
|
+
wetn --text "2.5平方电线"
|
|
113
|
+
weitn --text "二点五平方电线"
|
|
114
|
+
|
|
115
|
+
wetn --language en --text "13 dollars"
|
|
116
|
+
weitn --language en --text "thirteen dollars"
|
|
117
|
+
|
|
118
|
+
wetn --language de --text "23"
|
|
119
|
+
weitn --language de --text "dreiundzwanzig"
|
|
120
|
+
|
|
121
|
+
wetn --language ja --text "12時"
|
|
122
|
+
weitn --language ja --text "十二時"
|
|
123
|
+
|
|
124
|
+
wetn --language ko --text "오후 3시 30분"
|
|
125
|
+
weitn --language ko --text "오천원"
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
Each input produces two lines: the **tagged representation**, then the **verbalized result**. The tagger preserves raw field values; the verbalizer performs the conversion.
|
|
129
|
+
|
|
130
|
+
The same interfaces are available as `python -m tn` and `python -m itn`. Both accept `--text`, `--file PATH`, or UTF-8 text from standard input when neither option is supplied:
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
python -m tn --language en --file input.txt
|
|
134
|
+
python -m itn --language ko < input.txt
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Only input line endings are removed before processing. Other spaces are passed to the selected language's rules.
|
|
138
|
+
|
|
139
|
+
### Language-specific options
|
|
140
|
+
|
|
141
|
+
Inspect the options for the selected language and direction:
|
|
142
|
+
|
|
143
|
+
```bash
|
|
144
|
+
wetn --language zh --help
|
|
145
|
+
weitn --language ko --help
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
For example:
|
|
149
|
+
|
|
150
|
+
```bash
|
|
151
|
+
wetn --language zh --no-remove-erhua --text "这儿"
|
|
152
|
+
weitn --language zh --enable-0-to-9 --text "一二三"
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
Boolean options accept `--option` / `--no-option` and the legacy `--option True|False` form. Options from another language or direction are not accepted.
|
|
156
|
+
|
|
157
|
+
## Python API
|
|
158
|
+
|
|
159
|
+
Import the normalizer for the language you need and reuse the instance:
|
|
160
|
+
|
|
161
|
+
```python
|
|
162
|
+
from tn.chinese.normalizer import Normalizer
|
|
163
|
+
from itn.chinese.inverse_normalizer import InverseNormalizer
|
|
164
|
+
|
|
165
|
+
normalizer = Normalizer()
|
|
166
|
+
inverse_normalizer = InverseNormalizer()
|
|
167
|
+
|
|
168
|
+
print(normalizer.normalize("今天中午12点"))
|
|
169
|
+
print(inverse_normalizer.normalize("二点五平方电线"))
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
Other languages use the same class names under their packages. For example, Korean:
|
|
173
|
+
|
|
174
|
+
```python
|
|
175
|
+
from tn.korean.normalizer import Normalizer
|
|
176
|
+
from itn.korean.inverse_normalizer import InverseNormalizer
|
|
177
|
+
|
|
178
|
+
normalizer = Normalizer()
|
|
179
|
+
inverse_normalizer = InverseNormalizer()
|
|
180
|
+
|
|
181
|
+
print(normalizer.normalize("오후 3시 30분"))
|
|
182
|
+
print(inverse_normalizer.normalize("오천원"))
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
### Inspect tags
|
|
186
|
+
|
|
187
|
+
Tagging and verbalization are independently accessible:
|
|
188
|
+
|
|
189
|
+
```python
|
|
190
|
+
tagged = normalizer.tag("오후 3시 30분")
|
|
191
|
+
print(tagged)
|
|
192
|
+
print(normalizer.verbalize(tagged))
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
The hours and minutes remain written values in the tags, rather than being replaced with their Korean readings.
|
|
196
|
+
|
|
197
|
+
### Input/output span mapping
|
|
198
|
+
|
|
199
|
+
`normalize_with_mapping()` returns normalized text and the spans matched by grammar rules:
|
|
200
|
+
|
|
201
|
+
```python
|
|
202
|
+
from tn.chinese.normalizer import Normalizer
|
|
203
|
+
|
|
204
|
+
normalizer = Normalizer()
|
|
205
|
+
result = normalizer.normalize_with_mapping("今天中午12点")
|
|
206
|
+
|
|
207
|
+
print(result.output_text)
|
|
208
|
+
for mapping in result.mappings:
|
|
209
|
+
print(mapping.token_type, mapping.input_text, "=>", mapping.output_text)
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
Output:
|
|
213
|
+
|
|
214
|
+
```text
|
|
215
|
+
今天中午十二点
|
|
216
|
+
math 12 => 十二
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
Offsets are half-open Unicode character offsets, not UTF-8 byte offsets. Use `include_identity=True` to include unchanged tokens, or `result.as_dict()` to obtain a serializable representation. Mappings are traced through the WFSTs, not inferred from a text diff.
|
|
220
|
+
|
|
221
|
+
### N-best results
|
|
222
|
+
|
|
223
|
+
```python
|
|
224
|
+
outputs = normalizer.normalize("3/4", nbest=3)
|
|
225
|
+
results = normalizer.normalize_with_mapping("3/4", nbest=3)
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
`nbest=1` returns a string or a single `NormalizationResult`; `nbest>1` returns a list of up to that many unique outputs. The mapping API follows the same output ranking.
|
|
229
|
+
|
|
230
|
+
## FST caching
|
|
231
|
+
|
|
232
|
+
Graphs are cached automatically outside the source tree. Cache bundles record the pipeline's configuration and grammar-source fingerprint; changing either causes a new bundle to be built.
|
|
233
|
+
|
|
234
|
+
| Setting | Behavior |
|
|
235
|
+
| --- | --- |
|
|
236
|
+
| `cache_dir=None` | Use the platform's default user cache |
|
|
237
|
+
| `cache_dir="/path/to/cache"` | Choose an explicit cache root |
|
|
238
|
+
| `cache_dir=False` | Build in memory without persistent caching |
|
|
239
|
+
| `overwrite_cache=True` | Force rebuilding the selected bundle |
|
|
240
|
+
|
|
241
|
+
```python
|
|
242
|
+
from tn.chinese.normalizer import Normalizer
|
|
243
|
+
|
|
244
|
+
normalizer = Normalizer(cache_dir="/path/to/wetext-cache")
|
|
245
|
+
uncached = Normalizer(cache_dir=False)
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
The CLI exposes `--cache-dir PATH` and `--overwrite-cache`. Normal use does not require forced rebuilds.
|
|
249
|
+
|
|
250
|
+
## Development and deployment
|
|
251
|
+
|
|
252
|
+
Install development dependencies and run the tests:
|
|
253
|
+
|
|
254
|
+
```bash
|
|
255
|
+
pip install -e '.[dev]'
|
|
256
|
+
pre-commit install
|
|
257
|
+
pytest -q
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
Before adding or changing a language rule, read the [Python rule architecture guide](docs/python-rule-architecture.md). Rules preserve raw fields in the tagger, apply semantic conversion in the verbalizer, and use pipeline-owned rule inventories and field-order schemas.
|
|
261
|
+
|
|
262
|
+
| Guide | Scope |
|
|
263
|
+
| --- | --- |
|
|
264
|
+
| [TN rules](tn/README.md) | Text normalization rules and examples |
|
|
265
|
+
| [ITN rules](itn/README.md) | Inverse text normalization rules and examples |
|
|
266
|
+
| [Python rule architecture](docs/python-rule-architecture.md) | Tagging, verbalization, registration, and span mapping contracts |
|
|
267
|
+
| [C++ runtime](runtime/README.md) | Native build and deployment instructions |
|
|
268
|
+
|
|
269
|
+
Python cache bundles are **not** C++ runtime exports. Follow the runtime's model and build instructions for native deployment.
|
|
270
|
+
|
|
271
|
+
## Community
|
|
272
|
+
|
|
273
|
+
Use [GitHub Issues](https://github.com/wenet-e2e/WeTextProcessing/issues) for questions, bug reports, and feature requests.
|
|
274
|
+
|
|
275
|
+
For Chinese users, follow the WeNet official account using the QR code on the left. Scan the personal QR code on the right to contact the person who can invite you to the WeChat discussion group.
|
|
276
|
+
|
|
277
|
+
| <img src="https://github.com/robin1001/qr/blob/master/wenet.jpeg" width="250px"> | <img src="https://user-images.githubusercontent.com/13466943/203046432-f637180e-4c87-40cc-be05-ce48c65dd1ef.jpg" width="250px"> |
|
|
278
|
+
| ---- | ---- |
|
|
279
|
+
|
|
280
|
+
## Acknowledgments
|
|
281
|
+
|
|
282
|
+
- [OpenFst](https://www.openfst.org/twiki/bin/view/FST/WebHome) and [Pynini](https://www.openfst.org/twiki/bin/view/GRM/Pynini) for the finite-state toolkit.
|
|
283
|
+
- The [NeMo Text Processing](https://github.com/NVIDIA/NeMo-text-processing) team and community for text-processing grammars and resources, including the [Chinese TN reference](https://github.com/NVIDIA/NeMo-text-processing/tree/main/nemo_text_processing/text_normalization/zh).
|
|
284
|
+
- [Zhenxiang Ma](https://github.com/mzxcpp), [Jiayu Du](https://github.com/dophist), and [SpeechColab](https://github.com/SpeechColab) for their contributions.
|
|
285
|
+
- [Pynini](https://github.com/kylebgorman/pynini) for reference implementations of FAR loading and shortest-path decoding in the C++ runtime.
|
|
286
|
+
- [chinese_text_normalization](https://github.com/speechio/chinese_text_normalization/tree/master/thrax/src/cn) for Chinese ITN grammar resources.
|
|
287
|
+
|
|
288
|
+
## License
|
|
289
|
+
|
|
290
|
+
[Apache License 2.0](LICENSE).
|