WeTextProcessing 1.0.6.dev19__tar.gz → 1.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2275) hide show
  1. wetextprocessing-1.3.0/.github/workflows/android.yml +91 -0
  2. wetextprocessing-1.3.0/.github/workflows/runtime.yml +69 -0
  3. wetextprocessing-1.3.0/.github/workflows/unittest.yml +86 -0
  4. wetextprocessing-1.3.0/.github/workflows/wheels.yml +74 -0
  5. wetextprocessing-1.3.0/.gitignore +48 -0
  6. wetextprocessing-1.3.0/.pre-commit-config.yaml +23 -0
  7. wetextprocessing-1.3.0/.style.yapf +3 -0
  8. wetextprocessing-1.3.0/PKG-INFO +290 -0
  9. wetextprocessing-1.3.0/README.md +267 -0
  10. wetextprocessing-1.3.0/WeTextProcessing.egg-info/PKG-INFO +290 -0
  11. wetextprocessing-1.3.0/WeTextProcessing.egg-info/SOURCES.txt +2120 -0
  12. wetextprocessing-1.3.0/WeTextProcessing.egg-info/scm_file_list.json +2115 -0
  13. wetextprocessing-1.3.0/WeTextProcessing.egg-info/scm_version.json +8 -0
  14. wetextprocessing-1.3.0/conftest.py +19 -0
  15. wetextprocessing-1.3.0/docs/python-rule-architecture.md +106 -0
  16. wetextprocessing-1.3.0/itn/README.md +68 -0
  17. wetextprocessing-1.3.0/itn/arabic/data/measure/measurements.tsv +14 -0
  18. wetextprocessing-1.3.0/itn/arabic/data/money/currency_major.tsv +24 -0
  19. wetextprocessing-1.3.0/itn/arabic/data/money/currency_minor_plural.tsv +4 -0
  20. wetextprocessing-1.3.0/itn/arabic/data/money/currency_minor_singular.tsv +3 -0
  21. wetextprocessing-1.3.0/itn/arabic/data/money/local_currency.tsv +20 -0
  22. wetextprocessing-1.3.0/itn/arabic/data/number/digit.tsv +9 -0
  23. wetextprocessing-1.3.0/itn/arabic/data/number/digit_100.tsv +7 -0
  24. wetextprocessing-1.3.0/itn/arabic/data/number/digit_1000.tsv +7 -0
  25. wetextprocessing-1.3.0/itn/arabic/data/number/flops.tsv +72 -0
  26. wetextprocessing-1.3.0/itn/arabic/data/number/fraction_dual.tsv +9 -0
  27. wetextprocessing-1.3.0/itn/arabic/data/number/fraction_plural.tsv +8 -0
  28. wetextprocessing-1.3.0/itn/arabic/data/number/fraction_singular.tsv +9 -0
  29. wetextprocessing-1.3.0/itn/arabic/data/number/quantities.tsv +6 -0
  30. wetextprocessing-1.3.0/itn/arabic/data/number/teens.tsv +9 -0
  31. wetextprocessing-1.3.0/itn/arabic/data/number/tens.tsv +10 -0
  32. wetextprocessing-1.3.0/itn/arabic/inverse_normalizer.py +60 -0
  33. wetextprocessing-1.3.0/itn/arabic/rules/cardinal.py +66 -0
  34. wetextprocessing-1.3.0/itn/arabic/rules/decimal.py +58 -0
  35. wetextprocessing-1.3.0/itn/arabic/rules/fraction.py +72 -0
  36. wetextprocessing-1.3.0/itn/arabic/rules/measure.py +49 -0
  37. wetextprocessing-1.3.0/itn/arabic/rules/money.py +73 -0
  38. wetextprocessing-1.3.0/itn/arabic/rules/punctuation.py +29 -0
  39. wetextprocessing-1.3.0/itn/arabic/rules/word.py +28 -0
  40. wetextprocessing-1.3.0/itn/arabic/test/data/test_cases_cardinal.txt +34 -0
  41. wetextprocessing-1.3.0/itn/arabic/test/data/test_cases_decimal.txt +30 -0
  42. wetextprocessing-1.3.0/itn/arabic/test/data/test_cases_fraction.txt +20 -0
  43. wetextprocessing-1.3.0/itn/arabic/test/data/test_cases_measure.txt +15 -0
  44. wetextprocessing-1.3.0/itn/arabic/test/data/test_cases_money.txt +21 -0
  45. wetextprocessing-1.3.0/itn/arabic/test/data/test_cases_whitelist.txt +6 -0
  46. wetextprocessing-1.3.0/itn/arabic/test/normalizer_test.py +137 -0
  47. wetextprocessing-1.3.0/itn/armenian/data/currency.tsv +53 -0
  48. wetextprocessing-1.3.0/itn/armenian/data/measurement_dates.tsv +4 -0
  49. wetextprocessing-1.3.0/itn/armenian/data/measurements.tsv +48 -0
  50. wetextprocessing-1.3.0/itn/armenian/data/numbers/digit.tsv +11 -0
  51. wetextprocessing-1.3.0/itn/armenian/data/numbers/digits_no_one.tsv +10 -0
  52. wetextprocessing-1.3.0/itn/armenian/data/numbers/ties.tsv +10 -0
  53. wetextprocessing-1.3.0/itn/armenian/data/ordinals/digit.tsv +10 -0
  54. wetextprocessing-1.3.0/itn/armenian/data/time/hours.tsv +29 -0
  55. wetextprocessing-1.3.0/itn/armenian/data/time/minutes.tsv +65 -0
  56. wetextprocessing-1.3.0/itn/armenian/data/time/minutes_to.tsv +59 -0
  57. wetextprocessing-1.3.0/itn/armenian/data/time/to_hour.tsv +25 -0
  58. wetextprocessing-1.3.0/itn/armenian/data/whitelist.tsv +7 -0
  59. wetextprocessing-1.3.0/itn/armenian/inverse_normalizer.py +66 -0
  60. wetextprocessing-1.3.0/itn/armenian/rules/cardinal.py +67 -0
  61. wetextprocessing-1.3.0/itn/armenian/rules/decimal.py +67 -0
  62. wetextprocessing-1.3.0/itn/armenian/rules/fraction.py +40 -0
  63. wetextprocessing-1.3.0/itn/armenian/rules/measure.py +52 -0
  64. wetextprocessing-1.3.0/itn/armenian/rules/money.py +38 -0
  65. wetextprocessing-1.3.0/itn/armenian/rules/ordinal.py +37 -0
  66. wetextprocessing-1.3.0/itn/armenian/rules/punctuation.py +29 -0
  67. wetextprocessing-1.3.0/itn/armenian/rules/time.py +59 -0
  68. wetextprocessing-1.3.0/itn/armenian/rules/utils.py +58 -0
  69. wetextprocessing-1.3.0/itn/armenian/rules/whitelist.py +27 -0
  70. wetextprocessing-1.3.0/itn/armenian/rules/word.py +28 -0
  71. wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_cardinal.txt +23 -0
  72. wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_decimal.txt +14 -0
  73. wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_fraction.txt +24 -0
  74. wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_measure.txt +12 -0
  75. wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_money.txt +10 -0
  76. wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_ordinal.txt +11 -0
  77. wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_time.txt +9 -0
  78. wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_whitelist.txt +7 -0
  79. wetextprocessing-1.3.0/itn/armenian/test/data/test_cases_word.txt +50 -0
  80. wetextprocessing-1.3.0/itn/armenian/test/normalizer_test.py +158 -0
  81. wetextprocessing-1.3.0/itn/chinese/inverse_normalizer.py +104 -0
  82. wetextprocessing-1.3.0/itn/chinese/rules/cardinal.py +151 -0
  83. wetextprocessing-1.3.0/itn/chinese/rules/date.py +63 -0
  84. wetextprocessing-1.3.0/itn/chinese/rules/fraction.py +57 -0
  85. wetextprocessing-1.3.0/itn/chinese/rules/license_plate.py +38 -0
  86. wetextprocessing-1.3.0/itn/chinese/rules/math.py +37 -0
  87. wetextprocessing-1.3.0/itn/chinese/rules/measure.py +101 -0
  88. wetextprocessing-1.3.0/itn/chinese/rules/money.py +53 -0
  89. wetextprocessing-1.3.0/itn/chinese/rules/postprocessor.py +32 -0
  90. wetextprocessing-1.3.0/itn/chinese/rules/time.py +59 -0
  91. wetextprocessing-1.3.0/itn/chinese/rules/train_number.py +37 -0
  92. wetextprocessing-1.3.0/itn/chinese/rules/whitelist.py +34 -0
  93. wetextprocessing-1.3.0/itn/chinese/test/normalizer_test.py +242 -0
  94. wetextprocessing-1.3.0/itn/english/data/currency.tsv +168 -0
  95. wetextprocessing-1.3.0/itn/english/data/date_period.tsv +10 -0
  96. wetextprocessing-1.3.0/itn/english/data/electronic/domain.tsv +10 -0
  97. wetextprocessing-1.3.0/itn/english/data/electronic/server_name.tsv +17 -0
  98. wetextprocessing-1.3.0/itn/english/data/electronic/symbols.tsv +22 -0
  99. wetextprocessing-1.3.0/itn/english/data/electronic/url_symbols.tsv +5 -0
  100. wetextprocessing-1.3.0/itn/english/data/magnitudes.tsv +4 -0
  101. wetextprocessing-1.3.0/itn/english/data/measurements.tsv +149 -0
  102. wetextprocessing-1.3.0/itn/english/data/months.tsv +12 -0
  103. wetextprocessing-1.3.0/itn/english/data/months_cased.tsv +12 -0
  104. wetextprocessing-1.3.0/itn/english/data/numbers/digit.tsv +9 -0
  105. wetextprocessing-1.3.0/itn/english/data/numbers/hundred.tsv +1 -0
  106. wetextprocessing-1.3.0/itn/english/data/numbers/teen.tsv +10 -0
  107. wetextprocessing-1.3.0/itn/english/data/numbers/thousands.tsv +22 -0
  108. wetextprocessing-1.3.0/itn/english/data/numbers/ties.tsv +9 -0
  109. wetextprocessing-1.3.0/itn/english/data/ordinals/digit.tsv +9 -0
  110. wetextprocessing-1.3.0/itn/english/data/time/minute_to.tsv +59 -0
  111. wetextprocessing-1.3.0/itn/english/data/time/time_suffix.tsv +8 -0
  112. wetextprocessing-1.3.0/itn/english/data/time/time_suffix_cased.tsv +8 -0
  113. wetextprocessing-1.3.0/itn/english/data/time/time_zone.tsv +7 -0
  114. wetextprocessing-1.3.0/itn/english/data/time/time_zone_cased.tsv +7 -0
  115. wetextprocessing-1.3.0/itn/english/data/time/to_hour.tsv +12 -0
  116. wetextprocessing-1.3.0/itn/english/data/whitelist.tsv +417 -0
  117. wetextprocessing-1.3.0/itn/english/data/whitelist_tech.tsv +450 -0
  118. wetextprocessing-1.3.0/itn/english/data/year_suffix.tsv +6 -0
  119. wetextprocessing-1.3.0/itn/english/inverse_normalizer.py +91 -0
  120. wetextprocessing-1.3.0/itn/english/rules/cardinal.py +111 -0
  121. wetextprocessing-1.3.0/itn/english/rules/char.py +29 -0
  122. wetextprocessing-1.3.0/itn/english/rules/date.py +163 -0
  123. wetextprocessing-1.3.0/itn/english/rules/decimal.py +82 -0
  124. wetextprocessing-1.3.0/itn/english/rules/electronic.py +79 -0
  125. wetextprocessing-1.3.0/itn/english/rules/measure.py +106 -0
  126. wetextprocessing-1.3.0/itn/english/rules/money.py +121 -0
  127. wetextprocessing-1.3.0/itn/english/rules/ordinal.py +50 -0
  128. wetextprocessing-1.3.0/itn/english/rules/punctuation.py +31 -0
  129. wetextprocessing-1.3.0/itn/english/rules/telephone.py +147 -0
  130. wetextprocessing-1.3.0/itn/english/rules/time.py +138 -0
  131. wetextprocessing-1.3.0/itn/english/rules/whitelist.py +33 -0
  132. wetextprocessing-1.3.0/itn/english/rules/word.py +26 -0
  133. wetextprocessing-1.3.0/itn/english/test/data/en_cardinal.txt +28 -0
  134. wetextprocessing-1.3.0/itn/english/test/data/en_date.txt +44 -0
  135. wetextprocessing-1.3.0/itn/english/test/data/en_decimal.txt +63 -0
  136. wetextprocessing-1.3.0/itn/english/test/data/en_electronic.txt +25 -0
  137. wetextprocessing-1.3.0/itn/english/test/data/en_measure.txt +114 -0
  138. wetextprocessing-1.3.0/itn/english/test/data/en_money.txt +52 -0
  139. wetextprocessing-1.3.0/itn/english/test/data/en_ordinal.txt +34 -0
  140. wetextprocessing-1.3.0/itn/english/test/data/en_telephone.txt +24 -0
  141. wetextprocessing-1.3.0/itn/english/test/data/en_time.txt +36 -0
  142. wetextprocessing-1.3.0/itn/english/test/data/en_whitelist.txt +12 -0
  143. wetextprocessing-1.3.0/itn/english/test/data/en_word.txt +55 -0
  144. wetextprocessing-1.3.0/itn/english/test/normalizer_test.py +130 -0
  145. wetextprocessing-1.3.0/itn/french/data/electronic/domain.tsv +25 -0
  146. wetextprocessing-1.3.0/itn/french/data/electronic/server_name.tsv +17 -0
  147. wetextprocessing-1.3.0/itn/french/data/electronic/symbols.tsv +12 -0
  148. wetextprocessing-1.3.0/itn/french/data/fractions.tsv +41 -0
  149. wetextprocessing-1.3.0/itn/french/data/measurements/magnitudes.tsv +16 -0
  150. wetextprocessing-1.3.0/itn/french/data/measurements/measurements.tsv +27 -0
  151. wetextprocessing-1.3.0/itn/french/data/money/currency_major.tsv +40 -0
  152. wetextprocessing-1.3.0/itn/french/data/money/currency_minor.tsv +5 -0
  153. wetextprocessing-1.3.0/itn/french/data/months.tsv +12 -0
  154. wetextprocessing-1.3.0/itn/french/data/numbers/digit.tsv +10 -0
  155. wetextprocessing-1.3.0/itn/french/data/numbers/teen.tsv +6 -0
  156. wetextprocessing-1.3.0/itn/french/data/numbers/ties.tsv +13 -0
  157. wetextprocessing-1.3.0/itn/french/data/numbers/ties_unique.tsv +25 -0
  158. wetextprocessing-1.3.0/itn/french/data/numbers/zero.tsv +1 -0
  159. wetextprocessing-1.3.0/itn/french/data/ordinals/digits_root_change.tsv +17 -0
  160. wetextprocessing-1.3.0/itn/french/data/ordinals/firsts.tsv +4 -0
  161. wetextprocessing-1.3.0/itn/french/data/ordinals/key_nouns.tsv +1 -0
  162. wetextprocessing-1.3.0/itn/french/data/ordinals/second.tsv +4 -0
  163. wetextprocessing-1.3.0/itn/french/data/roman/digits_large.tsv +9 -0
  164. wetextprocessing-1.3.0/itn/french/data/roman/hundreds_large.tsv +9 -0
  165. wetextprocessing-1.3.0/itn/french/data/roman/ties_large.tsv +9 -0
  166. wetextprocessing-1.3.0/itn/french/data/time/hour_to_night.tsv +12 -0
  167. wetextprocessing-1.3.0/itn/french/data/time/hours.tsv +26 -0
  168. wetextprocessing-1.3.0/itn/french/data/time/hours_to.tsv +25 -0
  169. wetextprocessing-1.3.0/itn/french/data/time/minutes.tsv +63 -0
  170. wetextprocessing-1.3.0/itn/french/data/time/minutes_to.tsv +59 -0
  171. wetextprocessing-1.3.0/itn/french/data/time/time_suffix_am.tsv +1 -0
  172. wetextprocessing-1.3.0/itn/french/data/time/time_suffix_pm.tsv +2 -0
  173. wetextprocessing-1.3.0/itn/french/data/whitelist.tsv +16 -0
  174. wetextprocessing-1.3.0/itn/french/inverse_normalizer.py +74 -0
  175. wetextprocessing-1.3.0/itn/french/rules/cardinal.py +115 -0
  176. wetextprocessing-1.3.0/itn/french/rules/date.py +45 -0
  177. wetextprocessing-1.3.0/itn/french/rules/decimal.py +70 -0
  178. wetextprocessing-1.3.0/itn/french/rules/electronic.py +67 -0
  179. wetextprocessing-1.3.0/itn/french/rules/fraction.py +55 -0
  180. wetextprocessing-1.3.0/itn/french/rules/measure.py +55 -0
  181. wetextprocessing-1.3.0/itn/french/rules/money.py +67 -0
  182. wetextprocessing-1.3.0/itn/french/rules/ordinal.py +63 -0
  183. wetextprocessing-1.3.0/itn/french/rules/punctuation.py +29 -0
  184. wetextprocessing-1.3.0/itn/french/rules/telephone.py +59 -0
  185. wetextprocessing-1.3.0/itn/french/rules/time.py +70 -0
  186. wetextprocessing-1.3.0/itn/french/rules/whitelist.py +27 -0
  187. wetextprocessing-1.3.0/itn/french/rules/word.py +29 -0
  188. wetextprocessing-1.3.0/itn/french/test/data/test_cases_cardinal.txt +106 -0
  189. wetextprocessing-1.3.0/itn/french/test/data/test_cases_date.txt +6 -0
  190. wetextprocessing-1.3.0/itn/french/test/data/test_cases_decimal.txt +15 -0
  191. wetextprocessing-1.3.0/itn/french/test/data/test_cases_electronic.txt +10 -0
  192. wetextprocessing-1.3.0/itn/french/test/data/test_cases_fraction.txt +30 -0
  193. wetextprocessing-1.3.0/itn/french/test/data/test_cases_measure.txt +15 -0
  194. wetextprocessing-1.3.0/itn/french/test/data/test_cases_money.txt +22 -0
  195. wetextprocessing-1.3.0/itn/french/test/data/test_cases_ordinal.txt +23 -0
  196. wetextprocessing-1.3.0/itn/french/test/data/test_cases_telephone.txt +5 -0
  197. wetextprocessing-1.3.0/itn/french/test/data/test_cases_time.txt +18 -0
  198. wetextprocessing-1.3.0/itn/french/test/data/test_cases_whitelist.txt +8 -0
  199. wetextprocessing-1.3.0/itn/french/test/data/test_cases_word.txt +49 -0
  200. wetextprocessing-1.3.0/itn/french/test/normalizer_test.py +155 -0
  201. wetextprocessing-1.3.0/itn/german/inverse_normalizer.py +85 -0
  202. wetextprocessing-1.3.0/itn/german/rules/cardinal.py +48 -0
  203. wetextprocessing-1.3.0/itn/german/rules/date.py +63 -0
  204. wetextprocessing-1.3.0/itn/german/rules/decimal.py +67 -0
  205. wetextprocessing-1.3.0/itn/german/rules/electronic.py +58 -0
  206. wetextprocessing-1.3.0/itn/german/rules/fraction.py +75 -0
  207. wetextprocessing-1.3.0/itn/german/rules/measure.py +80 -0
  208. wetextprocessing-1.3.0/itn/german/rules/money.py +87 -0
  209. wetextprocessing-1.3.0/itn/german/rules/ordinal.py +31 -0
  210. wetextprocessing-1.3.0/itn/german/rules/punctuation.py +29 -0
  211. wetextprocessing-1.3.0/itn/german/rules/telephone.py +45 -0
  212. wetextprocessing-1.3.0/itn/german/rules/time.py +102 -0
  213. wetextprocessing-1.3.0/itn/german/rules/whitelist.py +30 -0
  214. wetextprocessing-1.3.0/itn/german/rules/word.py +33 -0
  215. wetextprocessing-1.3.0/itn/german/test/calendar_test.py +147 -0
  216. wetextprocessing-1.3.0/itn/german/test/data/test_cases_cardinal.txt +62 -0
  217. wetextprocessing-1.3.0/itn/german/test/data/test_cases_date.txt +22 -0
  218. wetextprocessing-1.3.0/itn/german/test/data/test_cases_decimal.txt +10 -0
  219. wetextprocessing-1.3.0/itn/german/test/data/test_cases_electronic.txt +9 -0
  220. wetextprocessing-1.3.0/itn/german/test/data/test_cases_fraction.txt +34 -0
  221. wetextprocessing-1.3.0/itn/german/test/data/test_cases_measure.txt +28 -0
  222. wetextprocessing-1.3.0/itn/german/test/data/test_cases_money.txt +23 -0
  223. wetextprocessing-1.3.0/itn/german/test/data/test_cases_ordinal.txt +20 -0
  224. wetextprocessing-1.3.0/itn/german/test/data/test_cases_telephone.txt +1 -0
  225. wetextprocessing-1.3.0/itn/german/test/data/test_cases_time.txt +24 -0
  226. wetextprocessing-1.3.0/itn/german/test/data/test_cases_whitelist.txt +6 -0
  227. wetextprocessing-1.3.0/itn/german/test/data/test_cases_word.txt +49 -0
  228. wetextprocessing-1.3.0/itn/german/test/financial_test.py +284 -0
  229. wetextprocessing-1.3.0/itn/german/test/inverse_normalizer_test.py +92 -0
  230. wetextprocessing-1.3.0/itn/german/test/numeric_test.py +219 -0
  231. wetextprocessing-1.3.0/itn/german/test/text_test.py +281 -0
  232. wetextprocessing-1.3.0/itn/german/test/time_test.py +70 -0
  233. wetextprocessing-1.3.0/itn/hebrew/data/decimals/minutes_exception.tsv +2 -0
  234. wetextprocessing-1.3.0/itn/hebrew/data/measurements.tsv +45 -0
  235. wetextprocessing-1.3.0/itn/hebrew/data/months.tsv +13 -0
  236. wetextprocessing-1.3.0/itn/hebrew/data/months_name2number.tsv +12 -0
  237. wetextprocessing-1.3.0/itn/hebrew/data/months_ordinal2number.tsv +12 -0
  238. wetextprocessing-1.3.0/itn/hebrew/data/numbers/decimal_fractions.tsv +6 -0
  239. wetextprocessing-1.3.0/itn/hebrew/data/numbers/digit.tsv +20 -0
  240. wetextprocessing-1.3.0/itn/hebrew/data/numbers/hundreds_exception.tsv +2 -0
  241. wetextprocessing-1.3.0/itn/hebrew/data/numbers/millions_exception.tsv +1 -0
  242. wetextprocessing-1.3.0/itn/hebrew/data/numbers/teen.tsv +21 -0
  243. wetextprocessing-1.3.0/itn/hebrew/data/numbers/thousands.tsv +8 -0
  244. wetextprocessing-1.3.0/itn/hebrew/data/numbers/thousands_exception.tsv +2 -0
  245. wetextprocessing-1.3.0/itn/hebrew/data/numbers/ties.tsv +8 -0
  246. wetextprocessing-1.3.0/itn/hebrew/data/numbers/viable_hours.tsv +15 -0
  247. wetextprocessing-1.3.0/itn/hebrew/data/numbers/zero.tsv +1 -0
  248. wetextprocessing-1.3.0/itn/hebrew/data/ordinals/digit.tsv +10 -0
  249. wetextprocessing-1.3.0/itn/hebrew/data/prefix.tsv +17 -0
  250. wetextprocessing-1.3.0/itn/hebrew/data/spaced_measurements.tsv +17 -0
  251. wetextprocessing-1.3.0/itn/hebrew/data/time/day_suffix.tsv +2 -0
  252. wetextprocessing-1.3.0/itn/hebrew/data/time/evening_suffix.tsv +2 -0
  253. wetextprocessing-1.3.0/itn/hebrew/data/time/hour_to_evening.tsv +7 -0
  254. wetextprocessing-1.3.0/itn/hebrew/data/time/hour_to_night.tsv +9 -0
  255. wetextprocessing-1.3.0/itn/hebrew/data/time/hour_to_noon.tsv +7 -0
  256. wetextprocessing-1.3.0/itn/hebrew/data/time/midnight_to_hour.tsv +1 -0
  257. wetextprocessing-1.3.0/itn/hebrew/data/time/minute_to.tsv +58 -0
  258. wetextprocessing-1.3.0/itn/hebrew/data/time/minute_to_verbose.tsv +6 -0
  259. wetextprocessing-1.3.0/itn/hebrew/data/time/minute_verbose.tsv +8 -0
  260. wetextprocessing-1.3.0/itn/hebrew/data/time/night_suffix.tsv +1 -0
  261. wetextprocessing-1.3.0/itn/hebrew/data/time/noon_suffix.tsv +3 -0
  262. wetextprocessing-1.3.0/itn/hebrew/data/time/to_hour.tsv +13 -0
  263. wetextprocessing-1.3.0/itn/hebrew/data/whitelist.tsv +20 -0
  264. wetextprocessing-1.3.0/itn/hebrew/inverse_normalizer.py +64 -0
  265. wetextprocessing-1.3.0/itn/hebrew/rules/cardinal.py +94 -0
  266. wetextprocessing-1.3.0/itn/hebrew/rules/date.py +62 -0
  267. wetextprocessing-1.3.0/itn/hebrew/rules/decimal.py +74 -0
  268. wetextprocessing-1.3.0/itn/hebrew/rules/measure.py +53 -0
  269. wetextprocessing-1.3.0/itn/hebrew/rules/punctuation.py +28 -0
  270. wetextprocessing-1.3.0/itn/hebrew/rules/time.py +101 -0
  271. wetextprocessing-1.3.0/itn/hebrew/rules/utils.py +78 -0
  272. wetextprocessing-1.3.0/itn/hebrew/rules/whitelist.py +30 -0
  273. wetextprocessing-1.3.0/itn/hebrew/rules/word.py +28 -0
  274. wetextprocessing-1.3.0/itn/hebrew/test/data/test_cases_cardinal.txt +138 -0
  275. wetextprocessing-1.3.0/itn/hebrew/test/data/test_cases_date.txt +29 -0
  276. wetextprocessing-1.3.0/itn/hebrew/test/data/test_cases_decimal.txt +66 -0
  277. wetextprocessing-1.3.0/itn/hebrew/test/data/test_cases_measure.txt +10 -0
  278. wetextprocessing-1.3.0/itn/hebrew/test/data/test_cases_time.txt +34 -0
  279. wetextprocessing-1.3.0/itn/hebrew/test/data/test_cases_whitelist.txt +4 -0
  280. wetextprocessing-1.3.0/itn/hebrew/test/data/test_full_sentences.txt +56 -0
  281. wetextprocessing-1.3.0/itn/hebrew/test/normalizer_test.py +178 -0
  282. wetextprocessing-1.3.0/itn/hindi/data/address/cities.tsv +36 -0
  283. wetextprocessing-1.3.0/itn/hindi/data/address/context_cues.tsv +85 -0
  284. wetextprocessing-1.3.0/itn/hindi/data/address/digit_passthrough.tsv +10 -0
  285. wetextprocessing-1.3.0/itn/hindi/data/address/ordinals.tsv +167 -0
  286. wetextprocessing-1.3.0/itn/hindi/data/address/special_characters.tsv +2 -0
  287. wetextprocessing-1.3.0/itn/hindi/data/address/states.tsv +36 -0
  288. wetextprocessing-1.3.0/itn/hindi/data/date/century.tsv +5 -0
  289. wetextprocessing-1.3.0/itn/hindi/data/date/months.tsv +14 -0
  290. wetextprocessing-1.3.0/itn/hindi/data/measure/measurements.tsv +295 -0
  291. wetextprocessing-1.3.0/itn/hindi/data/money/currency.tsv +181 -0
  292. wetextprocessing-1.3.0/itn/hindi/data/numbers/digit.tsv +13 -0
  293. wetextprocessing-1.3.0/itn/hindi/data/numbers/hundred.tsv +1 -0
  294. wetextprocessing-1.3.0/itn/hindi/data/numbers/labels_exception.tsv +4 -0
  295. wetextprocessing-1.3.0/itn/hindi/data/numbers/paune.tsv +231 -0
  296. wetextprocessing-1.3.0/itn/hindi/data/numbers/teens_and_ties.tsv +220 -0
  297. wetextprocessing-1.3.0/itn/hindi/data/numbers/thousands.tsv +8 -0
  298. wetextprocessing-1.3.0/itn/hindi/data/numbers/zero.tsv +1 -0
  299. wetextprocessing-1.3.0/itn/hindi/data/ordinals/digit.tsv +6 -0
  300. wetextprocessing-1.3.0/itn/hindi/data/ordinals/digit_fem.tsv +6 -0
  301. wetextprocessing-1.3.0/itn/hindi/data/ordinals/hundred_digit.tsv +12 -0
  302. wetextprocessing-1.3.0/itn/hindi/data/ordinals/hundred_digit_fem.tsv +12 -0
  303. wetextprocessing-1.3.0/itn/hindi/data/ordinals/morph_features.tsv +6 -0
  304. wetextprocessing-1.3.0/itn/hindi/data/ordinals/suffix.tsv +4 -0
  305. wetextprocessing-1.3.0/itn/hindi/data/ordinals/suffix_fem.tsv +2 -0
  306. wetextprocessing-1.3.0/itn/hindi/data/ordinals/teens_and_ties.tsv +1 -0
  307. wetextprocessing-1.3.0/itn/hindi/data/ordinals/teens_and_ties_fem.tsv +1 -0
  308. wetextprocessing-1.3.0/itn/hindi/data/telephone/context_cues.tsv +15 -0
  309. wetextprocessing-1.3.0/itn/hindi/data/telephone/country_codes.tsv +1 -0
  310. wetextprocessing-1.3.0/itn/hindi/data/telephone/eng_digit.tsv +9 -0
  311. wetextprocessing-1.3.0/itn/hindi/data/telephone/eng_zero.tsv +1 -0
  312. wetextprocessing-1.3.0/itn/hindi/data/telephone/teens_and_ties_eng_to_hin.tsv +90 -0
  313. wetextprocessing-1.3.0/itn/hindi/data/time/hour_for_paune.tsv +15 -0
  314. wetextprocessing-1.3.0/itn/hindi/data/whitelist/whitelist.tsv +7 -0
  315. wetextprocessing-1.3.0/itn/hindi/inverse_normalizer.py +78 -0
  316. wetextprocessing-1.3.0/itn/hindi/rules/cardinal.py +93 -0
  317. wetextprocessing-1.3.0/itn/hindi/rules/date.py +64 -0
  318. wetextprocessing-1.3.0/itn/hindi/rules/decimal.py +57 -0
  319. wetextprocessing-1.3.0/itn/hindi/rules/fraction.py +55 -0
  320. wetextprocessing-1.3.0/itn/hindi/rules/measure.py +83 -0
  321. wetextprocessing-1.3.0/itn/hindi/rules/money.py +81 -0
  322. wetextprocessing-1.3.0/itn/hindi/rules/ordinal.py +40 -0
  323. wetextprocessing-1.3.0/itn/hindi/rules/punctuation.py +31 -0
  324. wetextprocessing-1.3.0/itn/hindi/rules/telephone.py +65 -0
  325. wetextprocessing-1.3.0/itn/hindi/rules/time.py +53 -0
  326. wetextprocessing-1.3.0/itn/hindi/rules/utils.py +52 -0
  327. wetextprocessing-1.3.0/itn/hindi/rules/whitelist.py +32 -0
  328. wetextprocessing-1.3.0/itn/hindi/rules/word.py +30 -0
  329. wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_address.txt +25 -0
  330. wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_cardinal.txt +54 -0
  331. wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_date.txt +42 -0
  332. wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_decimal.txt +13 -0
  333. wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_fraction.txt +31 -0
  334. wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_measure.txt +48 -0
  335. wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_money.txt +50 -0
  336. wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_ordinal.txt +13 -0
  337. wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_telephone.txt +28 -0
  338. wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_time.txt +25 -0
  339. wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_whitelist.txt +8 -0
  340. wetextprocessing-1.3.0/itn/hindi/test/data/test_cases_word.txt +15 -0
  341. wetextprocessing-1.3.0/itn/hindi/test/normalizer_test.py +127 -0
  342. wetextprocessing-1.3.0/itn/hindi_english/__init__.py +0 -0
  343. wetextprocessing-1.3.0/itn/hindi_english/data/measurements.tsv +2 -0
  344. wetextprocessing-1.3.0/itn/hindi_english/inverse_normalizer.py +93 -0
  345. wetextprocessing-1.3.0/itn/hindi_english/rules/__init__.py +0 -0
  346. wetextprocessing-1.3.0/itn/hindi_english/rules/english.py +175 -0
  347. wetextprocessing-1.3.0/itn/hindi_english/test/__init__.py +0 -0
  348. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_address.txt +41 -0
  349. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_cardinal.txt +61 -0
  350. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_date.txt +37 -0
  351. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_date_cased.txt +70 -0
  352. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_decimal.txt +25 -0
  353. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_electronic.txt +17 -0
  354. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_fraction.txt +39 -0
  355. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_measure.txt +25 -0
  356. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_money.txt +41 -0
  357. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_ordinal.txt +26 -0
  358. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_telephone.txt +39 -0
  359. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_time.txt +29 -0
  360. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_whitelist.txt +17 -0
  361. wetextprocessing-1.3.0/itn/hindi_english/test/data/test_cases_word.txt +19 -0
  362. wetextprocessing-1.3.0/itn/hindi_english/test/normalizer_test.py +211 -0
  363. wetextprocessing-1.3.0/itn/hungarian/__init__.py +0 -0
  364. wetextprocessing-1.3.0/itn/hungarian/data/dates/days.tsv +31 -0
  365. wetextprocessing-1.3.0/itn/hungarian/data/dates/months.tsv +12 -0
  366. wetextprocessing-1.3.0/itn/hungarian/data/electronic/domain.tsv +15 -0
  367. wetextprocessing-1.3.0/itn/hungarian/data/electronic/server_name.tsv +15 -0
  368. wetextprocessing-1.3.0/itn/hungarian/data/electronic/symbols.tsv +23 -0
  369. wetextprocessing-1.3.0/itn/hungarian/data/inflection/endings.tsv +51 -0
  370. wetextprocessing-1.3.0/itn/hungarian/data/inflection/plural_endings.tsv +39 -0
  371. wetextprocessing-1.3.0/itn/hungarian/data/inflection/word_endings.tsv +1 -0
  372. wetextprocessing-1.3.0/itn/hungarian/data/measures/measurements.tsv +52 -0
  373. wetextprocessing-1.3.0/itn/hungarian/data/money/currency.tsv +11 -0
  374. wetextprocessing-1.3.0/itn/hungarian/data/money/currency_minor.tsv +11 -0
  375. wetextprocessing-1.3.0/itn/hungarian/data/number/digit.tsv +9 -0
  376. wetextprocessing-1.3.0/itn/hungarian/data/number/digit_inline.tsv +9 -0
  377. wetextprocessing-1.3.0/itn/hungarian/data/number/quantities.tsv +21 -0
  378. wetextprocessing-1.3.0/itn/hungarian/data/number/tens.tsv +9 -0
  379. wetextprocessing-1.3.0/itn/hungarian/data/number/tens_inline.tsv +9 -0
  380. wetextprocessing-1.3.0/itn/hungarian/data/number/zero.tsv +1 -0
  381. wetextprocessing-1.3.0/itn/hungarian/data/ordinals/endings.tsv +23 -0
  382. wetextprocessing-1.3.0/itn/hungarian/data/ordinals/exceptional.tsv +4 -0
  383. wetextprocessing-1.3.0/itn/hungarian/data/telephone/area_codes.tsv +71 -0
  384. wetextprocessing-1.3.0/itn/hungarian/data/telephone/country_codes.tsv +213 -0
  385. wetextprocessing-1.3.0/itn/hungarian/data/time/time_zone.tsv +9 -0
  386. wetextprocessing-1.3.0/itn/hungarian/data/whitelist.tsv +24 -0
  387. wetextprocessing-1.3.0/itn/hungarian/data/whitelist_inflect.tsv +39 -0
  388. wetextprocessing-1.3.0/itn/hungarian/data/whitelist_inflect_sg.tsv +14 -0
  389. wetextprocessing-1.3.0/itn/hungarian/inverse_normalizer.py +93 -0
  390. wetextprocessing-1.3.0/itn/hungarian/rules/__init__.py +0 -0
  391. wetextprocessing-1.3.0/itn/hungarian/rules/cardinal.py +99 -0
  392. wetextprocessing-1.3.0/itn/hungarian/rules/date.py +66 -0
  393. wetextprocessing-1.3.0/itn/hungarian/rules/decimal.py +80 -0
  394. wetextprocessing-1.3.0/itn/hungarian/rules/electronic.py +71 -0
  395. wetextprocessing-1.3.0/itn/hungarian/rules/fraction.py +41 -0
  396. wetextprocessing-1.3.0/itn/hungarian/rules/measure.py +74 -0
  397. wetextprocessing-1.3.0/itn/hungarian/rules/money.py +82 -0
  398. wetextprocessing-1.3.0/itn/hungarian/rules/ordinal.py +42 -0
  399. wetextprocessing-1.3.0/itn/hungarian/rules/punctuation.py +44 -0
  400. wetextprocessing-1.3.0/itn/hungarian/rules/telephone.py +68 -0
  401. wetextprocessing-1.3.0/itn/hungarian/rules/time.py +99 -0
  402. wetextprocessing-1.3.0/itn/hungarian/rules/utils.py +143 -0
  403. wetextprocessing-1.3.0/itn/hungarian/rules/whitelist.py +50 -0
  404. wetextprocessing-1.3.0/itn/hungarian/rules/word.py +38 -0
  405. wetextprocessing-1.3.0/itn/hungarian/test/__init__.py +0 -0
  406. wetextprocessing-1.3.0/itn/hungarian/test/normalizer_test.py +214 -0
  407. wetextprocessing-1.3.0/itn/italian/__init__.py +0 -0
  408. wetextprocessing-1.3.0/itn/italian/data/electronic/domain.tsv +14 -0
  409. wetextprocessing-1.3.0/itn/italian/data/electronic/server_name.tsv +11 -0
  410. wetextprocessing-1.3.0/itn/italian/data/electronic/symbols.tsv +22 -0
  411. wetextprocessing-1.3.0/itn/italian/data/measure/denominators.tsv +4 -0
  412. wetextprocessing-1.3.0/itn/italian/data/measure/units.tsv +51 -0
  413. wetextprocessing-1.3.0/itn/italian/data/money/currency.tsv +11 -0
  414. wetextprocessing-1.3.0/itn/italian/data/numbers/digit.tsv +9 -0
  415. wetextprocessing-1.3.0/itn/italian/data/numbers/hundreds.tsv +9 -0
  416. wetextprocessing-1.3.0/itn/italian/data/numbers/quantities.tsv +4 -0
  417. wetextprocessing-1.3.0/itn/italian/data/numbers/teen.tsv +10 -0
  418. wetextprocessing-1.3.0/itn/italian/data/numbers/tens.tsv +8 -0
  419. wetextprocessing-1.3.0/itn/italian/data/whitelist.tsv +19 -0
  420. wetextprocessing-1.3.0/itn/italian/inverse_normalizer.py +77 -0
  421. wetextprocessing-1.3.0/itn/italian/rules/__init__.py +0 -0
  422. wetextprocessing-1.3.0/itn/italian/rules/cardinal.py +100 -0
  423. wetextprocessing-1.3.0/itn/italian/rules/decimal.py +59 -0
  424. wetextprocessing-1.3.0/itn/italian/rules/electronic.py +70 -0
  425. wetextprocessing-1.3.0/itn/italian/rules/measure.py +52 -0
  426. wetextprocessing-1.3.0/itn/italian/rules/money.py +78 -0
  427. wetextprocessing-1.3.0/itn/italian/rules/punctuation.py +27 -0
  428. wetextprocessing-1.3.0/itn/italian/rules/time.py +84 -0
  429. wetextprocessing-1.3.0/itn/italian/rules/utils.py +49 -0
  430. wetextprocessing-1.3.0/itn/italian/rules/whitelist.py +31 -0
  431. wetextprocessing-1.3.0/itn/italian/rules/word.py +27 -0
  432. wetextprocessing-1.3.0/itn/italian/test/__init__.py +0 -0
  433. wetextprocessing-1.3.0/itn/italian/test/normalizer_test.py +277 -0
  434. wetextprocessing-1.3.0/itn/japanese/__init__.py +0 -0
  435. wetextprocessing-1.3.0/itn/japanese/data/default/blacklist.tsv +0 -0
  436. wetextprocessing-1.3.0/itn/japanese/inverse_normalizer.py +111 -0
  437. wetextprocessing-1.3.0/itn/japanese/rules/__init__.py +0 -0
  438. wetextprocessing-1.3.0/itn/japanese/rules/cardinal.py +143 -0
  439. wetextprocessing-1.3.0/itn/japanese/rules/char.py +31 -0
  440. wetextprocessing-1.3.0/itn/japanese/rules/date.py +68 -0
  441. wetextprocessing-1.3.0/itn/japanese/rules/fraction.py +66 -0
  442. wetextprocessing-1.3.0/itn/japanese/rules/math.py +40 -0
  443. wetextprocessing-1.3.0/itn/japanese/rules/measure.py +49 -0
  444. wetextprocessing-1.3.0/itn/japanese/rules/money.py +48 -0
  445. wetextprocessing-1.3.0/itn/japanese/rules/ordinal.py +36 -0
  446. wetextprocessing-1.3.0/itn/japanese/rules/postprocessor.py +46 -0
  447. wetextprocessing-1.3.0/itn/japanese/rules/preprocessor.py +31 -0
  448. wetextprocessing-1.3.0/itn/japanese/rules/time.py +52 -0
  449. wetextprocessing-1.3.0/itn/japanese/rules/whitelist.py +35 -0
  450. wetextprocessing-1.3.0/itn/japanese/test/__init__.py +0 -0
  451. wetextprocessing-1.3.0/itn/japanese/test/normalizer_test.py +215 -0
  452. wetextprocessing-1.3.0/itn/kinyarwanda/__init__.py +0 -0
  453. wetextprocessing-1.3.0/itn/kinyarwanda/data/numbers/digit.tsv +9 -0
  454. wetextprocessing-1.3.0/itn/kinyarwanda/data/numbers/digit_bi.tsv +9 -0
  455. wetextprocessing-1.3.0/itn/kinyarwanda/data/numbers/digit_e.tsv +9 -0
  456. wetextprocessing-1.3.0/itn/kinyarwanda/data/numbers/hundreds.tsv +9 -0
  457. wetextprocessing-1.3.0/itn/kinyarwanda/data/numbers/tens.tsv +8 -0
  458. wetextprocessing-1.3.0/itn/kinyarwanda/data/numbers/zero.tsv +1 -0
  459. wetextprocessing-1.3.0/itn/kinyarwanda/data/time/hours.tsv +12 -0
  460. wetextprocessing-1.3.0/itn/kinyarwanda/data/time/minutes.tsv +59 -0
  461. wetextprocessing-1.3.0/itn/kinyarwanda/data/whitelist.tsv +18 -0
  462. wetextprocessing-1.3.0/itn/kinyarwanda/inverse_normalizer.py +53 -0
  463. wetextprocessing-1.3.0/itn/kinyarwanda/rules/__init__.py +0 -0
  464. wetextprocessing-1.3.0/itn/kinyarwanda/rules/cardinal.py +131 -0
  465. wetextprocessing-1.3.0/itn/kinyarwanda/rules/punctuation.py +28 -0
  466. wetextprocessing-1.3.0/itn/kinyarwanda/rules/time.py +39 -0
  467. wetextprocessing-1.3.0/itn/kinyarwanda/rules/utils.py +39 -0
  468. wetextprocessing-1.3.0/itn/kinyarwanda/rules/whitelist.py +32 -0
  469. wetextprocessing-1.3.0/itn/kinyarwanda/rules/word.py +28 -0
  470. wetextprocessing-1.3.0/itn/kinyarwanda/test/__init__.py +0 -0
  471. wetextprocessing-1.3.0/itn/kinyarwanda/test/normalizer_test.py +194 -0
  472. wetextprocessing-1.3.0/itn/korean/__init__.py +3 -0
  473. wetextprocessing-1.3.0/itn/korean/data/__init__.py +13 -0
  474. wetextprocessing-1.3.0/itn/korean/data/currency.tsv +9 -0
  475. wetextprocessing-1.3.0/itn/korean/data/measure_units.tsv +61 -0
  476. wetextprocessing-1.3.0/itn/korean/data/months.tsv +12 -0
  477. wetextprocessing-1.3.0/itn/korean/data/numbers/__init__.py +13 -0
  478. wetextprocessing-1.3.0/itn/korean/data/numbers/digit.tsv +9 -0
  479. wetextprocessing-1.3.0/itn/korean/data/numbers/zero.tsv +1 -0
  480. wetextprocessing-1.3.0/itn/korean/data/ordinals/__init__.py +13 -0
  481. wetextprocessing-1.3.0/itn/korean/data/ordinals/counter_suffix.tsv +17 -0
  482. wetextprocessing-1.3.0/itn/korean/data/ordinals/digit.tsv +9 -0
  483. wetextprocessing-1.3.0/itn/korean/data/time/__init__.py +13 -0
  484. wetextprocessing-1.3.0/itn/korean/data/time/ten_prefix.tsv +4 -0
  485. wetextprocessing-1.3.0/itn/korean/data/time/time_hours.tsv +23 -0
  486. wetextprocessing-1.3.0/itn/korean/data/whitelist.tsv +35 -0
  487. wetextprocessing-1.3.0/itn/korean/inverse_normalizer.py +76 -0
  488. wetextprocessing-1.3.0/itn/korean/rules/__init__.py +3 -0
  489. wetextprocessing-1.3.0/itn/korean/rules/cardinal.py +44 -0
  490. wetextprocessing-1.3.0/itn/korean/rules/date.py +42 -0
  491. wetextprocessing-1.3.0/itn/korean/rules/decimal.py +43 -0
  492. wetextprocessing-1.3.0/itn/korean/rules/fraction.py +48 -0
  493. wetextprocessing-1.3.0/itn/korean/rules/measure.py +52 -0
  494. wetextprocessing-1.3.0/itn/korean/rules/money.py +34 -0
  495. wetextprocessing-1.3.0/itn/korean/rules/ordinal.py +49 -0
  496. wetextprocessing-1.3.0/itn/korean/rules/telephone.py +30 -0
  497. wetextprocessing-1.3.0/itn/korean/rules/time.py +61 -0
  498. wetextprocessing-1.3.0/itn/korean/rules/utils.py +22 -0
  499. wetextprocessing-1.3.0/itn/korean/rules/whitelist.py +28 -0
  500. wetextprocessing-1.3.0/itn/korean/rules/word.py +27 -0
  501. wetextprocessing-1.3.0/itn/korean/test/__init__.py +0 -0
  502. wetextprocessing-1.3.0/itn/korean/test/data/test_cases_cardinal.txt +37 -0
  503. wetextprocessing-1.3.0/itn/korean/test/data/test_cases_date.txt +15 -0
  504. wetextprocessing-1.3.0/itn/korean/test/data/test_cases_decimal.txt +14 -0
  505. wetextprocessing-1.3.0/itn/korean/test/data/test_cases_fraction.txt +29 -0
  506. wetextprocessing-1.3.0/itn/korean/test/data/test_cases_measure.txt +46 -0
  507. wetextprocessing-1.3.0/itn/korean/test/data/test_cases_money.txt +29 -0
  508. wetextprocessing-1.3.0/itn/korean/test/data/test_cases_ordinal.txt +17 -0
  509. wetextprocessing-1.3.0/itn/korean/test/data/test_cases_telephone.txt +7 -0
  510. wetextprocessing-1.3.0/itn/korean/test/data/test_cases_time.txt +15 -0
  511. wetextprocessing-1.3.0/itn/korean/test/test_inverse_normalizer.py +184 -0
  512. wetextprocessing-1.3.0/itn/main.py +23 -0
  513. wetextprocessing-1.3.0/itn/marathi/__init__.py +0 -0
  514. wetextprocessing-1.3.0/itn/marathi/data/date/eras.tsv +2 -0
  515. wetextprocessing-1.3.0/itn/marathi/data/date/months.tsv +12 -0
  516. wetextprocessing-1.3.0/itn/marathi/data/numbers/digit.tsv +9 -0
  517. wetextprocessing-1.3.0/itn/marathi/data/numbers/tens.tsv +90 -0
  518. wetextprocessing-1.3.0/itn/marathi/data/numbers/zero.tsv +1 -0
  519. wetextprocessing-1.3.0/itn/marathi/inverse_normalizer.py +56 -0
  520. wetextprocessing-1.3.0/itn/marathi/rules/__init__.py +0 -0
  521. wetextprocessing-1.3.0/itn/marathi/rules/cardinal.py +76 -0
  522. wetextprocessing-1.3.0/itn/marathi/rules/date.py +63 -0
  523. wetextprocessing-1.3.0/itn/marathi/rules/decimal.py +62 -0
  524. wetextprocessing-1.3.0/itn/marathi/rules/punctuation.py +29 -0
  525. wetextprocessing-1.3.0/itn/marathi/rules/time.py +69 -0
  526. wetextprocessing-1.3.0/itn/marathi/rules/utils.py +62 -0
  527. wetextprocessing-1.3.0/itn/marathi/rules/word.py +28 -0
  528. wetextprocessing-1.3.0/itn/marathi/test/__init__.py +0 -0
  529. wetextprocessing-1.3.0/itn/marathi/test/data/test_cases_cardinal.txt +33 -0
  530. wetextprocessing-1.3.0/itn/marathi/test/data/test_cases_date.txt +29 -0
  531. wetextprocessing-1.3.0/itn/marathi/test/data/test_cases_decimal.txt +29 -0
  532. wetextprocessing-1.3.0/itn/marathi/test/data/test_cases_time.txt +29 -0
  533. wetextprocessing-1.3.0/itn/marathi/test/data/test_cases_word.txt +11 -0
  534. wetextprocessing-1.3.0/itn/marathi/test/normalizer_test.py +164 -0
  535. wetextprocessing-1.3.0/itn/portuguese/__init__.py +0 -0
  536. wetextprocessing-1.3.0/itn/portuguese/data/currency_plural.tsv +5 -0
  537. wetextprocessing-1.3.0/itn/portuguese/data/currency_singular.tsv +5 -0
  538. wetextprocessing-1.3.0/itn/portuguese/data/electronic/domain.tsv +26 -0
  539. wetextprocessing-1.3.0/itn/portuguese/data/electronic/server_name.tsv +11 -0
  540. wetextprocessing-1.3.0/itn/portuguese/data/electronic/symbols.tsv +6 -0
  541. wetextprocessing-1.3.0/itn/portuguese/data/measurements_plural.tsv +56 -0
  542. wetextprocessing-1.3.0/itn/portuguese/data/measurements_singular.tsv +55 -0
  543. wetextprocessing-1.3.0/itn/portuguese/data/months.tsv +12 -0
  544. wetextprocessing-1.3.0/itn/portuguese/data/numbers/digit.tsv +11 -0
  545. wetextprocessing-1.3.0/itn/portuguese/data/numbers/hundreds.tsv +17 -0
  546. wetextprocessing-1.3.0/itn/portuguese/data/numbers/onehundred.tsv +1 -0
  547. wetextprocessing-1.3.0/itn/portuguese/data/numbers/teen.tsv +11 -0
  548. wetextprocessing-1.3.0/itn/portuguese/data/numbers/ties.tsv +8 -0
  549. wetextprocessing-1.3.0/itn/portuguese/data/numbers/twenties.tsv +9 -0
  550. wetextprocessing-1.3.0/itn/portuguese/data/numbers/zero.tsv +1 -0
  551. wetextprocessing-1.3.0/itn/portuguese/data/ordinals/digit.tsv +18 -0
  552. wetextprocessing-1.3.0/itn/portuguese/data/ordinals/hundreds.tsv +28 -0
  553. wetextprocessing-1.3.0/itn/portuguese/data/ordinals/ties.tsv +20 -0
  554. wetextprocessing-1.3.0/itn/portuguese/data/time/hours_to.tsv +23 -0
  555. wetextprocessing-1.3.0/itn/portuguese/data/time/minutes_to.tsv +59 -0
  556. wetextprocessing-1.3.0/itn/portuguese/data/time/time_suffix_am.tsv +2 -0
  557. wetextprocessing-1.3.0/itn/portuguese/data/time/time_suffix_pm.tsv +2 -0
  558. wetextprocessing-1.3.0/itn/portuguese/data/whitelist.tsv +5 -0
  559. wetextprocessing-1.3.0/itn/portuguese/inverse_normalizer.py +69 -0
  560. wetextprocessing-1.3.0/itn/portuguese/rules/__init__.py +0 -0
  561. wetextprocessing-1.3.0/itn/portuguese/rules/cardinal.py +117 -0
  562. wetextprocessing-1.3.0/itn/portuguese/rules/date.py +64 -0
  563. wetextprocessing-1.3.0/itn/portuguese/rules/decimal.py +76 -0
  564. wetextprocessing-1.3.0/itn/portuguese/rules/electronic.py +69 -0
  565. wetextprocessing-1.3.0/itn/portuguese/rules/measure.py +61 -0
  566. wetextprocessing-1.3.0/itn/portuguese/rules/money.py +89 -0
  567. wetextprocessing-1.3.0/itn/portuguese/rules/ordinal.py +56 -0
  568. wetextprocessing-1.3.0/itn/portuguese/rules/punctuation.py +32 -0
  569. wetextprocessing-1.3.0/itn/portuguese/rules/telephone.py +56 -0
  570. wetextprocessing-1.3.0/itn/portuguese/rules/time.py +107 -0
  571. wetextprocessing-1.3.0/itn/portuguese/rules/whitelist.py +32 -0
  572. wetextprocessing-1.3.0/itn/portuguese/rules/word.py +31 -0
  573. wetextprocessing-1.3.0/itn/portuguese/test/__init__.py +0 -0
  574. wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_cardinal.txt +70 -0
  575. wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_date.txt +15 -0
  576. wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_decimal.txt +26 -0
  577. wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_electronic.txt +13 -0
  578. wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_measure.txt +12 -0
  579. wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_money.txt +25 -0
  580. wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_ordinal.txt +19 -0
  581. wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_telephone.txt +27 -0
  582. wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_time.txt +19 -0
  583. wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_whitelist.txt +6 -0
  584. wetextprocessing-1.3.0/itn/portuguese/test/data/test_cases_word.txt +49 -0
  585. wetextprocessing-1.3.0/itn/portuguese/test/normalizer_test.py +156 -0
  586. wetextprocessing-1.3.0/itn/russian/__init__.py +0 -0
  587. wetextprocessing-1.3.0/itn/russian/data/currency/currency_plural.tsv +53 -0
  588. wetextprocessing-1.3.0/itn/russian/data/currency/currency_singular.tsv +48 -0
  589. wetextprocessing-1.3.0/itn/russian/data/electronic/domain.tsv +9 -0
  590. wetextprocessing-1.3.0/itn/russian/data/electronic/server_name.tsv +17 -0
  591. wetextprocessing-1.3.0/itn/russian/data/electronic/symbols.tsv +28 -0
  592. wetextprocessing-1.3.0/itn/russian/data/latin_to_cyrillic.tsv +38 -0
  593. wetextprocessing-1.3.0/itn/russian/data/measurements.tsv +321 -0
  594. wetextprocessing-1.3.0/itn/russian/data/months/abbr.tsv +12 -0
  595. wetextprocessing-1.3.0/itn/russian/data/months/abbr_to_name.tsv +62 -0
  596. wetextprocessing-1.3.0/itn/russian/data/months/numbers.tsv +21 -0
  597. wetextprocessing-1.3.0/itn/russian/data/numbers/1_cardinals_nominative.tsv +50 -0
  598. wetextprocessing-1.3.0/itn/russian/data/numbers/2_cardinals_genitive.tsv +45 -0
  599. wetextprocessing-1.3.0/itn/russian/data/numbers/3_cardinals_dative.tsv +45 -0
  600. wetextprocessing-1.3.0/itn/russian/data/numbers/4_cardinals_accusative.tsv +48 -0
  601. wetextprocessing-1.3.0/itn/russian/data/numbers/5_cardinals_instrumental.tsv +47 -0
  602. wetextprocessing-1.3.0/itn/russian/data/numbers/6_cardinals_prepositional.tsv +46 -0
  603. wetextprocessing-1.3.0/itn/russian/data/numbers/cardinal_endings.tsv +6 -0
  604. wetextprocessing-1.3.0/itn/russian/data/numbers/cardinals_alternatives.tsv +16 -0
  605. wetextprocessing-1.3.0/itn/russian/data/numbers/cardinals_nominative_case.tsv +37 -0
  606. wetextprocessing-1.3.0/itn/russian/data/numbers/decimal_delimiter.tsv +15 -0
  607. wetextprocessing-1.3.0/itn/russian/data/numbers/decimal_endings.tsv +37 -0
  608. wetextprocessing-1.3.0/itn/russian/data/numbers/digits_nominative_case.tsv +10 -0
  609. wetextprocessing-1.3.0/itn/russian/data/numbers/ordinal_endings.tsv +40 -0
  610. wetextprocessing-1.3.0/itn/russian/data/numbers/ordinals.tsv +752 -0
  611. wetextprocessing-1.3.0/itn/russian/data/numbers/quantity.tsv +16 -0
  612. wetextprocessing-1.3.0/itn/russian/data/time/increment_hour_cardinal.tsv +24 -0
  613. wetextprocessing-1.3.0/itn/russian/data/time/increment_hour_ordinal.tsv +24 -0
  614. wetextprocessing-1.3.0/itn/russian/data/time/minutes_to_hour.tsv +25 -0
  615. wetextprocessing-1.3.0/itn/russian/data/time/time_convert.tsv +12 -0
  616. wetextprocessing-1.3.0/itn/russian/data/whitelist.tsv +37 -0
  617. wetextprocessing-1.3.0/itn/russian/inverse_normalizer.py +71 -0
  618. wetextprocessing-1.3.0/itn/russian/rules/__init__.py +0 -0
  619. wetextprocessing-1.3.0/itn/russian/rules/cardinal.py +57 -0
  620. wetextprocessing-1.3.0/itn/russian/rules/date.py +78 -0
  621. wetextprocessing-1.3.0/itn/russian/rules/decimal.py +55 -0
  622. wetextprocessing-1.3.0/itn/russian/rules/electronic.py +67 -0
  623. wetextprocessing-1.3.0/itn/russian/rules/measure.py +45 -0
  624. wetextprocessing-1.3.0/itn/russian/rules/money.py +53 -0
  625. wetextprocessing-1.3.0/itn/russian/rules/ordinal.py +40 -0
  626. wetextprocessing-1.3.0/itn/russian/rules/punctuation.py +40 -0
  627. wetextprocessing-1.3.0/itn/russian/rules/telephone.py +45 -0
  628. wetextprocessing-1.3.0/itn/russian/rules/time.py +83 -0
  629. wetextprocessing-1.3.0/itn/russian/rules/utils.py +151 -0
  630. wetextprocessing-1.3.0/itn/russian/rules/whitelist.py +29 -0
  631. wetextprocessing-1.3.0/itn/russian/rules/word.py +30 -0
  632. wetextprocessing-1.3.0/itn/russian/test/__init__.py +0 -0
  633. wetextprocessing-1.3.0/itn/russian/test/data/test_cases_cardinal.txt +30 -0
  634. wetextprocessing-1.3.0/itn/russian/test/data/test_cases_date.txt +15 -0
  635. wetextprocessing-1.3.0/itn/russian/test/data/test_cases_decimal.txt +16 -0
  636. wetextprocessing-1.3.0/itn/russian/test/data/test_cases_electronic.txt +4 -0
  637. wetextprocessing-1.3.0/itn/russian/test/data/test_cases_measure.txt +16 -0
  638. wetextprocessing-1.3.0/itn/russian/test/data/test_cases_money.txt +9 -0
  639. wetextprocessing-1.3.0/itn/russian/test/data/test_cases_ordinal.txt +15 -0
  640. wetextprocessing-1.3.0/itn/russian/test/data/test_cases_ordinal_hard.txt +57 -0
  641. wetextprocessing-1.3.0/itn/russian/test/data/test_cases_telephone.txt +1 -0
  642. wetextprocessing-1.3.0/itn/russian/test/data/test_cases_time.txt +9 -0
  643. wetextprocessing-1.3.0/itn/russian/test/data/test_cases_whitelist.txt +4 -0
  644. wetextprocessing-1.3.0/itn/russian/test/data/test_cases_word.txt +50 -0
  645. wetextprocessing-1.3.0/itn/russian/test/normalizer_test.py +193 -0
  646. wetextprocessing-1.3.0/itn/spanish/__init__.py +0 -0
  647. wetextprocessing-1.3.0/itn/spanish/data/dates/months.tsv +12 -0
  648. wetextprocessing-1.3.0/itn/spanish/data/dates/months_cased.tsv +12 -0
  649. wetextprocessing-1.3.0/itn/spanish/data/dates/year_suffix.tsv +11 -0
  650. wetextprocessing-1.3.0/itn/spanish/data/dates/year_suffix_cased.tsv +11 -0
  651. wetextprocessing-1.3.0/itn/spanish/data/electronic/domain.tsv +27 -0
  652. wetextprocessing-1.3.0/itn/spanish/data/electronic/server_name.tsv +17 -0
  653. wetextprocessing-1.3.0/itn/spanish/data/electronic/symbols.tsv +22 -0
  654. wetextprocessing-1.3.0/itn/spanish/data/fractions/numbers_read_as_ordinals.tsv +28 -0
  655. wetextprocessing-1.3.0/itn/spanish/data/fractions/ordinal_exceptions.tsv +2 -0
  656. wetextprocessing-1.3.0/itn/spanish/data/measures/math_symbols.tsv +4 -0
  657. wetextprocessing-1.3.0/itn/spanish/data/measures/measurements_plural.tsv +34 -0
  658. wetextprocessing-1.3.0/itn/spanish/data/measures/measurements_singular.tsv +35 -0
  659. wetextprocessing-1.3.0/itn/spanish/data/money/currency_major_plural.tsv +43 -0
  660. wetextprocessing-1.3.0/itn/spanish/data/money/currency_major_plural_capitalized.tsv +75 -0
  661. wetextprocessing-1.3.0/itn/spanish/data/money/currency_major_plural_ext.tsv +196 -0
  662. wetextprocessing-1.3.0/itn/spanish/data/money/currency_major_singular.tsv +43 -0
  663. wetextprocessing-1.3.0/itn/spanish/data/money/currency_major_singular_capitalized.tsv +76 -0
  664. wetextprocessing-1.3.0/itn/spanish/data/money/currency_major_singular_ext.tsv +196 -0
  665. wetextprocessing-1.3.0/itn/spanish/data/money/currency_minor_plural.tsv +5 -0
  666. wetextprocessing-1.3.0/itn/spanish/data/money/currency_minor_plural_ext.tsv +62 -0
  667. wetextprocessing-1.3.0/itn/spanish/data/money/currency_minor_singular.tsv +5 -0
  668. wetextprocessing-1.3.0/itn/spanish/data/money/currency_minor_singular_ext.tsv +64 -0
  669. wetextprocessing-1.3.0/itn/spanish/data/months.tsv +12 -0
  670. wetextprocessing-1.3.0/itn/spanish/data/numbers/digit.tsv +11 -0
  671. wetextprocessing-1.3.0/itn/spanish/data/numbers/hundreds.tsv +18 -0
  672. wetextprocessing-1.3.0/itn/spanish/data/numbers/teen.tsv +10 -0
  673. wetextprocessing-1.3.0/itn/spanish/data/numbers/ties.tsv +8 -0
  674. wetextprocessing-1.3.0/itn/spanish/data/numbers/twenties.tsv +11 -0
  675. wetextprocessing-1.3.0/itn/spanish/data/numbers/zero.tsv +1 -0
  676. wetextprocessing-1.3.0/itn/spanish/data/ordinals/digit.tsv +22 -0
  677. wetextprocessing-1.3.0/itn/spanish/data/ordinals/digit_capitalized.tsv +22 -0
  678. wetextprocessing-1.3.0/itn/spanish/data/ordinals/hundreds.tsv +18 -0
  679. wetextprocessing-1.3.0/itn/spanish/data/ordinals/hundreds_capitalized.tsv +18 -0
  680. wetextprocessing-1.3.0/itn/spanish/data/ordinals/teen.tsv +60 -0
  681. wetextprocessing-1.3.0/itn/spanish/data/ordinals/teen_capitalized.tsv +60 -0
  682. wetextprocessing-1.3.0/itn/spanish/data/ordinals/ties.tsv +15 -0
  683. wetextprocessing-1.3.0/itn/spanish/data/ordinals/ties_capitalized.tsv +15 -0
  684. wetextprocessing-1.3.0/itn/spanish/data/ordinals/twenties.tsv +50 -0
  685. wetextprocessing-1.3.0/itn/spanish/data/ordinals/twenties_capitalized.tsv +50 -0
  686. wetextprocessing-1.3.0/itn/spanish/data/roman/digit.tsv +9 -0
  687. wetextprocessing-1.3.0/itn/spanish/data/roman/hundreds.tsv +9 -0
  688. wetextprocessing-1.3.0/itn/spanish/data/roman/thousands.tsv +3 -0
  689. wetextprocessing-1.3.0/itn/spanish/data/roman/ties.tsv +9 -0
  690. wetextprocessing-1.3.0/itn/spanish/data/time/minutes_to.tsv +6 -0
  691. wetextprocessing-1.3.0/itn/spanish/data/time/time_suffix.tsv +15 -0
  692. wetextprocessing-1.3.0/itn/spanish/data/time/time_suffix_cased.tsv +15 -0
  693. wetextprocessing-1.3.0/itn/spanish/data/time/time_to.tsv +25 -0
  694. wetextprocessing-1.3.0/itn/spanish/data/time/time_zone.tsv +42 -0
  695. wetextprocessing-1.3.0/itn/spanish/data/time/time_zone_cased.tsv +42 -0
  696. wetextprocessing-1.3.0/itn/spanish/data/time/time_zone_ext.tsv +105 -0
  697. wetextprocessing-1.3.0/itn/spanish/data/whitelist.tsv +47 -0
  698. wetextprocessing-1.3.0/itn/spanish/inverse_normalizer.py +77 -0
  699. wetextprocessing-1.3.0/itn/spanish/rules/__init__.py +0 -0
  700. wetextprocessing-1.3.0/itn/spanish/rules/cardinal.py +98 -0
  701. wetextprocessing-1.3.0/itn/spanish/rules/date.py +83 -0
  702. wetextprocessing-1.3.0/itn/spanish/rules/decimal.py +64 -0
  703. wetextprocessing-1.3.0/itn/spanish/rules/electronic.py +94 -0
  704. wetextprocessing-1.3.0/itn/spanish/rules/fraction.py +77 -0
  705. wetextprocessing-1.3.0/itn/spanish/rules/measure.py +84 -0
  706. wetextprocessing-1.3.0/itn/spanish/rules/money.py +95 -0
  707. wetextprocessing-1.3.0/itn/spanish/rules/ordinal.py +53 -0
  708. wetextprocessing-1.3.0/itn/spanish/rules/punctuation.py +30 -0
  709. wetextprocessing-1.3.0/itn/spanish/rules/telephone.py +70 -0
  710. wetextprocessing-1.3.0/itn/spanish/rules/time.py +112 -0
  711. wetextprocessing-1.3.0/itn/spanish/rules/utils.py +102 -0
  712. wetextprocessing-1.3.0/itn/spanish/rules/whitelist.py +47 -0
  713. wetextprocessing-1.3.0/itn/spanish/rules/word.py +32 -0
  714. wetextprocessing-1.3.0/itn/spanish/test/__init__.py +0 -0
  715. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_cardinal.txt +51 -0
  716. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_cardinal_cased.txt +30 -0
  717. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_date.txt +8 -0
  718. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_date_cased.txt +8 -0
  719. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_decimal.txt +29 -0
  720. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_decimal_cased.txt +6 -0
  721. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_electronic.txt +16 -0
  722. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_electronic_cased.txt +5 -0
  723. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_fraction.txt +12 -0
  724. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_measure.txt +20 -0
  725. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_measure_cased.txt +11 -0
  726. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_money.txt +24 -0
  727. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_money_cased.txt +6 -0
  728. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_ordinal.txt +30 -0
  729. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_ordinal_cased.txt +11 -0
  730. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_telephone.txt +9 -0
  731. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_telephone_cased.txt +6 -0
  732. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_time.txt +25 -0
  733. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_time_cased.txt +9 -0
  734. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_whitelist.txt +5 -0
  735. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_word.txt +49 -0
  736. wetextprocessing-1.3.0/itn/spanish/test/data/test_cases_word_cased.txt +11 -0
  737. wetextprocessing-1.3.0/itn/spanish/test/normalizer_test.py +143 -0
  738. wetextprocessing-1.3.0/itn/spanish_english/__init__.py +0 -0
  739. wetextprocessing-1.3.0/itn/spanish_english/data/en_whitelist.tsv +416 -0
  740. wetextprocessing-1.3.0/itn/spanish_english/data/es_whitelist.tsv +16 -0
  741. wetextprocessing-1.3.0/itn/spanish_english/data/measurements.tsv +3 -0
  742. wetextprocessing-1.3.0/itn/spanish_english/inverse_normalizer.py +99 -0
  743. wetextprocessing-1.3.0/itn/spanish_english/rules/__init__.py +0 -0
  744. wetextprocessing-1.3.0/itn/spanish_english/rules/english.py +230 -0
  745. wetextprocessing-1.3.0/itn/spanish_english/rules/prefixed.py +34 -0
  746. wetextprocessing-1.3.0/itn/spanish_english/rules/time.py +39 -0
  747. wetextprocessing-1.3.0/itn/spanish_english/rules/whitelist.py +38 -0
  748. wetextprocessing-1.3.0/itn/spanish_english/test/__init__.py +0 -0
  749. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_cardinal.txt +80 -0
  750. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_cardinal_cased.txt +60 -0
  751. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_date.txt +44 -0
  752. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_date_cased.txt +20 -0
  753. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_decimal.txt +92 -0
  754. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_decimal_cased.txt +19 -0
  755. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_electronic.txt +41 -0
  756. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_electronic_cased.txt +14 -0
  757. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_fraction.txt +12 -0
  758. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_measure.txt +132 -0
  759. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_measure_cased.txt +22 -0
  760. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_money.txt +76 -0
  761. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_money_cased.txt +16 -0
  762. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_ordinal.txt +64 -0
  763. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_ordinal_cased.txt +21 -0
  764. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_telephone.txt +32 -0
  765. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_telephone_cased.txt +19 -0
  766. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_time.txt +54 -0
  767. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_time_cased.txt +19 -0
  768. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_whitelist.txt +18 -0
  769. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_word.txt +104 -0
  770. wetextprocessing-1.3.0/itn/spanish_english/test/data/test_cases_word_cased.txt +17 -0
  771. wetextprocessing-1.3.0/itn/spanish_english/test/normalizer_test.py +207 -0
  772. wetextprocessing-1.3.0/itn/swedish/__init__.py +0 -0
  773. wetextprocessing-1.3.0/itn/swedish/data/dates/era_words.tsv +4 -0
  774. wetextprocessing-1.3.0/itn/swedish/data/dates/month_abbr.tsv +17 -0
  775. wetextprocessing-1.3.0/itn/swedish/data/dates/months.tsv +12 -0
  776. wetextprocessing-1.3.0/itn/swedish/data/electronic/domain.tsv +10 -0
  777. wetextprocessing-1.3.0/itn/swedish/data/electronic/server_name.tsv +29 -0
  778. wetextprocessing-1.3.0/itn/swedish/data/electronic/symbols.tsv +20 -0
  779. wetextprocessing-1.3.0/itn/swedish/data/numbers/digit.tsv +9 -0
  780. wetextprocessing-1.3.0/itn/swedish/data/numbers/millions.tsv +88 -0
  781. wetextprocessing-1.3.0/itn/swedish/data/numbers/teen.tsv +10 -0
  782. wetextprocessing-1.3.0/itn/swedish/data/numbers/ties.tsv +8 -0
  783. wetextprocessing-1.3.0/itn/swedish/data/numbers/zero.tsv +1 -0
  784. wetextprocessing-1.3.0/itn/swedish/data/ordinals/digit.tsv +9 -0
  785. wetextprocessing-1.3.0/itn/swedish/data/ordinals/teen.tsv +10 -0
  786. wetextprocessing-1.3.0/itn/swedish/data/ordinals/ties.tsv +8 -0
  787. wetextprocessing-1.3.0/itn/swedish/data/time/suffix.tsv +2 -0
  788. wetextprocessing-1.3.0/itn/swedish/data/time/time_zone.tsv +7 -0
  789. wetextprocessing-1.3.0/itn/swedish/data/whitelist.tsv +4 -0
  790. wetextprocessing-1.3.0/itn/swedish/inverse_normalizer.py +67 -0
  791. wetextprocessing-1.3.0/itn/swedish/rules/__init__.py +0 -0
  792. wetextprocessing-1.3.0/itn/swedish/rules/cardinal.py +88 -0
  793. wetextprocessing-1.3.0/itn/swedish/rules/date.py +91 -0
  794. wetextprocessing-1.3.0/itn/swedish/rules/decimal.py +68 -0
  795. wetextprocessing-1.3.0/itn/swedish/rules/electronic.py +69 -0
  796. wetextprocessing-1.3.0/itn/swedish/rules/fraction.py +90 -0
  797. wetextprocessing-1.3.0/itn/swedish/rules/ordinal.py +85 -0
  798. wetextprocessing-1.3.0/itn/swedish/rules/punctuation.py +29 -0
  799. wetextprocessing-1.3.0/itn/swedish/rules/telephone.py +59 -0
  800. wetextprocessing-1.3.0/itn/swedish/rules/time.py +99 -0
  801. wetextprocessing-1.3.0/itn/swedish/rules/utils.py +55 -0
  802. wetextprocessing-1.3.0/itn/swedish/rules/whitelist.py +29 -0
  803. wetextprocessing-1.3.0/itn/swedish/rules/word.py +29 -0
  804. wetextprocessing-1.3.0/itn/swedish/test/__init__.py +0 -0
  805. wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_cardinal.txt +24 -0
  806. wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_date.txt +9 -0
  807. wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_decimal.txt +15 -0
  808. wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_electronic.txt +12 -0
  809. wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_fraction.txt +3 -0
  810. wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_ordinal.txt +7 -0
  811. wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_telephone.txt +2 -0
  812. wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_time.txt +14 -0
  813. wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_whitelist.txt +1 -0
  814. wetextprocessing-1.3.0/itn/swedish/test/data/test_cases_word.txt +48 -0
  815. wetextprocessing-1.3.0/itn/swedish/test/normalizer_test.py +122 -0
  816. wetextprocessing-1.3.0/itn/test/utils_test.py +17 -0
  817. wetextprocessing-1.3.0/itn/utils.py +40 -0
  818. wetextprocessing-1.3.0/itn/vietnamese/__init__.py +0 -0
  819. wetextprocessing-1.3.0/itn/vietnamese/data/currency.tsv +11 -0
  820. wetextprocessing-1.3.0/itn/vietnamese/data/electronic/domain.tsv +11 -0
  821. wetextprocessing-1.3.0/itn/vietnamese/data/electronic/server_name.tsv +16 -0
  822. wetextprocessing-1.3.0/itn/vietnamese/data/electronic/symbols.tsv +21 -0
  823. wetextprocessing-1.3.0/itn/vietnamese/data/electronic/url_symbols.tsv +8 -0
  824. wetextprocessing-1.3.0/itn/vietnamese/data/magnitudes.tsv +5 -0
  825. wetextprocessing-1.3.0/itn/vietnamese/data/math/symbols.tsv +10 -0
  826. wetextprocessing-1.3.0/itn/vietnamese/data/measurements.tsv +185 -0
  827. wetextprocessing-1.3.0/itn/vietnamese/data/months.tsv +14 -0
  828. wetextprocessing-1.3.0/itn/vietnamese/data/numbers/digit.tsv +10 -0
  829. wetextprocessing-1.3.0/itn/vietnamese/data/numbers/digit_special.tsv +3 -0
  830. wetextprocessing-1.3.0/itn/vietnamese/data/numbers/hundred.tsv +1 -0
  831. wetextprocessing-1.3.0/itn/vietnamese/data/numbers/teen.tsv +11 -0
  832. wetextprocessing-1.3.0/itn/vietnamese/data/numbers/thousands.tsv +6 -0
  833. wetextprocessing-1.3.0/itn/vietnamese/data/numbers/ties.tsv +9 -0
  834. wetextprocessing-1.3.0/itn/vietnamese/data/numbers/zero.tsv +1 -0
  835. wetextprocessing-1.3.0/itn/vietnamese/data/ordinals/digit.tsv +12 -0
  836. wetextprocessing-1.3.0/itn/vietnamese/data/time/hours.tsv +29 -0
  837. wetextprocessing-1.3.0/itn/vietnamese/data/time/hours_to.tsv +25 -0
  838. wetextprocessing-1.3.0/itn/vietnamese/data/time/hours_to_night.tsv +12 -0
  839. wetextprocessing-1.3.0/itn/vietnamese/data/time/minutes.tsv +118 -0
  840. wetextprocessing-1.3.0/itn/vietnamese/data/time/minutes_to.tsv +59 -0
  841. wetextprocessing-1.3.0/itn/vietnamese/data/time/time_suffix.tsv +8 -0
  842. wetextprocessing-1.3.0/itn/vietnamese/data/time/time_zone.tsv +7 -0
  843. wetextprocessing-1.3.0/itn/vietnamese/data/time/zero_prefix.tsv +3 -0
  844. wetextprocessing-1.3.0/itn/vietnamese/data/whitelist.tsv +0 -0
  845. wetextprocessing-1.3.0/itn/vietnamese/inverse_normalizer.py +73 -0
  846. wetextprocessing-1.3.0/itn/vietnamese/rules/__init__.py +0 -0
  847. wetextprocessing-1.3.0/itn/vietnamese/rules/cardinal.py +99 -0
  848. wetextprocessing-1.3.0/itn/vietnamese/rules/date.py +76 -0
  849. wetextprocessing-1.3.0/itn/vietnamese/rules/decimal.py +72 -0
  850. wetextprocessing-1.3.0/itn/vietnamese/rules/electronic.py +60 -0
  851. wetextprocessing-1.3.0/itn/vietnamese/rules/fraction.py +41 -0
  852. wetextprocessing-1.3.0/itn/vietnamese/rules/measure.py +60 -0
  853. wetextprocessing-1.3.0/itn/vietnamese/rules/money.py +54 -0
  854. wetextprocessing-1.3.0/itn/vietnamese/rules/ordinal.py +32 -0
  855. wetextprocessing-1.3.0/itn/vietnamese/rules/punctuation.py +29 -0
  856. wetextprocessing-1.3.0/itn/vietnamese/rules/telephone.py +57 -0
  857. wetextprocessing-1.3.0/itn/vietnamese/rules/time.py +83 -0
  858. wetextprocessing-1.3.0/itn/vietnamese/rules/whitelist.py +30 -0
  859. wetextprocessing-1.3.0/itn/vietnamese/rules/word.py +28 -0
  860. wetextprocessing-1.3.0/itn/vietnamese/test/__init__.py +0 -0
  861. wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_cardinal.txt +142 -0
  862. wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_date.txt +69 -0
  863. wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_decimal.txt +74 -0
  864. wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_electronic.txt +18 -0
  865. wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_fraction.txt +14 -0
  866. wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_measure.txt +35 -0
  867. wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_money.txt +15 -0
  868. wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_ordinal.txt +12 -0
  869. wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_telephone.txt +25 -0
  870. wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_time.txt +31 -0
  871. wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_whitelist.txt +1 -0
  872. wetextprocessing-1.3.0/itn/vietnamese/test/data/test_cases_word.txt +17 -0
  873. wetextprocessing-1.3.0/itn/vietnamese/test/normalizer_test.py +144 -0
  874. wetextprocessing-1.3.0/pyproject.toml +125 -0
  875. wetextprocessing-1.3.0/runtime/CMakeLists.txt +51 -0
  876. wetextprocessing-1.3.0/runtime/CMakePresets.json +69 -0
  877. wetextprocessing-1.3.0/runtime/android/.gitignore +15 -0
  878. wetextprocessing-1.3.0/runtime/android/app/build.gradle.kts +75 -0
  879. wetextprocessing-1.3.0/runtime/android/app/src/main/AndroidManifest.xml +21 -0
  880. wetextprocessing-1.3.0/runtime/android/app/src/main/assets/README.md +29 -0
  881. wetextprocessing-1.3.0/runtime/android/app/src/main/java/com/wenet/WeTextProcessing/MainActivity.kt +140 -0
  882. wetextprocessing-1.3.0/runtime/android/app/src/main/java/com/wenet/WeTextProcessing/Theme.kt +46 -0
  883. wetextprocessing-1.3.0/runtime/android/app/src/main/java/com/wenet/WeTextProcessing/WeTextProcessing.kt +17 -0
  884. wetextprocessing-1.3.0/runtime/android/app/src/main/jniLibs/.gitignore +2 -0
  885. wetextprocessing-1.3.0/runtime/android/app/src/main/res/drawable/ic_launcher_background.xml +21 -0
  886. wetextprocessing-1.3.0/runtime/android/app/src/main/res/drawable/ic_launcher_foreground.xml +30 -0
  887. wetextprocessing-1.3.0/runtime/android/app/src/main/res/values/themes.xml +5 -0
  888. wetextprocessing-1.3.0/runtime/android/app/src/main/res/values-night/themes.xml +3 -0
  889. wetextprocessing-1.3.0/runtime/android/build.gradle.kts +4 -0
  890. wetextprocessing-1.3.0/runtime/android/gradle/gradle-daemon-jvm.properties +12 -0
  891. wetextprocessing-1.3.0/runtime/android/gradle/libs.versions.toml +21 -0
  892. wetextprocessing-1.3.0/runtime/android/gradle/wrapper/gradle-wrapper.properties +5 -0
  893. wetextprocessing-1.3.0/runtime/android/gradle.properties +17 -0
  894. wetextprocessing-1.3.0/runtime/android/settings.gradle.kts +26 -0
  895. wetextprocessing-1.3.0/runtime/bin/CMakeLists.txt +3 -0
  896. wetextprocessing-1.3.0/runtime/bindings/android/CMakeLists.txt +17 -0
  897. wetextprocessing-1.3.0/runtime/bindings/android/wetextprocessing.cc +80 -0
  898. wetextprocessing-1.3.0/runtime/cmake/glog.cmake +15 -0
  899. wetextprocessing-1.3.0/runtime/cmake/openfst.cmake +27 -0
  900. wetextprocessing-1.3.0/runtime/processor/CMakeLists.txt +15 -0
  901. wetextprocessing-1.3.0/runtime/processor/wetext_processor.cc +87 -0
  902. wetextprocessing-1.3.0/runtime/processor/wetext_token_parser.cc +201 -0
  903. wetextprocessing-1.3.0/runtime/processor/wetext_token_parser.h +98 -0
  904. wetextprocessing-1.3.0/runtime/test/CMakeLists.txt +22 -0
  905. wetextprocessing-1.3.0/runtime/test/processor_test.cc +78 -0
  906. wetextprocessing-1.3.0/runtime/test/token_parser_test.cc +122 -0
  907. wetextprocessing-1.3.0/runtime/utils/wetext_flags.h +20 -0
  908. wetextprocessing-1.3.0/runtime/utils/wetext_log.h +20 -0
  909. wetextprocessing-1.3.0/tn/README.md +81 -0
  910. wetextprocessing-1.3.0/tn/__init__.py +3 -0
  911. wetextprocessing-1.3.0/tn/_version.py +24 -0
  912. wetextprocessing-1.3.0/tn/alignment.py +254 -0
  913. wetextprocessing-1.3.0/tn/arabic/__init__.py +0 -0
  914. wetextprocessing-1.3.0/tn/arabic/data/measure/measurements.tsv +14 -0
  915. wetextprocessing-1.3.0/tn/arabic/data/money/currency_major.tsv +24 -0
  916. wetextprocessing-1.3.0/tn/arabic/data/money/currency_minor_plural.tsv +4 -0
  917. wetextprocessing-1.3.0/tn/arabic/data/money/currency_minor_singular.tsv +3 -0
  918. wetextprocessing-1.3.0/tn/arabic/data/money/local_currency.tsv +20 -0
  919. wetextprocessing-1.3.0/tn/arabic/data/number/digit.tsv +9 -0
  920. wetextprocessing-1.3.0/tn/arabic/data/number/digit_100.tsv +7 -0
  921. wetextprocessing-1.3.0/tn/arabic/data/number/digit_1000.tsv +7 -0
  922. wetextprocessing-1.3.0/tn/arabic/data/number/flops.tsv +72 -0
  923. wetextprocessing-1.3.0/tn/arabic/data/number/fraction_dual.tsv +9 -0
  924. wetextprocessing-1.3.0/tn/arabic/data/number/fraction_plural.tsv +8 -0
  925. wetextprocessing-1.3.0/tn/arabic/data/number/fraction_singular.tsv +9 -0
  926. wetextprocessing-1.3.0/tn/arabic/data/number/quantities.tsv +6 -0
  927. wetextprocessing-1.3.0/tn/arabic/data/number/teens.tsv +9 -0
  928. wetextprocessing-1.3.0/tn/arabic/data/number/tens.tsv +10 -0
  929. wetextprocessing-1.3.0/tn/arabic/normalizer.py +61 -0
  930. wetextprocessing-1.3.0/tn/arabic/rules/__init__.py +0 -0
  931. wetextprocessing-1.3.0/tn/arabic/rules/cardinal.py +86 -0
  932. wetextprocessing-1.3.0/tn/arabic/rules/decimal.py +69 -0
  933. wetextprocessing-1.3.0/tn/arabic/rules/fraction.py +79 -0
  934. wetextprocessing-1.3.0/tn/arabic/rules/measure.py +80 -0
  935. wetextprocessing-1.3.0/tn/arabic/rules/money.py +131 -0
  936. wetextprocessing-1.3.0/tn/arabic/rules/punctuation.py +44 -0
  937. wetextprocessing-1.3.0/tn/arabic/rules/word.py +32 -0
  938. wetextprocessing-1.3.0/tn/arabic/test/__init__.py +0 -0
  939. wetextprocessing-1.3.0/tn/arabic/test/data/test_cases_cardinal.txt +27 -0
  940. wetextprocessing-1.3.0/tn/arabic/test/data/test_cases_decimal.txt +30 -0
  941. wetextprocessing-1.3.0/tn/arabic/test/data/test_cases_fraction.txt +20 -0
  942. wetextprocessing-1.3.0/tn/arabic/test/data/test_cases_measure.txt +14 -0
  943. wetextprocessing-1.3.0/tn/arabic/test/data/test_cases_money.txt +27 -0
  944. wetextprocessing-1.3.0/tn/arabic/test/data/test_cases_whitelist.txt +6 -0
  945. wetextprocessing-1.3.0/tn/arabic/test/normalizer_test.py +162 -0
  946. wetextprocessing-1.3.0/tn/armenian/__init__.py +0 -0
  947. wetextprocessing-1.3.0/tn/armenian/data/currency.tsv +52 -0
  948. wetextprocessing-1.3.0/tn/armenian/data/measurement_dates.tsv +8 -0
  949. wetextprocessing-1.3.0/tn/armenian/data/measurements.tsv +44 -0
  950. wetextprocessing-1.3.0/tn/armenian/data/numbers/digit.tsv +9 -0
  951. wetextprocessing-1.3.0/tn/armenian/data/numbers/quantities.tsv +3 -0
  952. wetextprocessing-1.3.0/tn/armenian/data/numbers/ties.tsv +8 -0
  953. wetextprocessing-1.3.0/tn/armenian/data/time/hours.tsv +26 -0
  954. wetextprocessing-1.3.0/tn/armenian/data/time/minutes.tsv +61 -0
  955. wetextprocessing-1.3.0/tn/armenian/data/whitelist.tsv +14 -0
  956. wetextprocessing-1.3.0/tn/armenian/normalizer.py +65 -0
  957. wetextprocessing-1.3.0/tn/armenian/rules/__init__.py +0 -0
  958. wetextprocessing-1.3.0/tn/armenian/rules/cardinal.py +75 -0
  959. wetextprocessing-1.3.0/tn/armenian/rules/decimal.py +63 -0
  960. wetextprocessing-1.3.0/tn/armenian/rules/fraction.py +36 -0
  961. wetextprocessing-1.3.0/tn/armenian/rules/measure.py +49 -0
  962. wetextprocessing-1.3.0/tn/armenian/rules/money.py +38 -0
  963. wetextprocessing-1.3.0/tn/armenian/rules/ordinal.py +59 -0
  964. wetextprocessing-1.3.0/tn/armenian/rules/punctuation.py +31 -0
  965. wetextprocessing-1.3.0/tn/armenian/rules/time.py +34 -0
  966. wetextprocessing-1.3.0/tn/armenian/rules/utils.py +60 -0
  967. wetextprocessing-1.3.0/tn/armenian/rules/whitelist.py +31 -0
  968. wetextprocessing-1.3.0/tn/armenian/rules/word.py +28 -0
  969. wetextprocessing-1.3.0/tn/armenian/test/__init__.py +0 -0
  970. wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_cardinal.txt +12 -0
  971. wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_decimal.txt +9 -0
  972. wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_fraction.txt +10 -0
  973. wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_measure.txt +9 -0
  974. wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_money.txt +10 -0
  975. wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_ordinal.txt +11 -0
  976. wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_time.txt +9 -0
  977. wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_whitelist.txt +14 -0
  978. wetextprocessing-1.3.0/tn/armenian/test/data/test_cases_word.txt +50 -0
  979. wetextprocessing-1.3.0/tn/armenian/test/normalizer_test.py +156 -0
  980. wetextprocessing-1.3.0/tn/cache.py +822 -0
  981. wetextprocessing-1.3.0/tn/chinese/__init__.py +0 -0
  982. wetextprocessing-1.3.0/tn/chinese/data/cardinal/phone.tsv +28 -0
  983. wetextprocessing-1.3.0/tn/chinese/data/math/operator_unambiguous.tsv +6 -0
  984. wetextprocessing-1.3.0/tn/chinese/normalizer.py +112 -0
  985. wetextprocessing-1.3.0/tn/chinese/rules/__init__.py +0 -0
  986. wetextprocessing-1.3.0/tn/chinese/rules/cardinal.py +94 -0
  987. wetextprocessing-1.3.0/tn/chinese/rules/date.py +72 -0
  988. wetextprocessing-1.3.0/tn/chinese/rules/fraction.py +41 -0
  989. wetextprocessing-1.3.0/tn/chinese/rules/math.py +44 -0
  990. wetextprocessing-1.3.0/tn/chinese/rules/measure.py +77 -0
  991. wetextprocessing-1.3.0/tn/chinese/rules/money.py +45 -0
  992. wetextprocessing-1.3.0/tn/chinese/rules/postprocessor.py +63 -0
  993. wetextprocessing-1.3.0/tn/chinese/rules/range.py +29 -0
  994. wetextprocessing-1.3.0/tn/chinese/rules/sport.py +49 -0
  995. wetextprocessing-1.3.0/tn/chinese/rules/time.py +62 -0
  996. wetextprocessing-1.3.0/tn/chinese/rules/whitelist.py +46 -0
  997. wetextprocessing-1.3.0/tn/chinese/test/alignment_test.py +154 -0
  998. wetextprocessing-1.3.0/tn/chinese/test/cache_safety_test.py +505 -0
  999. wetextprocessing-1.3.0/tn/chinese/test/cache_test.py +192 -0
  1000. wetextprocessing-1.3.0/tn/chinese/test/data/normalizer.txt +46 -0
  1001. wetextprocessing-1.3.0/tn/chinese/test/data/normalizer_tag_oov.txt +2 -0
  1002. wetextprocessing-1.3.0/tn/chinese/test/normalizer_test.py +94 -0
  1003. wetextprocessing-1.3.0/tn/chinese/test/postprocessor_test.py +45 -0
  1004. wetextprocessing-1.3.0/tn/chinese/test/processor_test.py +496 -0
  1005. wetextprocessing-1.3.0/tn/chinese/test/token_parser_test.py +110 -0
  1006. wetextprocessing-1.3.0/tn/cli.py +278 -0
  1007. wetextprocessing-1.3.0/tn/english/__init__.py +0 -0
  1008. wetextprocessing-1.3.0/tn/english/data/__init__.py +0 -0
  1009. wetextprocessing-1.3.0/tn/english/data/date/__init__.py +0 -0
  1010. wetextprocessing-1.3.0/tn/english/data/electronic/__init__.py +0 -0
  1011. wetextprocessing-1.3.0/tn/english/data/measure/unit_alternatives.tsv +42 -0
  1012. wetextprocessing-1.3.0/tn/english/data/money/__init__.py +0 -0
  1013. wetextprocessing-1.3.0/tn/english/data/number/fraction.tsv +18 -0
  1014. wetextprocessing-1.3.0/tn/english/data/number/zero.tsv +1 -0
  1015. wetextprocessing-1.3.0/tn/english/data/ordinal/__init__.py +0 -0
  1016. wetextprocessing-1.3.0/tn/english/data/ordinal/teen.tsv +1 -0
  1017. wetextprocessing-1.3.0/tn/english/data/roman/__init__.py +0 -0
  1018. wetextprocessing-1.3.0/tn/english/data/telephone/__init__.py +0 -0
  1019. wetextprocessing-1.3.0/tn/english/data/whitelist/__init__.py +0 -0
  1020. wetextprocessing-1.3.0/tn/english/data/whitelist/symbol.tsv +23 -0
  1021. wetextprocessing-1.3.0/tn/english/normalizer.py +98 -0
  1022. wetextprocessing-1.3.0/tn/english/rules/__init__.py +0 -0
  1023. wetextprocessing-1.3.0/tn/english/rules/cardinal.py +149 -0
  1024. wetextprocessing-1.3.0/tn/english/rules/date.py +270 -0
  1025. wetextprocessing-1.3.0/tn/english/rules/decimal.py +145 -0
  1026. wetextprocessing-1.3.0/tn/english/rules/electronic.py +189 -0
  1027. wetextprocessing-1.3.0/tn/english/rules/fraction.py +132 -0
  1028. wetextprocessing-1.3.0/tn/english/rules/measure.py +313 -0
  1029. wetextprocessing-1.3.0/tn/english/rules/money.py +118 -0
  1030. wetextprocessing-1.3.0/tn/english/rules/ordinal.py +75 -0
  1031. wetextprocessing-1.3.0/tn/english/rules/punctuation.py +75 -0
  1032. wetextprocessing-1.3.0/tn/english/rules/range.py +117 -0
  1033. wetextprocessing-1.3.0/tn/english/rules/roman.py +141 -0
  1034. wetextprocessing-1.3.0/tn/english/rules/serial.py +88 -0
  1035. wetextprocessing-1.3.0/tn/english/rules/telephone.py +129 -0
  1036. wetextprocessing-1.3.0/tn/english/rules/time.py +140 -0
  1037. wetextprocessing-1.3.0/tn/english/rules/whitelist.py +130 -0
  1038. wetextprocessing-1.3.0/tn/english/rules/word.py +61 -0
  1039. wetextprocessing-1.3.0/tn/english/test/__init__.py +0 -0
  1040. wetextprocessing-1.3.0/tn/english/test/data/date.txt +14 -0
  1041. wetextprocessing-1.3.0/tn/english/test/data/normalizer.txt +11 -0
  1042. wetextprocessing-1.3.0/tn/english/test/electronic_test.py +55 -0
  1043. wetextprocessing-1.3.0/tn/english/test/nbest_test.py +87 -0
  1044. wetextprocessing-1.3.0/tn/english/test/normalizer_test.py +206 -0
  1045. wetextprocessing-1.3.0/tn/french/__init__.py +0 -0
  1046. wetextprocessing-1.3.0/tn/french/data/dates/eras.tsv +8 -0
  1047. wetextprocessing-1.3.0/tn/french/data/dates/months.tsv +12 -0
  1048. wetextprocessing-1.3.0/tn/french/data/fractions/irregular_denominators.tsv +7 -0
  1049. wetextprocessing-1.3.0/tn/french/data/numbers/digit.tsv +9 -0
  1050. wetextprocessing-1.3.0/tn/french/data/numbers/eighty.tsv +1 -0
  1051. wetextprocessing-1.3.0/tn/french/data/numbers/ninety.tsv +1 -0
  1052. wetextprocessing-1.3.0/tn/french/data/numbers/quantities.tsv +10 -0
  1053. wetextprocessing-1.3.0/tn/french/data/numbers/seventy.tsv +1 -0
  1054. wetextprocessing-1.3.0/tn/french/data/numbers/teens.tsv +9 -0
  1055. wetextprocessing-1.3.0/tn/french/data/numbers/ten.tsv +1 -0
  1056. wetextprocessing-1.3.0/tn/french/data/numbers/tens_simple.tsv +5 -0
  1057. wetextprocessing-1.3.0/tn/french/data/numbers/zero.tsv +1 -0
  1058. wetextprocessing-1.3.0/tn/french/data/ordinals/irregular_numbers.tsv +30 -0
  1059. wetextprocessing-1.3.0/tn/french/data/ordinals/suffixes.tsv +6 -0
  1060. wetextprocessing-1.3.0/tn/french/data/punctuation/symbol.tsv +23 -0
  1061. wetextprocessing-1.3.0/tn/french/data/whitelist.tsv +13 -0
  1062. wetextprocessing-1.3.0/tn/french/normalizer.py +66 -0
  1063. wetextprocessing-1.3.0/tn/french/rules/__init__.py +0 -0
  1064. wetextprocessing-1.3.0/tn/french/rules/cardinal.py +110 -0
  1065. wetextprocessing-1.3.0/tn/french/rules/date.py +68 -0
  1066. wetextprocessing-1.3.0/tn/french/rules/decimal.py +62 -0
  1067. wetextprocessing-1.3.0/tn/french/rules/fraction.py +60 -0
  1068. wetextprocessing-1.3.0/tn/french/rules/ordinal.py +56 -0
  1069. wetextprocessing-1.3.0/tn/french/rules/punctuation.py +43 -0
  1070. wetextprocessing-1.3.0/tn/french/rules/whitelist.py +29 -0
  1071. wetextprocessing-1.3.0/tn/french/rules/word.py +29 -0
  1072. wetextprocessing-1.3.0/tn/french/test/__init__.py +0 -0
  1073. wetextprocessing-1.3.0/tn/french/test/data/test_cases_cardinal.txt +14 -0
  1074. wetextprocessing-1.3.0/tn/french/test/data/test_cases_date.txt +13 -0
  1075. wetextprocessing-1.3.0/tn/french/test/data/test_cases_decimal.txt +7 -0
  1076. wetextprocessing-1.3.0/tn/french/test/data/test_cases_fraction.txt +12 -0
  1077. wetextprocessing-1.3.0/tn/french/test/data/test_cases_ordinal.txt +15 -0
  1078. wetextprocessing-1.3.0/tn/french/test/data/test_cases_whitelist.txt +7 -0
  1079. wetextprocessing-1.3.0/tn/french/test/data/test_cases_word.txt +49 -0
  1080. wetextprocessing-1.3.0/tn/french/test/normalizer_test.py +157 -0
  1081. wetextprocessing-1.3.0/tn/german/__init__.py +0 -0
  1082. wetextprocessing-1.3.0/tn/german/data/__init__.py +13 -0
  1083. wetextprocessing-1.3.0/tn/german/data/electronic/__init__.py +13 -0
  1084. wetextprocessing-1.3.0/tn/german/data/electronic/domain.tsv +844 -0
  1085. wetextprocessing-1.3.0/tn/german/data/electronic/server_name.tsv +17 -0
  1086. wetextprocessing-1.3.0/tn/german/data/electronic/symbols.tsv +20 -0
  1087. wetextprocessing-1.3.0/tn/german/data/fractions.tsv +30 -0
  1088. wetextprocessing-1.3.0/tn/german/data/measure/__init__.py +13 -0
  1089. wetextprocessing-1.3.0/tn/german/data/measure/measurements.tsv +82 -0
  1090. wetextprocessing-1.3.0/tn/german/data/measure/suppletive.tsv +10 -0
  1091. wetextprocessing-1.3.0/tn/german/data/money/__init__.py +13 -0
  1092. wetextprocessing-1.3.0/tn/german/data/money/currency.tsv +25 -0
  1093. wetextprocessing-1.3.0/tn/german/data/money/currency_minor_plural.tsv +3 -0
  1094. wetextprocessing-1.3.0/tn/german/data/money/currency_minor_singular.tsv +3 -0
  1095. wetextprocessing-1.3.0/tn/german/data/months/__init__.py +13 -0
  1096. wetextprocessing-1.3.0/tn/german/data/months/abbr_to_name.tsv +13 -0
  1097. wetextprocessing-1.3.0/tn/german/data/months/numbers.tsv +24 -0
  1098. wetextprocessing-1.3.0/tn/german/data/numbers/__init__.py +13 -0
  1099. wetextprocessing-1.3.0/tn/german/data/numbers/digit.tsv +8 -0
  1100. wetextprocessing-1.3.0/tn/german/data/numbers/ones.tsv +3 -0
  1101. wetextprocessing-1.3.0/tn/german/data/numbers/quantities.tsv +12 -0
  1102. wetextprocessing-1.3.0/tn/german/data/numbers/teen.tsv +10 -0
  1103. wetextprocessing-1.3.0/tn/german/data/numbers/ties.tsv +8 -0
  1104. wetextprocessing-1.3.0/tn/german/data/numbers/zero.tsv +1 -0
  1105. wetextprocessing-1.3.0/tn/german/data/ordinals/__init__.py +13 -0
  1106. wetextprocessing-1.3.0/tn/german/data/ordinals/digit.tsv +9 -0
  1107. wetextprocessing-1.3.0/tn/german/data/ordinals/thousands.tsv +4 -0
  1108. wetextprocessing-1.3.0/tn/german/data/ordinals/ties.tsv +8 -0
  1109. wetextprocessing-1.3.0/tn/german/data/time/__init__.py +13 -0
  1110. wetextprocessing-1.3.0/tn/german/data/time/hour_to.tsv +12 -0
  1111. wetextprocessing-1.3.0/tn/german/data/time/hour_to_night.tsv +13 -0
  1112. wetextprocessing-1.3.0/tn/german/data/time/minute_to.tsv +59 -0
  1113. wetextprocessing-1.3.0/tn/german/data/time/time_zone.tsv +7 -0
  1114. wetextprocessing-1.3.0/tn/german/data/whitelist.tsv +21 -0
  1115. wetextprocessing-1.3.0/tn/german/normalizer.py +77 -0
  1116. wetextprocessing-1.3.0/tn/german/rules/__init__.py +0 -0
  1117. wetextprocessing-1.3.0/tn/german/rules/cardinal.py +120 -0
  1118. wetextprocessing-1.3.0/tn/german/rules/date.py +98 -0
  1119. wetextprocessing-1.3.0/tn/german/rules/decimal.py +71 -0
  1120. wetextprocessing-1.3.0/tn/german/rules/electronic.py +67 -0
  1121. wetextprocessing-1.3.0/tn/german/rules/fraction.py +65 -0
  1122. wetextprocessing-1.3.0/tn/german/rules/measure.py +117 -0
  1123. wetextprocessing-1.3.0/tn/german/rules/money.py +144 -0
  1124. wetextprocessing-1.3.0/tn/german/rules/ordinal.py +46 -0
  1125. wetextprocessing-1.3.0/tn/german/rules/punctuation.py +39 -0
  1126. wetextprocessing-1.3.0/tn/german/rules/telephone.py +48 -0
  1127. wetextprocessing-1.3.0/tn/german/rules/time.py +123 -0
  1128. wetextprocessing-1.3.0/tn/german/rules/whitelist.py +34 -0
  1129. wetextprocessing-1.3.0/tn/german/rules/word.py +34 -0
  1130. wetextprocessing-1.3.0/tn/german/test/__init__.py +0 -0
  1131. wetextprocessing-1.3.0/tn/german/test/calendar_test.py +168 -0
  1132. wetextprocessing-1.3.0/tn/german/test/data/test_cases_cardinal.txt +50 -0
  1133. wetextprocessing-1.3.0/tn/german/test/data/test_cases_date.txt +15 -0
  1134. wetextprocessing-1.3.0/tn/german/test/data/test_cases_decimal.txt +10 -0
  1135. wetextprocessing-1.3.0/tn/german/test/data/test_cases_electronic.txt +19 -0
  1136. wetextprocessing-1.3.0/tn/german/test/data/test_cases_fraction.txt +34 -0
  1137. wetextprocessing-1.3.0/tn/german/test/data/test_cases_measure.txt +29 -0
  1138. wetextprocessing-1.3.0/tn/german/test/data/test_cases_money.txt +21 -0
  1139. wetextprocessing-1.3.0/tn/german/test/data/test_cases_normalize_with_audio.txt +131 -0
  1140. wetextprocessing-1.3.0/tn/german/test/data/test_cases_ordinal.txt +19 -0
  1141. wetextprocessing-1.3.0/tn/german/test/data/test_cases_telephone.txt +4 -0
  1142. wetextprocessing-1.3.0/tn/german/test/data/test_cases_time.txt +28 -0
  1143. wetextprocessing-1.3.0/tn/german/test/data/test_cases_whitelist.txt +6 -0
  1144. wetextprocessing-1.3.0/tn/german/test/data/test_cases_word.txt +49 -0
  1145. wetextprocessing-1.3.0/tn/german/test/financial_test.py +167 -0
  1146. wetextprocessing-1.3.0/tn/german/test/normalizer_test.py +136 -0
  1147. wetextprocessing-1.3.0/tn/german/test/numeric_test.py +185 -0
  1148. wetextprocessing-1.3.0/tn/german/test/text_test.py +235 -0
  1149. wetextprocessing-1.3.0/tn/hebrew/__init__.py +0 -0
  1150. wetextprocessing-1.3.0/tn/hebrew/data/measurements.tsv +29 -0
  1151. wetextprocessing-1.3.0/tn/hebrew/data/prefix.tsv +17 -0
  1152. wetextprocessing-1.3.0/tn/hebrew/data/spaced_measurements.tsv +16 -0
  1153. wetextprocessing-1.3.0/tn/hebrew/data/whitelist.tsv +9 -0
  1154. wetextprocessing-1.3.0/tn/hebrew/normalizer.py +58 -0
  1155. wetextprocessing-1.3.0/tn/hebrew/rules/__init__.py +0 -0
  1156. wetextprocessing-1.3.0/tn/hebrew/rules/cardinal.py +43 -0
  1157. wetextprocessing-1.3.0/tn/hebrew/rules/date.py +84 -0
  1158. wetextprocessing-1.3.0/tn/hebrew/rules/decimal.py +71 -0
  1159. wetextprocessing-1.3.0/tn/hebrew/rules/measure.py +56 -0
  1160. wetextprocessing-1.3.0/tn/hebrew/rules/numbers.py +135 -0
  1161. wetextprocessing-1.3.0/tn/hebrew/rules/punctuation.py +29 -0
  1162. wetextprocessing-1.3.0/tn/hebrew/rules/time.py +94 -0
  1163. wetextprocessing-1.3.0/tn/hebrew/rules/utils.py +59 -0
  1164. wetextprocessing-1.3.0/tn/hebrew/rules/whitelist.py +26 -0
  1165. wetextprocessing-1.3.0/tn/hebrew/rules/word.py +27 -0
  1166. wetextprocessing-1.3.0/tn/hebrew/test/__init__.py +0 -0
  1167. wetextprocessing-1.3.0/tn/hebrew/test/normalizer_test.py +245 -0
  1168. wetextprocessing-1.3.0/tn/hindi/__init__.py +0 -0
  1169. wetextprocessing-1.3.0/tn/hindi/data/address/cities.tsv +36 -0
  1170. wetextprocessing-1.3.0/tn/hindi/data/address/context.tsv +47 -0
  1171. wetextprocessing-1.3.0/tn/hindi/data/address/letters.tsv +26 -0
  1172. wetextprocessing-1.3.0/tn/hindi/data/address/special_characters.tsv +2 -0
  1173. wetextprocessing-1.3.0/tn/hindi/data/address/states.tsv +36 -0
  1174. wetextprocessing-1.3.0/tn/hindi/data/date/days.tsv +62 -0
  1175. wetextprocessing-1.3.0/tn/hindi/data/date/months.tsv +24 -0
  1176. wetextprocessing-1.3.0/tn/hindi/data/date/prefixes.tsv +4 -0
  1177. wetextprocessing-1.3.0/tn/hindi/data/date/suffixes.tsv +10 -0
  1178. wetextprocessing-1.3.0/tn/hindi/data/date/year_suffix.tsv +2 -0
  1179. wetextprocessing-1.3.0/tn/hindi/data/electronic/domain.tsv +24 -0
  1180. wetextprocessing-1.3.0/tn/hindi/data/electronic/elements.tsv +132 -0
  1181. wetextprocessing-1.3.0/tn/hindi/data/electronic/file_extensions.tsv +34 -0
  1182. wetextprocessing-1.3.0/tn/hindi/data/electronic/protocols.tsv +5 -0
  1183. wetextprocessing-1.3.0/tn/hindi/data/electronic/subscript_digit.tsv +10 -0
  1184. wetextprocessing-1.3.0/tn/hindi/data/electronic/symbol_classes.tsv +16 -0
  1185. wetextprocessing-1.3.0/tn/hindi/data/electronic/symbols.tsv +36 -0
  1186. wetextprocessing-1.3.0/tn/hindi/data/fraction/common_fractions.tsv +10 -0
  1187. wetextprocessing-1.3.0/tn/hindi/data/measure/quarterly_units_list.tsv +5 -0
  1188. wetextprocessing-1.3.0/tn/hindi/data/measure/quarterly_units_map.tsv +10 -0
  1189. wetextprocessing-1.3.0/tn/hindi/data/measure/unit.tsv +146 -0
  1190. wetextprocessing-1.3.0/tn/hindi/data/measure/unit_year_formal.tsv +1 -0
  1191. wetextprocessing-1.3.0/tn/hindi/data/money/currency.tsv +9 -0
  1192. wetextprocessing-1.3.0/tn/hindi/data/money/currency_singular.tsv +9 -0
  1193. wetextprocessing-1.3.0/tn/hindi/data/money/major_minor_currencies.tsv +10 -0
  1194. wetextprocessing-1.3.0/tn/hindi/data/numbers/digit.tsv +18 -0
  1195. wetextprocessing-1.3.0/tn/hindi/data/numbers/hundred.tsv +1 -0
  1196. wetextprocessing-1.3.0/tn/hindi/data/numbers/teens_and_ties.tsv +90 -0
  1197. wetextprocessing-1.3.0/tn/hindi/data/numbers/teens_and_ties_en.tsv +90 -0
  1198. wetextprocessing-1.3.0/tn/hindi/data/numbers/thousands.tsv +8 -0
  1199. wetextprocessing-1.3.0/tn/hindi/data/numbers/zero.tsv +2 -0
  1200. wetextprocessing-1.3.0/tn/hindi/data/ordinal/en_to_hi_digit.tsv +10 -0
  1201. wetextprocessing-1.3.0/tn/hindi/data/ordinal/exceptions.tsv +50 -0
  1202. wetextprocessing-1.3.0/tn/hindi/data/ordinal/suffixes.tsv +3 -0
  1203. wetextprocessing-1.3.0/tn/hindi/data/ordinal/suffixes_map.tsv +1 -0
  1204. wetextprocessing-1.3.0/tn/hindi/data/roman/roman_ordinal_exceptions.tsv +10 -0
  1205. wetextprocessing-1.3.0/tn/hindi/data/roman/roman_to_spoken.tsv +100 -0
  1206. wetextprocessing-1.3.0/tn/hindi/data/serial/chars.tsv +68 -0
  1207. wetextprocessing-1.3.0/tn/hindi/data/serial/power_special.tsv +4 -0
  1208. wetextprocessing-1.3.0/tn/hindi/data/serial/special_symbols.tsv +4 -0
  1209. wetextprocessing-1.3.0/tn/hindi/data/telephone/credit_context.tsv +3 -0
  1210. wetextprocessing-1.3.0/tn/hindi/data/telephone/landline_context.tsv +5 -0
  1211. wetextprocessing-1.3.0/tn/hindi/data/telephone/mobile_context.tsv +4 -0
  1212. wetextprocessing-1.3.0/tn/hindi/data/telephone/number.tsv +10 -0
  1213. wetextprocessing-1.3.0/tn/hindi/data/telephone/pincode_context.tsv +4 -0
  1214. wetextprocessing-1.3.0/tn/hindi/data/time/hours.tsv +50 -0
  1215. wetextprocessing-1.3.0/tn/hindi/data/time/minutes.tsv +120 -0
  1216. wetextprocessing-1.3.0/tn/hindi/data/time/seconds.tsv +120 -0
  1217. wetextprocessing-1.3.0/tn/hindi/data/whitelist/abbreviations.tsv +7 -0
  1218. wetextprocessing-1.3.0/tn/hindi/data/whitelist/paune_mappings.tsv +100 -0
  1219. wetextprocessing-1.3.0/tn/hindi/normalizer.py +95 -0
  1220. wetextprocessing-1.3.0/tn/hindi/rules/__init__.py +0 -0
  1221. wetextprocessing-1.3.0/tn/hindi/rules/address.py +97 -0
  1222. wetextprocessing-1.3.0/tn/hindi/rules/cardinal.py +117 -0
  1223. wetextprocessing-1.3.0/tn/hindi/rules/date.py +135 -0
  1224. wetextprocessing-1.3.0/tn/hindi/rules/decimal.py +42 -0
  1225. wetextprocessing-1.3.0/tn/hindi/rules/electronic.py +145 -0
  1226. wetextprocessing-1.3.0/tn/hindi/rules/fraction.py +62 -0
  1227. wetextprocessing-1.3.0/tn/hindi/rules/measure.py +110 -0
  1228. wetextprocessing-1.3.0/tn/hindi/rules/money.py +104 -0
  1229. wetextprocessing-1.3.0/tn/hindi/rules/ordinal.py +37 -0
  1230. wetextprocessing-1.3.0/tn/hindi/rules/punctuation.py +50 -0
  1231. wetextprocessing-1.3.0/tn/hindi/rules/roman.py +70 -0
  1232. wetextprocessing-1.3.0/tn/hindi/rules/serial.py +82 -0
  1233. wetextprocessing-1.3.0/tn/hindi/rules/telephone.py +98 -0
  1234. wetextprocessing-1.3.0/tn/hindi/rules/time.py +62 -0
  1235. wetextprocessing-1.3.0/tn/hindi/rules/utils.py +54 -0
  1236. wetextprocessing-1.3.0/tn/hindi/rules/whitelist.py +37 -0
  1237. wetextprocessing-1.3.0/tn/hindi/rules/word.py +31 -0
  1238. wetextprocessing-1.3.0/tn/hindi/test/__init__.py +0 -0
  1239. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_address.txt +52 -0
  1240. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_cardinal.txt +161 -0
  1241. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_date.txt +39 -0
  1242. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_decimal.txt +24 -0
  1243. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_electronic.txt +65 -0
  1244. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_fraction.txt +27 -0
  1245. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_measure.txt +69 -0
  1246. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_money.txt +144 -0
  1247. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_ordinal.txt +72 -0
  1248. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_roman.txt +23 -0
  1249. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_serial.txt +31 -0
  1250. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_telephone.txt +25 -0
  1251. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_time.txt +18 -0
  1252. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_whitelist.txt +14 -0
  1253. wetextprocessing-1.3.0/tn/hindi/test/data/test_cases_word.txt +20 -0
  1254. wetextprocessing-1.3.0/tn/hindi/test/normalizer_test.py +137 -0
  1255. wetextprocessing-1.3.0/tn/hungarian/__init__.py +0 -0
  1256. wetextprocessing-1.3.0/tn/hungarian/data/dates/days.tsv +31 -0
  1257. wetextprocessing-1.3.0/tn/hungarian/data/dates/month_abbr.tsv +12 -0
  1258. wetextprocessing-1.3.0/tn/hungarian/data/dates/months.tsv +12 -0
  1259. wetextprocessing-1.3.0/tn/hungarian/data/dates/months_roman.tsv +12 -0
  1260. wetextprocessing-1.3.0/tn/hungarian/data/electronic/domain.tsv +15 -0
  1261. wetextprocessing-1.3.0/tn/hungarian/data/electronic/server_name.tsv +15 -0
  1262. wetextprocessing-1.3.0/tn/hungarian/data/electronic/symbols.tsv +23 -0
  1263. wetextprocessing-1.3.0/tn/hungarian/data/fractions/fraction_symbols.tsv +19 -0
  1264. wetextprocessing-1.3.0/tn/hungarian/data/inflection/endings.tsv +51 -0
  1265. wetextprocessing-1.3.0/tn/hungarian/data/inflection/plural_endings.tsv +39 -0
  1266. wetextprocessing-1.3.0/tn/hungarian/data/inflection/word_endings.tsv +1 -0
  1267. wetextprocessing-1.3.0/tn/hungarian/data/measures/measurements.tsv +52 -0
  1268. wetextprocessing-1.3.0/tn/hungarian/data/money/alphabet.tsv +44 -0
  1269. wetextprocessing-1.3.0/tn/hungarian/data/money/currency.tsv +11 -0
  1270. wetextprocessing-1.3.0/tn/hungarian/data/money/currency_minor.tsv +11 -0
  1271. wetextprocessing-1.3.0/tn/hungarian/data/number/digit.tsv +9 -0
  1272. wetextprocessing-1.3.0/tn/hungarian/data/number/digit_inline.tsv +9 -0
  1273. wetextprocessing-1.3.0/tn/hungarian/data/number/quantities.tsv +21 -0
  1274. wetextprocessing-1.3.0/tn/hungarian/data/number/tens.tsv +9 -0
  1275. wetextprocessing-1.3.0/tn/hungarian/data/number/tens_inline.tsv +9 -0
  1276. wetextprocessing-1.3.0/tn/hungarian/data/number/zero.tsv +1 -0
  1277. wetextprocessing-1.3.0/tn/hungarian/data/ordinals/endings.tsv +23 -0
  1278. wetextprocessing-1.3.0/tn/hungarian/data/ordinals/exceptional.tsv +4 -0
  1279. wetextprocessing-1.3.0/tn/hungarian/data/telephone/area_codes.tsv +71 -0
  1280. wetextprocessing-1.3.0/tn/hungarian/data/telephone/country_codes.tsv +213 -0
  1281. wetextprocessing-1.3.0/tn/hungarian/data/telephone/telephone_abbr.tsv +3 -0
  1282. wetextprocessing-1.3.0/tn/hungarian/data/time/time_zone.tsv +9 -0
  1283. wetextprocessing-1.3.0/tn/hungarian/data/whitelist.tsv +24 -0
  1284. wetextprocessing-1.3.0/tn/hungarian/data/whitelist_inflect.tsv +39 -0
  1285. wetextprocessing-1.3.0/tn/hungarian/data/whitelist_inflect_sg.tsv +14 -0
  1286. wetextprocessing-1.3.0/tn/hungarian/normalizer.py +80 -0
  1287. wetextprocessing-1.3.0/tn/hungarian/rules/__init__.py +0 -0
  1288. wetextprocessing-1.3.0/tn/hungarian/rules/cardinal.py +98 -0
  1289. wetextprocessing-1.3.0/tn/hungarian/rules/date.py +104 -0
  1290. wetextprocessing-1.3.0/tn/hungarian/rules/decimal.py +85 -0
  1291. wetextprocessing-1.3.0/tn/hungarian/rules/electronic.py +69 -0
  1292. wetextprocessing-1.3.0/tn/hungarian/rules/fraction.py +55 -0
  1293. wetextprocessing-1.3.0/tn/hungarian/rules/measure.py +58 -0
  1294. wetextprocessing-1.3.0/tn/hungarian/rules/money.py +81 -0
  1295. wetextprocessing-1.3.0/tn/hungarian/rules/ordinal.py +41 -0
  1296. wetextprocessing-1.3.0/tn/hungarian/rules/punctuation.py +46 -0
  1297. wetextprocessing-1.3.0/tn/hungarian/rules/telephone.py +71 -0
  1298. wetextprocessing-1.3.0/tn/hungarian/rules/time.py +106 -0
  1299. wetextprocessing-1.3.0/tn/hungarian/rules/utils.py +156 -0
  1300. wetextprocessing-1.3.0/tn/hungarian/rules/whitelist.py +32 -0
  1301. wetextprocessing-1.3.0/tn/hungarian/rules/word.py +38 -0
  1302. wetextprocessing-1.3.0/tn/hungarian/test/__init__.py +0 -0
  1303. wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_cardinal.txt +69 -0
  1304. wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_date.txt +17 -0
  1305. wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_decimal.txt +6 -0
  1306. wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_electronic.txt +8 -0
  1307. wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_fraction.txt +4 -0
  1308. wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_measure.txt +6 -0
  1309. wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_money.txt +9 -0
  1310. wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_ordinal.txt +5 -0
  1311. wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_telephone.txt +9 -0
  1312. wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_time.txt +10 -0
  1313. wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_whitelist.txt +10 -0
  1314. wetextprocessing-1.3.0/tn/hungarian/test/data/test_cases_word.txt +52 -0
  1315. wetextprocessing-1.3.0/tn/hungarian/test/normalizer_test.py +210 -0
  1316. wetextprocessing-1.3.0/tn/italian/__init__.py +0 -0
  1317. wetextprocessing-1.3.0/tn/italian/data/electronic/domain.tsv +14 -0
  1318. wetextprocessing-1.3.0/tn/italian/data/electronic/server_name.tsv +11 -0
  1319. wetextprocessing-1.3.0/tn/italian/data/electronic/symbols.tsv +22 -0
  1320. wetextprocessing-1.3.0/tn/italian/data/measure/measurements.tsv +64 -0
  1321. wetextprocessing-1.3.0/tn/italian/data/measure/suppletive.tsv +11 -0
  1322. wetextprocessing-1.3.0/tn/italian/data/money/currency_major.tsv +11 -0
  1323. wetextprocessing-1.3.0/tn/italian/data/money/currency_minor.tsv +11 -0
  1324. wetextprocessing-1.3.0/tn/italian/data/money/currency_plural_fem.tsv +3 -0
  1325. wetextprocessing-1.3.0/tn/italian/data/money/currency_plural_masc.tsv +11 -0
  1326. wetextprocessing-1.3.0/tn/italian/data/numbers/digit.tsv +9 -0
  1327. wetextprocessing-1.3.0/tn/italian/data/numbers/hundreds.tsv +8 -0
  1328. wetextprocessing-1.3.0/tn/italian/data/numbers/quantities.tsv +2 -0
  1329. wetextprocessing-1.3.0/tn/italian/data/numbers/teen.tsv +10 -0
  1330. wetextprocessing-1.3.0/tn/italian/data/numbers/tens.tsv +8 -0
  1331. wetextprocessing-1.3.0/tn/italian/data/numbers/tens_eight.tsv +8 -0
  1332. wetextprocessing-1.3.0/tn/italian/data/numbers/tens_one.tsv +8 -0
  1333. wetextprocessing-1.3.0/tn/italian/data/numbers/zero.tsv +1 -0
  1334. wetextprocessing-1.3.0/tn/italian/data/whitelist/whitelist.tsv +65 -0
  1335. wetextprocessing-1.3.0/tn/italian/normalizer.py +60 -0
  1336. wetextprocessing-1.3.0/tn/italian/rules/__init__.py +0 -0
  1337. wetextprocessing-1.3.0/tn/italian/rules/cardinal.py +99 -0
  1338. wetextprocessing-1.3.0/tn/italian/rules/decimal.py +82 -0
  1339. wetextprocessing-1.3.0/tn/italian/rules/electronic.py +80 -0
  1340. wetextprocessing-1.3.0/tn/italian/rules/measure.py +73 -0
  1341. wetextprocessing-1.3.0/tn/italian/rules/money.py +92 -0
  1342. wetextprocessing-1.3.0/tn/italian/rules/punctuation.py +30 -0
  1343. wetextprocessing-1.3.0/tn/italian/rules/time.py +61 -0
  1344. wetextprocessing-1.3.0/tn/italian/rules/utils.py +74 -0
  1345. wetextprocessing-1.3.0/tn/italian/rules/whitelist.py +27 -0
  1346. wetextprocessing-1.3.0/tn/italian/rules/word.py +28 -0
  1347. wetextprocessing-1.3.0/tn/italian/test/__init__.py +0 -0
  1348. wetextprocessing-1.3.0/tn/italian/test/data/test_cases_cardinal.txt +13 -0
  1349. wetextprocessing-1.3.0/tn/italian/test/data/test_cases_decimal.txt +4 -0
  1350. wetextprocessing-1.3.0/tn/italian/test/data/test_cases_electronic.txt +13 -0
  1351. wetextprocessing-1.3.0/tn/italian/test/data/test_cases_measure.txt +7 -0
  1352. wetextprocessing-1.3.0/tn/italian/test/data/test_cases_money.txt +5 -0
  1353. wetextprocessing-1.3.0/tn/italian/test/data/test_cases_time.txt +6 -0
  1354. wetextprocessing-1.3.0/tn/italian/test/data/test_cases_whitelist.txt +4 -0
  1355. wetextprocessing-1.3.0/tn/italian/test/normalizer_test.py +209 -0
  1356. wetextprocessing-1.3.0/tn/japanese/__init__.py +0 -0
  1357. wetextprocessing-1.3.0/tn/japanese/data/default/blacklist.tsv +0 -0
  1358. wetextprocessing-1.3.0/tn/japanese/normalizer.py +114 -0
  1359. wetextprocessing-1.3.0/tn/japanese/rules/cardinal.py +113 -0
  1360. wetextprocessing-1.3.0/tn/japanese/rules/date.py +82 -0
  1361. wetextprocessing-1.3.0/tn/japanese/rules/fraction.py +43 -0
  1362. wetextprocessing-1.3.0/tn/japanese/rules/math.py +45 -0
  1363. wetextprocessing-1.3.0/tn/japanese/rules/measure.py +54 -0
  1364. wetextprocessing-1.3.0/tn/japanese/rules/money.py +46 -0
  1365. wetextprocessing-1.3.0/tn/japanese/rules/postprocessor.py +50 -0
  1366. wetextprocessing-1.3.0/tn/japanese/rules/preprocessor.py +31 -0
  1367. wetextprocessing-1.3.0/tn/japanese/rules/range.py +32 -0
  1368. wetextprocessing-1.3.0/tn/japanese/rules/sport.py +58 -0
  1369. wetextprocessing-1.3.0/tn/japanese/rules/time.py +65 -0
  1370. wetextprocessing-1.3.0/tn/japanese/rules/transliteration.py +40 -0
  1371. wetextprocessing-1.3.0/tn/japanese/rules/whitelist.py +40 -0
  1372. wetextprocessing-1.3.0/tn/japanese/test/__init__.py +0 -0
  1373. wetextprocessing-1.3.0/tn/japanese/test/normalizer_test.py +145 -0
  1374. wetextprocessing-1.3.0/tn/japanese/test/utils.py +32 -0
  1375. wetextprocessing-1.3.0/tn/kinyarwanda/__init__.py +0 -0
  1376. wetextprocessing-1.3.0/tn/kinyarwanda/data/numbers/digit.tsv +9 -0
  1377. wetextprocessing-1.3.0/tn/kinyarwanda/data/numbers/digit_bi.tsv +9 -0
  1378. wetextprocessing-1.3.0/tn/kinyarwanda/data/numbers/digit_e.tsv +9 -0
  1379. wetextprocessing-1.3.0/tn/kinyarwanda/data/numbers/hundreds.tsv +9 -0
  1380. wetextprocessing-1.3.0/tn/kinyarwanda/data/numbers/tens.tsv +8 -0
  1381. wetextprocessing-1.3.0/tn/kinyarwanda/data/numbers/zero.tsv +1 -0
  1382. wetextprocessing-1.3.0/tn/kinyarwanda/data/time/hours.tsv +12 -0
  1383. wetextprocessing-1.3.0/tn/kinyarwanda/data/time/minutes.tsv +60 -0
  1384. wetextprocessing-1.3.0/tn/kinyarwanda/data/whitelist.tsv +175 -0
  1385. wetextprocessing-1.3.0/tn/kinyarwanda/normalizer.py +56 -0
  1386. wetextprocessing-1.3.0/tn/kinyarwanda/rules/__init__.py +0 -0
  1387. wetextprocessing-1.3.0/tn/kinyarwanda/rules/cardinal.py +117 -0
  1388. wetextprocessing-1.3.0/tn/kinyarwanda/rules/punctuation.py +31 -0
  1389. wetextprocessing-1.3.0/tn/kinyarwanda/rules/time.py +40 -0
  1390. wetextprocessing-1.3.0/tn/kinyarwanda/rules/utils.py +43 -0
  1391. wetextprocessing-1.3.0/tn/kinyarwanda/rules/whitelist.py +28 -0
  1392. wetextprocessing-1.3.0/tn/kinyarwanda/rules/word.py +29 -0
  1393. wetextprocessing-1.3.0/tn/kinyarwanda/test/__init__.py +0 -0
  1394. wetextprocessing-1.3.0/tn/kinyarwanda/test/data/test_cases_cardinal.txt +57 -0
  1395. wetextprocessing-1.3.0/tn/kinyarwanda/test/data/test_cases_time.txt +14 -0
  1396. wetextprocessing-1.3.0/tn/kinyarwanda/test/data/test_cases_whitelist.txt +3 -0
  1397. wetextprocessing-1.3.0/tn/kinyarwanda/test/data/test_cases_word.txt +26 -0
  1398. wetextprocessing-1.3.0/tn/kinyarwanda/test/normalizer_test.py +198 -0
  1399. wetextprocessing-1.3.0/tn/korean/__init__.py +16 -0
  1400. wetextprocessing-1.3.0/tn/korean/data/date/exceptions.tsv +2 -0
  1401. wetextprocessing-1.3.0/tn/korean/data/date/week.tsv +8 -0
  1402. wetextprocessing-1.3.0/tn/korean/data/electronic/cc_cues.tsv +11 -0
  1403. wetextprocessing-1.3.0/tn/korean/data/electronic/domain.tsv +28 -0
  1404. wetextprocessing-1.3.0/tn/korean/data/electronic/extensions.tsv +6 -0
  1405. wetextprocessing-1.3.0/tn/korean/data/electronic/symbol.tsv +12 -0
  1406. wetextprocessing-1.3.0/tn/korean/data/measure/unit.tsv +18 -0
  1407. wetextprocessing-1.3.0/tn/korean/data/money/currency_major.tsv +22 -0
  1408. wetextprocessing-1.3.0/tn/korean/data/number/counter_suffix.tsv +16 -0
  1409. wetextprocessing-1.3.0/tn/korean/data/number/digit.tsv +9 -0
  1410. wetextprocessing-1.3.0/tn/korean/data/number/native_ones.tsv +9 -0
  1411. wetextprocessing-1.3.0/tn/korean/data/number/tens.tsv +9 -0
  1412. wetextprocessing-1.3.0/tn/korean/data/number/zero.tsv +1 -0
  1413. wetextprocessing-1.3.0/tn/korean/data/ordinal/digit.tsv +8 -0
  1414. wetextprocessing-1.3.0/tn/korean/data/ordinal/exceptions.tsv +5 -0
  1415. wetextprocessing-1.3.0/tn/korean/data/ordinal/tens.tsv +2 -0
  1416. wetextprocessing-1.3.0/tn/korean/data/ordinal/tens_prefix.tsv +3 -0
  1417. wetextprocessing-1.3.0/tn/korean/data/time/division.tsv +7 -0
  1418. wetextprocessing-1.3.0/tn/korean/data/time/hour.tsv +12 -0
  1419. wetextprocessing-1.3.0/tn/korean/data/whitelist.tsv +39 -0
  1420. wetextprocessing-1.3.0/tn/korean/normalizer.py +79 -0
  1421. wetextprocessing-1.3.0/tn/korean/rules/__init__.py +3 -0
  1422. wetextprocessing-1.3.0/tn/korean/rules/cardinal.py +91 -0
  1423. wetextprocessing-1.3.0/tn/korean/rules/date.py +129 -0
  1424. wetextprocessing-1.3.0/tn/korean/rules/decimal.py +53 -0
  1425. wetextprocessing-1.3.0/tn/korean/rules/electronic.py +84 -0
  1426. wetextprocessing-1.3.0/tn/korean/rules/fraction.py +81 -0
  1427. wetextprocessing-1.3.0/tn/korean/rules/measure.py +82 -0
  1428. wetextprocessing-1.3.0/tn/korean/rules/money.py +60 -0
  1429. wetextprocessing-1.3.0/tn/korean/rules/ordinal.py +62 -0
  1430. wetextprocessing-1.3.0/tn/korean/rules/punctuation.py +16 -0
  1431. wetextprocessing-1.3.0/tn/korean/rules/telephone.py +40 -0
  1432. wetextprocessing-1.3.0/tn/korean/rules/time.py +81 -0
  1433. wetextprocessing-1.3.0/tn/korean/rules/whitelist.py +17 -0
  1434. wetextprocessing-1.3.0/tn/korean/rules/word.py +18 -0
  1435. wetextprocessing-1.3.0/tn/korean/test/__init__.py +3 -0
  1436. wetextprocessing-1.3.0/tn/korean/test/data/test_cases_cardinal.txt +68 -0
  1437. wetextprocessing-1.3.0/tn/korean/test/data/test_cases_date.txt +29 -0
  1438. wetextprocessing-1.3.0/tn/korean/test/data/test_cases_decimal.txt +28 -0
  1439. wetextprocessing-1.3.0/tn/korean/test/data/test_cases_electronic.txt +20 -0
  1440. wetextprocessing-1.3.0/tn/korean/test/data/test_cases_fraction.txt +28 -0
  1441. wetextprocessing-1.3.0/tn/korean/test/data/test_cases_measure.txt +40 -0
  1442. wetextprocessing-1.3.0/tn/korean/test/data/test_cases_money.txt +64 -0
  1443. wetextprocessing-1.3.0/tn/korean/test/data/test_cases_ordinal.txt +54 -0
  1444. wetextprocessing-1.3.0/tn/korean/test/data/test_cases_telephone.txt +39 -0
  1445. wetextprocessing-1.3.0/tn/korean/test/data/test_cases_time.txt +25 -0
  1446. wetextprocessing-1.3.0/tn/korean/test/date_test.py +99 -0
  1447. wetextprocessing-1.3.0/tn/korean/test/decimal_test.py +67 -0
  1448. wetextprocessing-1.3.0/tn/korean/test/electronic_test.py +58 -0
  1449. wetextprocessing-1.3.0/tn/korean/test/fraction_test.py +89 -0
  1450. wetextprocessing-1.3.0/tn/korean/test/measure_test.py +90 -0
  1451. wetextprocessing-1.3.0/tn/korean/test/money_test.py +92 -0
  1452. wetextprocessing-1.3.0/tn/korean/test/normalizer_test.py +146 -0
  1453. wetextprocessing-1.3.0/tn/korean/test/ordinal_test.py +78 -0
  1454. wetextprocessing-1.3.0/tn/korean/test/telephone_test.py +40 -0
  1455. wetextprocessing-1.3.0/tn/korean/test/time_test.py +51 -0
  1456. wetextprocessing-1.3.0/tn/main.py +23 -0
  1457. wetextprocessing-1.3.0/tn/marathi/__init__.py +0 -0
  1458. wetextprocessing-1.3.0/tn/marathi/data/date/eras.tsv +2 -0
  1459. wetextprocessing-1.3.0/tn/marathi/data/date/months.tsv +12 -0
  1460. wetextprocessing-1.3.0/tn/marathi/data/numbers/digit.tsv +9 -0
  1461. wetextprocessing-1.3.0/tn/marathi/data/numbers/tens.tsv +90 -0
  1462. wetextprocessing-1.3.0/tn/marathi/data/numbers/zero.tsv +1 -0
  1463. wetextprocessing-1.3.0/tn/marathi/normalizer.py +53 -0
  1464. wetextprocessing-1.3.0/tn/marathi/rules/__init__.py +0 -0
  1465. wetextprocessing-1.3.0/tn/marathi/rules/cardinal.py +76 -0
  1466. wetextprocessing-1.3.0/tn/marathi/rules/date.py +70 -0
  1467. wetextprocessing-1.3.0/tn/marathi/rules/decimal.py +59 -0
  1468. wetextprocessing-1.3.0/tn/marathi/rules/punctuation.py +30 -0
  1469. wetextprocessing-1.3.0/tn/marathi/rules/time.py +86 -0
  1470. wetextprocessing-1.3.0/tn/marathi/rules/utils.py +76 -0
  1471. wetextprocessing-1.3.0/tn/marathi/rules/word.py +28 -0
  1472. wetextprocessing-1.3.0/tn/marathi/test/__init__.py +0 -0
  1473. wetextprocessing-1.3.0/tn/marathi/test/normalizer_test.py +221 -0
  1474. wetextprocessing-1.3.0/tn/portuguese/__init__.py +0 -0
  1475. wetextprocessing-1.3.0/tn/portuguese/data/date/months.tsv +21 -0
  1476. wetextprocessing-1.3.0/tn/portuguese/data/date/numeric_separators.tsv +3 -0
  1477. wetextprocessing-1.3.0/tn/portuguese/data/date/verbal_phrases.tsv +1 -0
  1478. wetextprocessing-1.3.0/tn/portuguese/data/electronic/domain.tsv +7 -0
  1479. wetextprocessing-1.3.0/tn/portuguese/data/electronic/electronic_spoken_unit.tsv +16 -0
  1480. wetextprocessing-1.3.0/tn/portuguese/data/electronic/server_name.tsv +6 -0
  1481. wetextprocessing-1.3.0/tn/portuguese/data/electronic/symbols.tsv +22 -0
  1482. wetextprocessing-1.3.0/tn/portuguese/data/fractions/ordinal_exceptions.tsv +2 -0
  1483. wetextprocessing-1.3.0/tn/portuguese/data/fractions/powers_of_ten.tsv +1 -0
  1484. wetextprocessing-1.3.0/tn/portuguese/data/fractions/specials.tsv +5 -0
  1485. wetextprocessing-1.3.0/tn/portuguese/data/measure/measurements_plural.tsv +64 -0
  1486. wetextprocessing-1.3.0/tn/portuguese/data/measure/measurements_singular.tsv +65 -0
  1487. wetextprocessing-1.3.0/tn/portuguese/data/money/currency_major.tsv +5 -0
  1488. wetextprocessing-1.3.0/tn/portuguese/data/money/currency_major_plural.tsv +5 -0
  1489. wetextprocessing-1.3.0/tn/portuguese/data/money/currency_minor.tsv +5 -0
  1490. wetextprocessing-1.3.0/tn/portuguese/data/money/currency_minor_plural.tsv +1 -0
  1491. wetextprocessing-1.3.0/tn/portuguese/data/numbers/cardinal_specials.tsv +4 -0
  1492. wetextprocessing-1.3.0/tn/portuguese/data/numbers/decimal_specials.tsv +2 -0
  1493. wetextprocessing-1.3.0/tn/portuguese/data/numbers/digit.tsv +9 -0
  1494. wetextprocessing-1.3.0/tn/portuguese/data/numbers/hundreds.tsv +8 -0
  1495. wetextprocessing-1.3.0/tn/portuguese/data/numbers/quantity_words.tsv +9 -0
  1496. wetextprocessing-1.3.0/tn/portuguese/data/numbers/scales.tsv +4 -0
  1497. wetextprocessing-1.3.0/tn/portuguese/data/numbers/teens.tsv +10 -0
  1498. wetextprocessing-1.3.0/tn/portuguese/data/numbers/tens.tsv +8 -0
  1499. wetextprocessing-1.3.0/tn/portuguese/data/numbers/zero.tsv +1 -0
  1500. wetextprocessing-1.3.0/tn/portuguese/data/ordinals/digit.tsv +10 -0
  1501. wetextprocessing-1.3.0/tn/portuguese/data/ordinals/feminine.tsv +11 -0
  1502. wetextprocessing-1.3.0/tn/portuguese/data/ordinals/hundreds.tsv +10 -0
  1503. wetextprocessing-1.3.0/tn/portuguese/data/ordinals/ties.tsv +8 -0
  1504. wetextprocessing-1.3.0/tn/portuguese/data/telephone/ip_prompt.tsv +2 -0
  1505. wetextprocessing-1.3.0/tn/portuguese/data/telephone/telephone_prompt.tsv +4 -0
  1506. wetextprocessing-1.3.0/tn/portuguese/data/time/day_period_suffix.tsv +4 -0
  1507. wetextprocessing-1.3.0/tn/portuguese/data/whitelist/symbol.tsv +23 -0
  1508. wetextprocessing-1.3.0/tn/portuguese/normalizer.py +74 -0
  1509. wetextprocessing-1.3.0/tn/portuguese/rules/__init__.py +0 -0
  1510. wetextprocessing-1.3.0/tn/portuguese/rules/cardinal.py +170 -0
  1511. wetextprocessing-1.3.0/tn/portuguese/rules/date.py +69 -0
  1512. wetextprocessing-1.3.0/tn/portuguese/rules/decimal.py +67 -0
  1513. wetextprocessing-1.3.0/tn/portuguese/rules/electronic.py +82 -0
  1514. wetextprocessing-1.3.0/tn/portuguese/rules/fraction.py +79 -0
  1515. wetextprocessing-1.3.0/tn/portuguese/rules/measure.py +66 -0
  1516. wetextprocessing-1.3.0/tn/portuguese/rules/money.py +117 -0
  1517. wetextprocessing-1.3.0/tn/portuguese/rules/ordinal.py +82 -0
  1518. wetextprocessing-1.3.0/tn/portuguese/rules/punctuation.py +47 -0
  1519. wetextprocessing-1.3.0/tn/portuguese/rules/telephone.py +71 -0
  1520. wetextprocessing-1.3.0/tn/portuguese/rules/time.py +101 -0
  1521. wetextprocessing-1.3.0/tn/portuguese/rules/whitelist.py +37 -0
  1522. wetextprocessing-1.3.0/tn/portuguese/rules/word.py +35 -0
  1523. wetextprocessing-1.3.0/tn/portuguese/test/__init__.py +0 -0
  1524. wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_cardinal.txt +121 -0
  1525. wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_date.txt +26 -0
  1526. wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_decimal.txt +58 -0
  1527. wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_electronic.txt +13 -0
  1528. wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_fraction.txt +22 -0
  1529. wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_measure.txt +18 -0
  1530. wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_money.txt +25 -0
  1531. wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_ordinal.txt +39 -0
  1532. wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_telephone.txt +7 -0
  1533. wetextprocessing-1.3.0/tn/portuguese/test/data/test_cases_time.txt +30 -0
  1534. wetextprocessing-1.3.0/tn/portuguese/test/normalizer_test.py +166 -0
  1535. wetextprocessing-1.3.0/tn/processor.py +518 -0
  1536. wetextprocessing-1.3.0/tn/russian/__init__.py +0 -0
  1537. wetextprocessing-1.3.0/tn/russian/data/currency/currency_agreement.tsv +10 -0
  1538. wetextprocessing-1.3.0/tn/russian/data/currency/currency_plural.tsv +53 -0
  1539. wetextprocessing-1.3.0/tn/russian/data/currency/currency_singular.tsv +48 -0
  1540. wetextprocessing-1.3.0/tn/russian/data/electronic/domain.tsv +9 -0
  1541. wetextprocessing-1.3.0/tn/russian/data/electronic/server_name.tsv +17 -0
  1542. wetextprocessing-1.3.0/tn/russian/data/electronic/symbols.tsv +28 -0
  1543. wetextprocessing-1.3.0/tn/russian/data/latin_to_cyrillic.tsv +38 -0
  1544. wetextprocessing-1.3.0/tn/russian/data/measurements.tsv +321 -0
  1545. wetextprocessing-1.3.0/tn/russian/data/months/abbr.tsv +12 -0
  1546. wetextprocessing-1.3.0/tn/russian/data/months/abbr_to_name.tsv +62 -0
  1547. wetextprocessing-1.3.0/tn/russian/data/months/numbers.tsv +21 -0
  1548. wetextprocessing-1.3.0/tn/russian/data/numbers/1_cardinals_nominative.tsv +50 -0
  1549. wetextprocessing-1.3.0/tn/russian/data/numbers/2_cardinals_genitive.tsv +45 -0
  1550. wetextprocessing-1.3.0/tn/russian/data/numbers/3_cardinals_dative.tsv +45 -0
  1551. wetextprocessing-1.3.0/tn/russian/data/numbers/4_cardinals_accusative.tsv +48 -0
  1552. wetextprocessing-1.3.0/tn/russian/data/numbers/5_cardinals_instrumental.tsv +47 -0
  1553. wetextprocessing-1.3.0/tn/russian/data/numbers/6_cardinals_prepositional.tsv +46 -0
  1554. wetextprocessing-1.3.0/tn/russian/data/numbers/cardinals_alternatives.tsv +16 -0
  1555. wetextprocessing-1.3.0/tn/russian/data/numbers/cardinals_nominative_case.tsv +37 -0
  1556. wetextprocessing-1.3.0/tn/russian/data/numbers/decimal_delimiter.tsv +15 -0
  1557. wetextprocessing-1.3.0/tn/russian/data/numbers/decimal_endings.tsv +37 -0
  1558. wetextprocessing-1.3.0/tn/russian/data/numbers/digits_nominative_case.tsv +10 -0
  1559. wetextprocessing-1.3.0/tn/russian/data/numbers/ordinal_endings.tsv +40 -0
  1560. wetextprocessing-1.3.0/tn/russian/data/numbers/ordinals.tsv +752 -0
  1561. wetextprocessing-1.3.0/tn/russian/data/numbers/quantity.tsv +16 -0
  1562. wetextprocessing-1.3.0/tn/russian/data/time/increment_hour_cardinal.tsv +24 -0
  1563. wetextprocessing-1.3.0/tn/russian/data/time/increment_hour_ordinal.tsv +24 -0
  1564. wetextprocessing-1.3.0/tn/russian/data/time/minutes_to_hour.tsv +25 -0
  1565. wetextprocessing-1.3.0/tn/russian/data/time/time_convert.tsv +12 -0
  1566. wetextprocessing-1.3.0/tn/russian/data/whitelist.tsv +37 -0
  1567. wetextprocessing-1.3.0/tn/russian/normalizer.py +75 -0
  1568. wetextprocessing-1.3.0/tn/russian/rules/__init__.py +0 -0
  1569. wetextprocessing-1.3.0/tn/russian/rules/cardinal.py +92 -0
  1570. wetextprocessing-1.3.0/tn/russian/rules/date.py +93 -0
  1571. wetextprocessing-1.3.0/tn/russian/rules/decimal.py +88 -0
  1572. wetextprocessing-1.3.0/tn/russian/rules/electronic.py +69 -0
  1573. wetextprocessing-1.3.0/tn/russian/rules/measure.py +70 -0
  1574. wetextprocessing-1.3.0/tn/russian/rules/money.py +73 -0
  1575. wetextprocessing-1.3.0/tn/russian/rules/ordinal.py +45 -0
  1576. wetextprocessing-1.3.0/tn/russian/rules/punctuation.py +46 -0
  1577. wetextprocessing-1.3.0/tn/russian/rules/telephone.py +44 -0
  1578. wetextprocessing-1.3.0/tn/russian/rules/time.py +80 -0
  1579. wetextprocessing-1.3.0/tn/russian/rules/utils.py +293 -0
  1580. wetextprocessing-1.3.0/tn/russian/rules/whitelist.py +41 -0
  1581. wetextprocessing-1.3.0/tn/russian/rules/word.py +34 -0
  1582. wetextprocessing-1.3.0/tn/russian/test/__init__.py +0 -0
  1583. wetextprocessing-1.3.0/tn/russian/test/data/test_cases_cardinal.txt +28 -0
  1584. wetextprocessing-1.3.0/tn/russian/test/data/test_cases_cardinal_normalize_with_audio.txt +51 -0
  1585. wetextprocessing-1.3.0/tn/russian/test/data/test_cases_date.txt +6 -0
  1586. wetextprocessing-1.3.0/tn/russian/test/data/test_cases_decimal.txt +17 -0
  1587. wetextprocessing-1.3.0/tn/russian/test/data/test_cases_electronic.txt +4 -0
  1588. wetextprocessing-1.3.0/tn/russian/test/data/test_cases_measure.txt +13 -0
  1589. wetextprocessing-1.3.0/tn/russian/test/data/test_cases_money.txt +9 -0
  1590. wetextprocessing-1.3.0/tn/russian/test/data/test_cases_ordinal.txt +75 -0
  1591. wetextprocessing-1.3.0/tn/russian/test/data/test_cases_telephone.txt +1 -0
  1592. wetextprocessing-1.3.0/tn/russian/test/data/test_cases_time.txt +9 -0
  1593. wetextprocessing-1.3.0/tn/russian/test/data/test_cases_whitelist.txt +4 -0
  1594. wetextprocessing-1.3.0/tn/russian/test/data/test_cases_word.txt +3 -0
  1595. wetextprocessing-1.3.0/tn/russian/test/normalizer_test.py +214 -0
  1596. wetextprocessing-1.3.0/tn/spanish/__init__.py +0 -0
  1597. wetextprocessing-1.3.0/tn/spanish/data/dates/days_abbr.tsv +27 -0
  1598. wetextprocessing-1.3.0/tn/spanish/data/dates/months.tsv +12 -0
  1599. wetextprocessing-1.3.0/tn/spanish/data/dates/months_abbr.tsv +44 -0
  1600. wetextprocessing-1.3.0/tn/spanish/data/dates/year_suffix.tsv +51 -0
  1601. wetextprocessing-1.3.0/tn/spanish/data/electronic/domain.tsv +6 -0
  1602. wetextprocessing-1.3.0/tn/spanish/data/electronic/server_name.tsv +11 -0
  1603. wetextprocessing-1.3.0/tn/spanish/data/electronic/symbols.tsv +22 -0
  1604. wetextprocessing-1.3.0/tn/spanish/data/fractions/fraction_symbols.tsv +15 -0
  1605. wetextprocessing-1.3.0/tn/spanish/data/fractions/ordinal_exceptions.tsv +2 -0
  1606. wetextprocessing-1.3.0/tn/spanish/data/fractions/powers_of_ten.tsv +9 -0
  1607. wetextprocessing-1.3.0/tn/spanish/data/fractions/powers_of_ten_fem.tsv +6 -0
  1608. wetextprocessing-1.3.0/tn/spanish/data/measures/math_symbols.tsv +7 -0
  1609. wetextprocessing-1.3.0/tn/spanish/data/measures/measurements.tsv +18 -0
  1610. wetextprocessing-1.3.0/tn/spanish/data/measures/measurements_complex.tsv +5 -0
  1611. wetextprocessing-1.3.0/tn/spanish/data/measures/measurements_plural_fem.tsv +3 -0
  1612. wetextprocessing-1.3.0/tn/spanish/data/measures/measurements_plural_masc.tsv +21 -0
  1613. wetextprocessing-1.3.0/tn/spanish/data/money/currency_major.tsv +64 -0
  1614. wetextprocessing-1.3.0/tn/spanish/data/money/currency_major_ext.tsv +215 -0
  1615. wetextprocessing-1.3.0/tn/spanish/data/money/currency_minor.tsv +64 -0
  1616. wetextprocessing-1.3.0/tn/spanish/data/money/currency_minor_ext.tsv +181 -0
  1617. wetextprocessing-1.3.0/tn/spanish/data/money/currency_plural_fem.tsv +8 -0
  1618. wetextprocessing-1.3.0/tn/spanish/data/money/currency_plural_fem_ext.tsv +56 -0
  1619. wetextprocessing-1.3.0/tn/spanish/data/money/currency_plural_masc.tsv +51 -0
  1620. wetextprocessing-1.3.0/tn/spanish/data/money/currency_plural_masc_ext.tsv +199 -0
  1621. wetextprocessing-1.3.0/tn/spanish/data/numbers/digit.tsv +9 -0
  1622. wetextprocessing-1.3.0/tn/spanish/data/numbers/hundreds.tsv +8 -0
  1623. wetextprocessing-1.3.0/tn/spanish/data/numbers/quantities.tsv +6 -0
  1624. wetextprocessing-1.3.0/tn/spanish/data/numbers/teen.tsv +10 -0
  1625. wetextprocessing-1.3.0/tn/spanish/data/numbers/ties.tsv +7 -0
  1626. wetextprocessing-1.3.0/tn/spanish/data/numbers/twenties.tsv +10 -0
  1627. wetextprocessing-1.3.0/tn/spanish/data/numbers/zero.tsv +1 -0
  1628. wetextprocessing-1.3.0/tn/spanish/data/ordinals/digit.tsv +11 -0
  1629. wetextprocessing-1.3.0/tn/spanish/data/ordinals/gender_suffix.tsv +20 -0
  1630. wetextprocessing-1.3.0/tn/spanish/data/ordinals/hundreds.tsv +18 -0
  1631. wetextprocessing-1.3.0/tn/spanish/data/ordinals/roman_exceptions.tsv +10 -0
  1632. wetextprocessing-1.3.0/tn/spanish/data/ordinals/teen.tsv +10 -0
  1633. wetextprocessing-1.3.0/tn/spanish/data/ordinals/ties.tsv +8 -0
  1634. wetextprocessing-1.3.0/tn/spanish/data/ordinals/twenties.tsv +11 -0
  1635. wetextprocessing-1.3.0/tn/spanish/data/roman/digit.tsv +9 -0
  1636. wetextprocessing-1.3.0/tn/spanish/data/roman/hundreds.tsv +9 -0
  1637. wetextprocessing-1.3.0/tn/spanish/data/roman/thousands.tsv +3 -0
  1638. wetextprocessing-1.3.0/tn/spanish/data/roman/ties.tsv +9 -0
  1639. wetextprocessing-1.3.0/tn/spanish/data/telephone/ip_prompt.tsv +2 -0
  1640. wetextprocessing-1.3.0/tn/spanish/data/telephone/telephone_prompt.tsv +10 -0
  1641. wetextprocessing-1.3.0/tn/spanish/data/time/afternoon_times.tsv +6 -0
  1642. wetextprocessing-1.3.0/tn/spanish/data/time/alt_minutes.tsv +2 -0
  1643. wetextprocessing-1.3.0/tn/spanish/data/time/evening_times.tsv +5 -0
  1644. wetextprocessing-1.3.0/tn/spanish/data/time/hour_to_12.tsv +12 -0
  1645. wetextprocessing-1.3.0/tn/spanish/data/time/hour_to_24.tsv +25 -0
  1646. wetextprocessing-1.3.0/tn/spanish/data/time/hour_to_night.tsv +13 -0
  1647. wetextprocessing-1.3.0/tn/spanish/data/time/minute_to.tsv +59 -0
  1648. wetextprocessing-1.3.0/tn/spanish/data/time/morning_times.tsv +12 -0
  1649. wetextprocessing-1.3.0/tn/spanish/data/time/time_suffix.tsv +20 -0
  1650. wetextprocessing-1.3.0/tn/spanish/data/time/time_zone.tsv +42 -0
  1651. wetextprocessing-1.3.0/tn/spanish/data/time/time_zone_ext.tsv +105 -0
  1652. wetextprocessing-1.3.0/tn/spanish/data/whitelist.tsv +62 -0
  1653. wetextprocessing-1.3.0/tn/spanish/normalizer.py +79 -0
  1654. wetextprocessing-1.3.0/tn/spanish/rules/__init__.py +0 -0
  1655. wetextprocessing-1.3.0/tn/spanish/rules/cardinal.py +125 -0
  1656. wetextprocessing-1.3.0/tn/spanish/rules/date.py +96 -0
  1657. wetextprocessing-1.3.0/tn/spanish/rules/decimal.py +122 -0
  1658. wetextprocessing-1.3.0/tn/spanish/rules/electronic.py +69 -0
  1659. wetextprocessing-1.3.0/tn/spanish/rules/fraction.py +189 -0
  1660. wetextprocessing-1.3.0/tn/spanish/rules/measure.py +177 -0
  1661. wetextprocessing-1.3.0/tn/spanish/rules/money.py +186 -0
  1662. wetextprocessing-1.3.0/tn/spanish/rules/ordinal.py +119 -0
  1663. wetextprocessing-1.3.0/tn/spanish/rules/punctuation.py +48 -0
  1664. wetextprocessing-1.3.0/tn/spanish/rules/telephone.py +93 -0
  1665. wetextprocessing-1.3.0/tn/spanish/rules/time.py +171 -0
  1666. wetextprocessing-1.3.0/tn/spanish/rules/utils.py +170 -0
  1667. wetextprocessing-1.3.0/tn/spanish/rules/whitelist.py +39 -0
  1668. wetextprocessing-1.3.0/tn/spanish/rules/word.py +39 -0
  1669. wetextprocessing-1.3.0/tn/spanish/test/__init__.py +0 -0
  1670. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_cardinal.txt +69 -0
  1671. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_date.txt +21 -0
  1672. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_decimal.txt +24 -0
  1673. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_electronic.txt +22 -0
  1674. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_fraction.txt +75 -0
  1675. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_measure.txt +32 -0
  1676. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_money.txt +69 -0
  1677. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_normalize_with_audio.txt +162 -0
  1678. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_ordinal.txt +120 -0
  1679. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_telephone.txt +15 -0
  1680. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_time.txt +35 -0
  1681. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_whitelist.txt +7 -0
  1682. wetextprocessing-1.3.0/tn/spanish/test/data/test_cases_word.txt +48 -0
  1683. wetextprocessing-1.3.0/tn/spanish/test/normalizer_test.py +166 -0
  1684. wetextprocessing-1.3.0/tn/swedish/__init__.py +0 -0
  1685. wetextprocessing-1.3.0/tn/swedish/data/abbreviations_nondet.tsv +27 -0
  1686. wetextprocessing-1.3.0/tn/swedish/data/dates/era_suffix.tsv +18 -0
  1687. wetextprocessing-1.3.0/tn/swedish/data/dates/era_words.tsv +4 -0
  1688. wetextprocessing-1.3.0/tn/swedish/data/dates/month_abbr.tsv +17 -0
  1689. wetextprocessing-1.3.0/tn/swedish/data/dates/months.tsv +12 -0
  1690. wetextprocessing-1.3.0/tn/swedish/data/electronic/domain.tsv +10 -0
  1691. wetextprocessing-1.3.0/tn/swedish/data/electronic/server_name.tsv +29 -0
  1692. wetextprocessing-1.3.0/tn/swedish/data/electronic/symbols.tsv +20 -0
  1693. wetextprocessing-1.3.0/tn/swedish/data/math_operations.tsv +8 -0
  1694. wetextprocessing-1.3.0/tn/swedish/data/measure/greek_lower.tsv +25 -0
  1695. wetextprocessing-1.3.0/tn/swedish/data/measure/greek_upper.tsv +25 -0
  1696. wetextprocessing-1.3.0/tn/swedish/data/measure/unit.tsv +147 -0
  1697. wetextprocessing-1.3.0/tn/swedish/data/measure/unit_neuter.tsv +13 -0
  1698. wetextprocessing-1.3.0/tn/swedish/data/measure/unit_plural.tsv +121 -0
  1699. wetextprocessing-1.3.0/tn/swedish/data/money/currency_major.tsv +75 -0
  1700. wetextprocessing-1.3.0/tn/swedish/data/money/currency_major_nt.tsv +2 -0
  1701. wetextprocessing-1.3.0/tn/swedish/data/money/currency_minor_plural.tsv +79 -0
  1702. wetextprocessing-1.3.0/tn/swedish/data/money/currency_minor_singular.tsv +78 -0
  1703. wetextprocessing-1.3.0/tn/swedish/data/money/currency_plurals.tsv +44 -0
  1704. wetextprocessing-1.3.0/tn/swedish/data/numbers/digit.tsv +9 -0
  1705. wetextprocessing-1.3.0/tn/swedish/data/numbers/fraction.tsv +18 -0
  1706. wetextprocessing-1.3.0/tn/swedish/data/numbers/millions.tsv +88 -0
  1707. wetextprocessing-1.3.0/tn/swedish/data/numbers/millions_abbr.tsv +4 -0
  1708. wetextprocessing-1.3.0/tn/swedish/data/numbers/teen.tsv +10 -0
  1709. wetextprocessing-1.3.0/tn/swedish/data/numbers/ties.tsv +8 -0
  1710. wetextprocessing-1.3.0/tn/swedish/data/numbers/zero.tsv +1 -0
  1711. wetextprocessing-1.3.0/tn/swedish/data/ordinals/digit.tsv +9 -0
  1712. wetextprocessing-1.3.0/tn/swedish/data/ordinals/teen.tsv +10 -0
  1713. wetextprocessing-1.3.0/tn/swedish/data/ordinals/ties.tsv +8 -0
  1714. wetextprocessing-1.3.0/tn/swedish/data/ordinals/zero.tsv +1 -0
  1715. wetextprocessing-1.3.0/tn/swedish/data/telephone/ip_prompt.tsv +2 -0
  1716. wetextprocessing-1.3.0/tn/swedish/data/telephone/special_numbers.tsv +4 -0
  1717. wetextprocessing-1.3.0/tn/swedish/data/telephone/ssn_prompt.tsv +3 -0
  1718. wetextprocessing-1.3.0/tn/swedish/data/telephone/telephone_abbr.tsv +24 -0
  1719. wetextprocessing-1.3.0/tn/swedish/data/telephone/telephone_prompt.tsv +5 -0
  1720. wetextprocessing-1.3.0/tn/swedish/data/time/suffix.tsv +16 -0
  1721. wetextprocessing-1.3.0/tn/swedish/data/time/time_zone.tsv +7 -0
  1722. wetextprocessing-1.3.0/tn/swedish/data/whitelist.tsv +58 -0
  1723. wetextprocessing-1.3.0/tn/swedish/normalizer.py +75 -0
  1724. wetextprocessing-1.3.0/tn/swedish/rules/__init__.py +0 -0
  1725. wetextprocessing-1.3.0/tn/swedish/rules/cardinal.py +172 -0
  1726. wetextprocessing-1.3.0/tn/swedish/rules/date.py +122 -0
  1727. wetextprocessing-1.3.0/tn/swedish/rules/decimal.py +110 -0
  1728. wetextprocessing-1.3.0/tn/swedish/rules/electronic.py +75 -0
  1729. wetextprocessing-1.3.0/tn/swedish/rules/fraction.py +85 -0
  1730. wetextprocessing-1.3.0/tn/swedish/rules/measure.py +143 -0
  1731. wetextprocessing-1.3.0/tn/swedish/rules/money.py +127 -0
  1732. wetextprocessing-1.3.0/tn/swedish/rules/ordinal.py +132 -0
  1733. wetextprocessing-1.3.0/tn/swedish/rules/punctuation.py +47 -0
  1734. wetextprocessing-1.3.0/tn/swedish/rules/telephone.py +89 -0
  1735. wetextprocessing-1.3.0/tn/swedish/rules/time.py +98 -0
  1736. wetextprocessing-1.3.0/tn/swedish/rules/utils.py +63 -0
  1737. wetextprocessing-1.3.0/tn/swedish/rules/whitelist.py +38 -0
  1738. wetextprocessing-1.3.0/tn/swedish/rules/word.py +33 -0
  1739. wetextprocessing-1.3.0/tn/swedish/test/__init__.py +0 -0
  1740. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_cardinal.txt +68 -0
  1741. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_date.txt +17 -0
  1742. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_decimal.txt +7 -0
  1743. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_electronic.txt +12 -0
  1744. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_fraction.txt +2 -0
  1745. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_measure.txt +7 -0
  1746. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_money.txt +65 -0
  1747. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_normalize_with_audio.txt +14 -0
  1748. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_ordinal.txt +7 -0
  1749. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_telephone.txt +7 -0
  1750. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_time.txt +12 -0
  1751. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_whitelist.txt +1 -0
  1752. wetextprocessing-1.3.0/tn/swedish/test/data/test_cases_word.txt +48 -0
  1753. wetextprocessing-1.3.0/tn/swedish/test/normalizer_test.py +195 -0
  1754. wetextprocessing-1.3.0/tn/test/cli_test.py +337 -0
  1755. wetextprocessing-1.3.0/tn/test/punctuation_spacing_test.py +22 -0
  1756. wetextprocessing-1.3.0/tn/test/robustness_test.py +32 -0
  1757. wetextprocessing-1.3.0/tn/token_parser.py +217 -0
  1758. wetextprocessing-1.3.0/tn/utils.py +94 -0
  1759. wetextprocessing-1.3.0/tn/vietnamese/__init__.py +0 -0
  1760. wetextprocessing-1.3.0/tn/vietnamese/data/date/days.tsv +40 -0
  1761. wetextprocessing-1.3.0/tn/vietnamese/data/date/months.tsv +21 -0
  1762. wetextprocessing-1.3.0/tn/vietnamese/data/date/year_suffix.tsv +4 -0
  1763. wetextprocessing-1.3.0/tn/vietnamese/data/fraction/denominator_exceptions.tsv +1 -0
  1764. wetextprocessing-1.3.0/tn/vietnamese/data/measure/base_units.tsv +20 -0
  1765. wetextprocessing-1.3.0/tn/vietnamese/data/measure/measurements_minimal.tsv +33 -0
  1766. wetextprocessing-1.3.0/tn/vietnamese/data/measure/prefixes.tsv +17 -0
  1767. wetextprocessing-1.3.0/tn/vietnamese/data/money/currency.tsv +51 -0
  1768. wetextprocessing-1.3.0/tn/vietnamese/data/money/currency_minor.tsv +7 -0
  1769. wetextprocessing-1.3.0/tn/vietnamese/data/money/per_unit_bases.tsv +8 -0
  1770. wetextprocessing-1.3.0/tn/vietnamese/data/money/per_unit_non_metric.tsv +28 -0
  1771. wetextprocessing-1.3.0/tn/vietnamese/data/money/per_unit_prefixes.tsv +6 -0
  1772. wetextprocessing-1.3.0/tn/vietnamese/data/numbers/digit.tsv +9 -0
  1773. wetextprocessing-1.3.0/tn/vietnamese/data/numbers/digit_special.tsv +3 -0
  1774. wetextprocessing-1.3.0/tn/vietnamese/data/numbers/magnitudes.tsv +8 -0
  1775. wetextprocessing-1.3.0/tn/vietnamese/data/numbers/quantity_abbr.tsv +7 -0
  1776. wetextprocessing-1.3.0/tn/vietnamese/data/numbers/teen.tsv +10 -0
  1777. wetextprocessing-1.3.0/tn/vietnamese/data/numbers/ties.tsv +8 -0
  1778. wetextprocessing-1.3.0/tn/vietnamese/data/numbers/zero.tsv +1 -0
  1779. wetextprocessing-1.3.0/tn/vietnamese/data/ordinal/ordinal_exceptions.tsv +2 -0
  1780. wetextprocessing-1.3.0/tn/vietnamese/data/roman/key_word.tsv +16 -0
  1781. wetextprocessing-1.3.0/tn/vietnamese/data/roman/roman_numerals.tsv +13 -0
  1782. wetextprocessing-1.3.0/tn/vietnamese/data/time/time_units.tsv +4 -0
  1783. wetextprocessing-1.3.0/tn/vietnamese/data/time/time_zones.tsv +18 -0
  1784. wetextprocessing-1.3.0/tn/vietnamese/data/whitelist.tsv +0 -0
  1785. wetextprocessing-1.3.0/tn/vietnamese/normalizer.py +106 -0
  1786. wetextprocessing-1.3.0/tn/vietnamese/rules/__init__.py +0 -0
  1787. wetextprocessing-1.3.0/tn/vietnamese/rules/cardinal.py +163 -0
  1788. wetextprocessing-1.3.0/tn/vietnamese/rules/date.py +81 -0
  1789. wetextprocessing-1.3.0/tn/vietnamese/rules/decimal.py +85 -0
  1790. wetextprocessing-1.3.0/tn/vietnamese/rules/fraction.py +50 -0
  1791. wetextprocessing-1.3.0/tn/vietnamese/rules/measure.py +57 -0
  1792. wetextprocessing-1.3.0/tn/vietnamese/rules/money.py +87 -0
  1793. wetextprocessing-1.3.0/tn/vietnamese/rules/ordinal.py +34 -0
  1794. wetextprocessing-1.3.0/tn/vietnamese/rules/punctuation.py +31 -0
  1795. wetextprocessing-1.3.0/tn/vietnamese/rules/range.py +36 -0
  1796. wetextprocessing-1.3.0/tn/vietnamese/rules/roman.py +59 -0
  1797. wetextprocessing-1.3.0/tn/vietnamese/rules/time.py +86 -0
  1798. wetextprocessing-1.3.0/tn/vietnamese/rules/whitelist.py +29 -0
  1799. wetextprocessing-1.3.0/tn/vietnamese/rules/word.py +36 -0
  1800. wetextprocessing-1.3.0/tn/vietnamese/test/__init__.py +0 -0
  1801. wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_cardinal.txt +109 -0
  1802. wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_date.txt +13 -0
  1803. wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_decimal.txt +29 -0
  1804. wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_fraction.txt +13 -0
  1805. wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_measure.txt +63 -0
  1806. wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_money.txt +30 -0
  1807. wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_ordinal.txt +30 -0
  1808. wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_range.txt +11 -0
  1809. wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_roman.txt +57 -0
  1810. wetextprocessing-1.3.0/tn/vietnamese/test/data/test_cases_time.txt +117 -0
  1811. wetextprocessing-1.3.0/tn/vietnamese/test/normalizer_test.py +146 -0
  1812. wetextprocessing-1.0.6.dev19/.github/workflows/unittest.yml +0 -83
  1813. wetextprocessing-1.0.6.dev19/.github/workflows/wheels.yml +0 -71
  1814. wetextprocessing-1.0.6.dev19/.gitignore +0 -47
  1815. wetextprocessing-1.0.6.dev19/.pre-commit-config.yaml +0 -23
  1816. wetextprocessing-1.0.6.dev19/PKG-INFO +0 -160
  1817. wetextprocessing-1.0.6.dev19/README.md +0 -137
  1818. wetextprocessing-1.0.6.dev19/WeTextProcessing.egg-info/PKG-INFO +0 -160
  1819. wetextprocessing-1.0.6.dev19/WeTextProcessing.egg-info/SOURCES.txt +0 -466
  1820. wetextprocessing-1.0.6.dev19/itn/README.md +0 -60
  1821. wetextprocessing-1.0.6.dev19/itn/chinese/inverse_normalizer.py +0 -88
  1822. wetextprocessing-1.0.6.dev19/itn/chinese/rules/cardinal.py +0 -178
  1823. wetextprocessing-1.0.6.dev19/itn/chinese/rules/date.py +0 -56
  1824. wetextprocessing-1.0.6.dev19/itn/chinese/rules/fraction.py +0 -56
  1825. wetextprocessing-1.0.6.dev19/itn/chinese/rules/license_plate.py +0 -37
  1826. wetextprocessing-1.0.6.dev19/itn/chinese/rules/math.py +0 -37
  1827. wetextprocessing-1.0.6.dev19/itn/chinese/rules/measure.py +0 -116
  1828. wetextprocessing-1.0.6.dev19/itn/chinese/rules/money.py +0 -59
  1829. wetextprocessing-1.0.6.dev19/itn/chinese/rules/postprocessor.py +0 -32
  1830. wetextprocessing-1.0.6.dev19/itn/chinese/rules/time.py +0 -60
  1831. wetextprocessing-1.0.6.dev19/itn/chinese/rules/train_number.py +0 -36
  1832. wetextprocessing-1.0.6.dev19/itn/chinese/rules/whitelist.py +0 -33
  1833. wetextprocessing-1.0.6.dev19/itn/chinese/test/normalizer_test.py +0 -117
  1834. wetextprocessing-1.0.6.dev19/itn/japanese/inverse_normalizer.py +0 -96
  1835. wetextprocessing-1.0.6.dev19/itn/japanese/rules/cardinal.py +0 -158
  1836. wetextprocessing-1.0.6.dev19/itn/japanese/rules/date.py +0 -71
  1837. wetextprocessing-1.0.6.dev19/itn/japanese/rules/fraction.py +0 -64
  1838. wetextprocessing-1.0.6.dev19/itn/japanese/rules/math.py +0 -39
  1839. wetextprocessing-1.0.6.dev19/itn/japanese/rules/measure.py +0 -48
  1840. wetextprocessing-1.0.6.dev19/itn/japanese/rules/money.py +0 -45
  1841. wetextprocessing-1.0.6.dev19/itn/japanese/rules/ordinal.py +0 -34
  1842. wetextprocessing-1.0.6.dev19/itn/japanese/rules/postprocessor.py +0 -46
  1843. wetextprocessing-1.0.6.dev19/itn/japanese/rules/preprocessor.py +0 -31
  1844. wetextprocessing-1.0.6.dev19/itn/japanese/rules/time.py +0 -53
  1845. wetextprocessing-1.0.6.dev19/itn/japanese/rules/whitelist.py +0 -33
  1846. wetextprocessing-1.0.6.dev19/itn/japanese/test/normalizer_test.py +0 -113
  1847. wetextprocessing-1.0.6.dev19/itn/main.py +0 -72
  1848. wetextprocessing-1.0.6.dev19/itn/zh_itn_tagger.fst +0 -0
  1849. wetextprocessing-1.0.6.dev19/itn/zh_itn_verbalizer.fst +0 -0
  1850. wetextprocessing-1.0.6.dev19/pyproject.toml +0 -57
  1851. wetextprocessing-1.0.6.dev19/runtime/CMakeLists.txt +0 -38
  1852. wetextprocessing-1.0.6.dev19/runtime/android/.gitignore +0 -15
  1853. wetextprocessing-1.0.6.dev19/runtime/android/app/build.gradle +0 -105
  1854. wetextprocessing-1.0.6.dev19/runtime/android/app/src/androidTest/java/com/mobvoi/WeTextProcessing/ExampleInstrumentedTest.java +0 -26
  1855. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/AndroidManifest.xml +0 -22
  1856. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/assets/README.md +0 -1
  1857. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/cpp/CMakeLists.txt +0 -22
  1858. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/cpp/wetextprocessing.cc +0 -80
  1859. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/java/com/mobvoi/WeTextProcessing/MainActivity.java +0 -78
  1860. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/java/com/mobvoi/WeTextProcessing/WeTextProcessing.java +0 -12
  1861. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/drawable/ic_launcher_background.xml +0 -170
  1862. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/drawable-v24/ic_launcher_foreground.xml +0 -30
  1863. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/layout/activity_main.xml +0 -59
  1864. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/mipmap-hdpi/ic_launcher.png +0 -0
  1865. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/mipmap-hdpi/ic_launcher_round.png +0 -0
  1866. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/mipmap-mdpi/ic_launcher.png +0 -0
  1867. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/mipmap-mdpi/ic_launcher_round.png +0 -0
  1868. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/mipmap-xhdpi/ic_launcher.png +0 -0
  1869. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/mipmap-xhdpi/ic_launcher_round.png +0 -0
  1870. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png +0 -0
  1871. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.png +0 -0
  1872. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png +0 -0
  1873. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.png +0 -0
  1874. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/values/attrs.xml +0 -17
  1875. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/values/colors.xml +0 -16
  1876. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/values/themes.xml +0 -16
  1877. wetextprocessing-1.0.6.dev19/runtime/android/app/src/main/res/values-night/themes.xml +0 -16
  1878. wetextprocessing-1.0.6.dev19/runtime/android/app/src/test/java/com/mobvoi/WeTextProcessing/ExampleUnitTest.java +0 -17
  1879. wetextprocessing-1.0.6.dev19/runtime/android/build.gradle +0 -21
  1880. wetextprocessing-1.0.6.dev19/runtime/android/gradle/wrapper/gradle-wrapper.properties +0 -5
  1881. wetextprocessing-1.0.6.dev19/runtime/android/gradle.properties +0 -19
  1882. wetextprocessing-1.0.6.dev19/runtime/android/settings.gradle +0 -2
  1883. wetextprocessing-1.0.6.dev19/runtime/bin/CMakeLists.txt +0 -3
  1884. wetextprocessing-1.0.6.dev19/runtime/cmake/glog.cmake +0 -6
  1885. wetextprocessing-1.0.6.dev19/runtime/cmake/openfst.cmake +0 -43
  1886. wetextprocessing-1.0.6.dev19/runtime/patch/CPPLINT.cfg +0 -1
  1887. wetextprocessing-1.0.6.dev19/runtime/patch/openfst/src/CMakeLists.txt +0 -23
  1888. wetextprocessing-1.0.6.dev19/runtime/patch/openfst/src/extensions/special/CMakeLists.txt +0 -67
  1889. wetextprocessing-1.0.6.dev19/runtime/patch/openfst/src/include/fst/flags.h +0 -219
  1890. wetextprocessing-1.0.6.dev19/runtime/patch/openfst/src/include/fst/log.h +0 -78
  1891. wetextprocessing-1.0.6.dev19/runtime/patch/openfst/src/lib/flags.cc +0 -160
  1892. wetextprocessing-1.0.6.dev19/runtime/patch/openfst/src/test/CMakeLists.txt +0 -55
  1893. wetextprocessing-1.0.6.dev19/runtime/processor/CMakeLists.txt +0 -22
  1894. wetextprocessing-1.0.6.dev19/runtime/processor/wetext_processor.cc +0 -83
  1895. wetextprocessing-1.0.6.dev19/runtime/processor/wetext_token_parser.cc +0 -167
  1896. wetextprocessing-1.0.6.dev19/runtime/processor/wetext_token_parser.h +0 -96
  1897. wetextprocessing-1.0.6.dev19/runtime/test/CMakeLists.txt +0 -18
  1898. wetextprocessing-1.0.6.dev19/runtime/test/processor_test.cc +0 -72
  1899. wetextprocessing-1.0.6.dev19/runtime/test/token_parser_test.cc +0 -103
  1900. wetextprocessing-1.0.6.dev19/runtime/utils/wetext_flags.h +0 -23
  1901. wetextprocessing-1.0.6.dev19/runtime/utils/wetext_log.h +0 -23
  1902. wetextprocessing-1.0.6.dev19/tn/README.md +0 -69
  1903. wetextprocessing-1.0.6.dev19/tn/_version.py +0 -34
  1904. wetextprocessing-1.0.6.dev19/tn/chinese/normalizer.py +0 -104
  1905. wetextprocessing-1.0.6.dev19/tn/chinese/rules/cardinal.py +0 -92
  1906. wetextprocessing-1.0.6.dev19/tn/chinese/rules/date.py +0 -64
  1907. wetextprocessing-1.0.6.dev19/tn/chinese/rules/fraction.py +0 -49
  1908. wetextprocessing-1.0.6.dev19/tn/chinese/rules/math.py +0 -39
  1909. wetextprocessing-1.0.6.dev19/tn/chinese/rules/measure.py +0 -69
  1910. wetextprocessing-1.0.6.dev19/tn/chinese/rules/money.py +0 -51
  1911. wetextprocessing-1.0.6.dev19/tn/chinese/rules/postprocessor.py +0 -48
  1912. wetextprocessing-1.0.6.dev19/tn/chinese/rules/preprocessor.py +0 -31
  1913. wetextprocessing-1.0.6.dev19/tn/chinese/rules/sport.py +0 -47
  1914. wetextprocessing-1.0.6.dev19/tn/chinese/rules/time.py +0 -59
  1915. wetextprocessing-1.0.6.dev19/tn/chinese/rules/whitelist.py +0 -45
  1916. wetextprocessing-1.0.6.dev19/tn/chinese/test/data/normalizer.txt +0 -48
  1917. wetextprocessing-1.0.6.dev19/tn/chinese/test/normalizer_test.py +0 -41
  1918. wetextprocessing-1.0.6.dev19/tn/chinese/test/postprocessor_test.py +0 -29
  1919. wetextprocessing-1.0.6.dev19/tn/chinese/test/preprocessor_test.py +0 -29
  1920. wetextprocessing-1.0.6.dev19/tn/chinese/test/token_parser_test.py +0 -76
  1921. wetextprocessing-1.0.6.dev19/tn/en_tn_tagger.fst +0 -0
  1922. wetextprocessing-1.0.6.dev19/tn/en_tn_verbalizer.fst +0 -0
  1923. wetextprocessing-1.0.6.dev19/tn/english/data/measure/unit_alternatives.tsv +0 -43
  1924. wetextprocessing-1.0.6.dev19/tn/english/normalizer.py +0 -94
  1925. wetextprocessing-1.0.6.dev19/tn/english/rules/cardinal.py +0 -173
  1926. wetextprocessing-1.0.6.dev19/tn/english/rules/date.py +0 -352
  1927. wetextprocessing-1.0.6.dev19/tn/english/rules/decimal.py +0 -192
  1928. wetextprocessing-1.0.6.dev19/tn/english/rules/electronic.py +0 -199
  1929. wetextprocessing-1.0.6.dev19/tn/english/rules/fraction.py +0 -122
  1930. wetextprocessing-1.0.6.dev19/tn/english/rules/measure.py +0 -354
  1931. wetextprocessing-1.0.6.dev19/tn/english/rules/money.py +0 -114
  1932. wetextprocessing-1.0.6.dev19/tn/english/rules/ordinal.py +0 -91
  1933. wetextprocessing-1.0.6.dev19/tn/english/rules/punctuation.py +0 -81
  1934. wetextprocessing-1.0.6.dev19/tn/english/rules/range.py +0 -131
  1935. wetextprocessing-1.0.6.dev19/tn/english/rules/roman.py +0 -159
  1936. wetextprocessing-1.0.6.dev19/tn/english/rules/telephone.py +0 -165
  1937. wetextprocessing-1.0.6.dev19/tn/english/rules/time.py +0 -188
  1938. wetextprocessing-1.0.6.dev19/tn/english/rules/whitelist.py +0 -140
  1939. wetextprocessing-1.0.6.dev19/tn/english/rules/word.py +0 -52
  1940. wetextprocessing-1.0.6.dev19/tn/english/test/data/date.txt +0 -14
  1941. wetextprocessing-1.0.6.dev19/tn/english/test/data/normalizer.txt +0 -8
  1942. wetextprocessing-1.0.6.dev19/tn/english/test/electronic_test.py +0 -28
  1943. wetextprocessing-1.0.6.dev19/tn/english/test/normalizer_test.py +0 -28
  1944. wetextprocessing-1.0.6.dev19/tn/japanese/normalizer.py +0 -106
  1945. wetextprocessing-1.0.6.dev19/tn/japanese/rules/cardinal.py +0 -132
  1946. wetextprocessing-1.0.6.dev19/tn/japanese/rules/char.py +0 -29
  1947. wetextprocessing-1.0.6.dev19/tn/japanese/rules/date.py +0 -67
  1948. wetextprocessing-1.0.6.dev19/tn/japanese/rules/fraction.py +0 -49
  1949. wetextprocessing-1.0.6.dev19/tn/japanese/rules/math.py +0 -37
  1950. wetextprocessing-1.0.6.dev19/tn/japanese/rules/measure.py +0 -48
  1951. wetextprocessing-1.0.6.dev19/tn/japanese/rules/money.py +0 -51
  1952. wetextprocessing-1.0.6.dev19/tn/japanese/rules/postprocessor.py +0 -46
  1953. wetextprocessing-1.0.6.dev19/tn/japanese/rules/preprocessor.py +0 -31
  1954. wetextprocessing-1.0.6.dev19/tn/japanese/rules/sport.py +0 -50
  1955. wetextprocessing-1.0.6.dev19/tn/japanese/rules/time.py +0 -54
  1956. wetextprocessing-1.0.6.dev19/tn/japanese/rules/transliteration.py +0 -33
  1957. wetextprocessing-1.0.6.dev19/tn/japanese/rules/whitelist.py +0 -36
  1958. wetextprocessing-1.0.6.dev19/tn/japanese/test/normalizer_test.py +0 -42
  1959. wetextprocessing-1.0.6.dev19/tn/main.py +0 -73
  1960. wetextprocessing-1.0.6.dev19/tn/processor.py +0 -124
  1961. wetextprocessing-1.0.6.dev19/tn/token_parser.py +0 -155
  1962. wetextprocessing-1.0.6.dev19/tn/utils.py +0 -84
  1963. wetextprocessing-1.0.6.dev19/tn/zh_tn_tagger.fst +0 -0
  1964. wetextprocessing-1.0.6.dev19/tn/zh_tn_verbalizer.fst +0 -0
  1965. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/.clang-format +0 -0
  1966. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/.flake8 +0 -0
  1967. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/CPPLINT.cfg +0 -0
  1968. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/LICENSE +0 -0
  1969. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/WeTextProcessing.egg-info/dependency_links.txt +0 -0
  1970. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/WeTextProcessing.egg-info/entry_points.txt +0 -0
  1971. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/WeTextProcessing.egg-info/requires.txt +0 -0
  1972. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/WeTextProcessing.egg-info/top_level.txt +0 -0
  1973. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/__init__.py +0 -0
  1974. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/__main__.py +0 -0
  1975. {wetextprocessing-1.0.6.dev19/itn/chinese → wetextprocessing-1.3.0/itn/arabic}/__init__.py +0 -0
  1976. {wetextprocessing-1.0.6.dev19/itn/chinese → wetextprocessing-1.3.0/itn/arabic}/rules/__init__.py +0 -0
  1977. {wetextprocessing-1.0.6.dev19/itn/japanese → wetextprocessing-1.3.0/itn/arabic/test}/__init__.py +0 -0
  1978. {wetextprocessing-1.0.6.dev19/itn/japanese/rules → wetextprocessing-1.3.0/itn/armenian}/__init__.py +0 -0
  1979. {wetextprocessing-1.0.6.dev19/itn/japanese/test → wetextprocessing-1.3.0/itn/armenian/rules}/__init__.py +0 -0
  1980. {wetextprocessing-1.0.6.dev19/tn → wetextprocessing-1.3.0/itn/armenian/test}/__init__.py +0 -0
  1981. {wetextprocessing-1.0.6.dev19/tn → wetextprocessing-1.3.0/itn}/chinese/__init__.py +0 -0
  1982. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/date/dd.tsv +0 -0
  1983. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/date/mm.tsv +0 -0
  1984. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/default/blacklist.tsv +0 -0
  1985. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/default/whitelist.tsv +0 -0
  1986. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/license_plate/province.tsv +0 -0
  1987. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/math/operator.tsv +0 -0
  1988. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/measure/units_en.tsv +0 -0
  1989. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/measure/units_zh.tsv +0 -0
  1990. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/money/code.tsv +0 -0
  1991. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/money/symbol.tsv +0 -0
  1992. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/number/digit.tsv +0 -0
  1993. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/number/digit_zh.tsv +0 -0
  1994. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/number/dot.tsv +0 -0
  1995. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/number/repeat_digit.tsv +0 -0
  1996. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/number/sign.tsv +0 -0
  1997. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/number/special_dash.tsv +0 -0
  1998. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/number/special_tilde.tsv +0 -0
  1999. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/number/zero.tsv +0 -0
  2000. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/time/hour.tsv +0 -0
  2001. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/time/minute.tsv +0 -0
  2002. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/time/minute_zero.tsv +0 -0
  2003. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/time/noon.tsv +0 -0
  2004. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/time/second.tsv +0 -0
  2005. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/data/train_number/prefix.tsv +0 -0
  2006. {wetextprocessing-1.0.6.dev19/tn → wetextprocessing-1.3.0/itn}/chinese/rules/__init__.py +0 -0
  2007. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/rules/char.py +0 -0
  2008. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/__init__.py +0 -0
  2009. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/cardinal.txt +0 -0
  2010. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/char.txt +0 -0
  2011. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/date.txt +0 -0
  2012. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/fraction.txt +0 -0
  2013. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/license_plate.txt +0 -0
  2014. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/math.txt +0 -0
  2015. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/measure.txt +0 -0
  2016. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/money.txt +0 -0
  2017. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/normalizer.txt +0 -0
  2018. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/normalizer_disable_standalone_number_disable_0_to_9.txt +0 -0
  2019. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/normalizer_disable_standalone_number_enable_0_to_9.txt +0 -0
  2020. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/normalizer_enable_standalone_number_disable_0_to_9.txt +0 -0
  2021. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/number.txt +0 -0
  2022. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/time.txt +0 -0
  2023. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/train_number.txt +0 -0
  2024. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/data/whitelist.txt +0 -0
  2025. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/chinese/test/utils.py +0 -0
  2026. {wetextprocessing-1.0.6.dev19/tn → wetextprocessing-1.3.0/itn}/english/__init__.py +0 -0
  2027. {wetextprocessing-1.0.6.dev19/tn/english/data/number → wetextprocessing-1.3.0/itn/english/data/numbers}/zero.tsv +0 -0
  2028. {wetextprocessing-1.0.6.dev19/tn/english/data/ordinal → wetextprocessing-1.3.0/itn/english/data/ordinals}/teen.tsv +0 -0
  2029. {wetextprocessing-1.0.6.dev19/tn/english/data → wetextprocessing-1.3.0/itn/english/rules}/__init__.py +0 -0
  2030. {wetextprocessing-1.0.6.dev19/tn/english/data/date → wetextprocessing-1.3.0/itn/english/test}/__init__.py +0 -0
  2031. {wetextprocessing-1.0.6.dev19/itn/japanese → wetextprocessing-1.3.0/itn/english}/test/utils.py +0 -0
  2032. {wetextprocessing-1.0.6.dev19/tn/english/data/electronic → wetextprocessing-1.3.0/itn/french}/__init__.py +0 -0
  2033. {wetextprocessing-1.0.6.dev19/tn/english/data/money → wetextprocessing-1.3.0/itn/french/rules}/__init__.py +0 -0
  2034. {wetextprocessing-1.0.6.dev19/tn/english/data/ordinal → wetextprocessing-1.3.0/itn/french/test}/__init__.py +0 -0
  2035. {wetextprocessing-1.0.6.dev19/tn/english/data/roman → wetextprocessing-1.3.0/itn/german}/__init__.py +0 -0
  2036. {wetextprocessing-1.0.6.dev19/tn/english/data/telephone → wetextprocessing-1.3.0/itn/german/rules}/__init__.py +0 -0
  2037. {wetextprocessing-1.0.6.dev19/tn/english/data/whitelist → wetextprocessing-1.3.0/itn/german/test}/__init__.py +0 -0
  2038. {wetextprocessing-1.0.6.dev19/tn/english/rules → wetextprocessing-1.3.0/itn/hebrew}/__init__.py +0 -0
  2039. {wetextprocessing-1.0.6.dev19/tn/english/test → wetextprocessing-1.3.0/itn/hebrew/rules}/__init__.py +0 -0
  2040. {wetextprocessing-1.0.6.dev19/tn/japanese → wetextprocessing-1.3.0/itn/hebrew/test}/__init__.py +0 -0
  2041. {wetextprocessing-1.0.6.dev19/tn/japanese/test → wetextprocessing-1.3.0/itn/hindi}/__init__.py +0 -0
  2042. /wetextprocessing-1.0.6.dev19/itn/japanese/data/default/blacklist.tsv → /wetextprocessing-1.3.0/itn/hindi/rules/__init__.py +0 -0
  2043. /wetextprocessing-1.0.6.dev19/tn/japanese/data/default/blacklist.tsv → /wetextprocessing-1.3.0/itn/hindi/test/__init__.py +0 -0
  2044. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/char/common_chinese_char.tsv +0 -0
  2045. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/char/fullwidth_to_halfwidth.tsv +0 -0
  2046. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/char/hiragana_and_katakana.tsv +0 -0
  2047. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/char/oov_tags.tsv +0 -0
  2048. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/char/punctuations_ja.tsv +0 -0
  2049. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/date/day.tsv +0 -0
  2050. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/date/month.tsv +0 -0
  2051. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/date/week.tsv +0 -0
  2052. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/default/whitelist.tsv +0 -0
  2053. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/math/operator.tsv +0 -0
  2054. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/measure/unit_en.tsv +0 -0
  2055. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/measure/unit_ja.tsv +0 -0
  2056. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/money/symbol.tsv +0 -0
  2057. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/number/digit.tsv +0 -0
  2058. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/number/dot.tsv +0 -0
  2059. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/number/hundred.tsv +0 -0
  2060. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/number/hundred_digit.tsv +0 -0
  2061. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/number/sign.tsv +0 -0
  2062. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/number/teen.tsv +0 -0
  2063. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/number/thousands.tsv +0 -0
  2064. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/number/ties.tsv +0 -0
  2065. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/number/zero.tsv +0 -0
  2066. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/time/hour.tsv +0 -0
  2067. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/time/minute.tsv +0 -0
  2068. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/data/time/second.tsv +0 -0
  2069. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/cardinal.txt +0 -0
  2070. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/char.txt +0 -0
  2071. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/date.txt +0 -0
  2072. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/fraction.txt +0 -0
  2073. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/math.txt +0 -0
  2074. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/measure.txt +0 -0
  2075. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/money.txt +0 -0
  2076. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/normalizer_disable_standalone_number_disable_0_to_9.txt +0 -0
  2077. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/normalizer_disable_standalone_number_enable_0_to_9.txt +0 -0
  2078. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/normalizer_enable_standalone_number_disable_0_to_9.txt +0 -0
  2079. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/number.txt +0 -0
  2080. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/time.txt +0 -0
  2081. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/itn/japanese/test/data/whitelist.txt +0 -0
  2082. {wetextprocessing-1.0.6.dev19/tn/chinese → wetextprocessing-1.3.0/itn/japanese}/test/utils.py +0 -0
  2083. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/README.md +0 -0
  2084. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/android/app/.gitignore +0 -0
  2085. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/android/app/proguard-rules.pro +0 -0
  2086. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/android/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml +0 -0
  2087. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/android/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml +0 -0
  2088. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/android/app/src/main/res/values/strings.xml +0 -0
  2089. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/android/app/wenet.keystore +0 -0
  2090. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/android/gradle/wrapper/gradle-wrapper.jar +0 -0
  2091. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/android/gradlew +0 -0
  2092. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/android/gradlew.bat +0 -0
  2093. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/bin/processor_main.cc +0 -0
  2094. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/cmake/gflags.cmake +0 -0
  2095. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/cmake/gtest.cmake +0 -0
  2096. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/processor/wetext_processor.h +0 -0
  2097. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/processor/wetext_processor_c_api.cc +0 -0
  2098. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/processor/wetext_processor_c_api.h +0 -0
  2099. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/test/string_test.cc +0 -0
  2100. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/utils/CMakeLists.txt +0 -0
  2101. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/utils/wetext_string.cc +0 -0
  2102. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/runtime/utils/wetext_string.h +0 -0
  2103. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/setup.cfg +0 -0
  2104. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/__main__.py +0 -0
  2105. {wetextprocessing-1.0.6.dev19/tn/english → wetextprocessing-1.3.0/tn/arabic}/data/number/fraction.tsv +0 -0
  2106. {wetextprocessing-1.0.6.dev19/tn/english/data/whitelist → wetextprocessing-1.3.0/tn/arabic/data/punctuation}/symbol.tsv +0 -0
  2107. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/char/charset_extension.tsv +0 -0
  2108. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/char/charset_national_standard_2013_8105.tsv +0 -0
  2109. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/char/fullwidth_to_halfwidth.tsv +0 -0
  2110. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/char/punctuations_zh.tsv +0 -0
  2111. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/char/traditional_to_simple.tsv +0 -0
  2112. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/date/d.tsv +0 -0
  2113. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/date/dd.tsv +0 -0
  2114. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/date/m.tsv +0 -0
  2115. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/date/mm.tsv +0 -0
  2116. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/default/blacklist.tsv +0 -0
  2117. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/default/whitelist.tsv +0 -0
  2118. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/erhua/whitelist.tsv +0 -0
  2119. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/math/operator.tsv +0 -0
  2120. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/measure/units_en.tsv +0 -0
  2121. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/measure/units_zh.tsv +0 -0
  2122. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/money/code.tsv +0 -0
  2123. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/money/symbol.tsv +0 -0
  2124. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/number/digit.tsv +0 -0
  2125. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/number/dot.tsv +0 -0
  2126. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/number/sign.tsv +0 -0
  2127. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/number/teen.tsv +0 -0
  2128. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/number/zero.tsv +0 -0
  2129. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/sport/club.tsv +0 -0
  2130. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/sport/country.tsv +0 -0
  2131. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/time/hour.tsv +0 -0
  2132. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/time/minute.tsv +0 -0
  2133. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/time/noon.tsv +0 -0
  2134. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/data/time/second.tsv +0 -0
  2135. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/rules/char.py +0 -0
  2136. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/__init__.py +0 -0
  2137. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/cardinal_test.py +0 -0
  2138. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/char_test.py +0 -0
  2139. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/cardinal.txt +0 -0
  2140. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/char.txt +0 -0
  2141. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/date.txt +0 -0
  2142. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/fraction.txt +0 -0
  2143. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/math.txt +0 -0
  2144. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/measure.txt +0 -0
  2145. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/money.txt +0 -0
  2146. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/number.txt +0 -0
  2147. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/postprocessor.txt +0 -0
  2148. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/preprocessor.txt +0 -0
  2149. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/sport.txt +0 -0
  2150. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/time.txt +0 -0
  2151. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/data/whitelist.txt +0 -0
  2152. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/date_test.py +0 -0
  2153. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/fraction_test.py +0 -0
  2154. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/math_test.py +0 -0
  2155. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/measure_test.py +0 -0
  2156. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/money_test.py +0 -0
  2157. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/sport_test.py +0 -0
  2158. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/time_test.py +0 -0
  2159. {wetextprocessing-1.0.6.dev19/tn/english → wetextprocessing-1.3.0/tn/chinese}/test/utils.py +0 -0
  2160. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/chinese/test/whitelist_test.py +0 -0
  2161. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/address/__init__.py +0 -0
  2162. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/address/address_word.tsv +0 -0
  2163. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/address/state.tsv +0 -0
  2164. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/date/day.tsv +0 -0
  2165. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/date/month_abbr.tsv +0 -0
  2166. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/date/month_name.tsv +0 -0
  2167. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/date/month_number.tsv +0 -0
  2168. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/date/week.tsv +0 -0
  2169. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/date/year_suffix.tsv +0 -0
  2170. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/electronic/domain.tsv +0 -0
  2171. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/electronic/symbol.tsv +0 -0
  2172. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/electronic/words.tsv +0 -0
  2173. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/measure/__init__.py +0 -0
  2174. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/measure/math_operation.tsv +0 -0
  2175. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/measure/unit.tsv +0 -0
  2176. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/money/currency_major.tsv +0 -0
  2177. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/money/currency_minor_plural.tsv +0 -0
  2178. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/money/currency_minor_singular.tsv +0 -0
  2179. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/money/per_unit.tsv +0 -0
  2180. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/number/__init__.py +0 -0
  2181. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/number/cardinal_number_name.far +0 -0
  2182. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/number/cardinal_number_name_au.far +0 -0
  2183. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/number/digit.tsv +0 -0
  2184. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/number/hundred.tsv +0 -0
  2185. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/number/quantity_abbr.tsv +0 -0
  2186. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/number/teen.tsv +0 -0
  2187. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/number/thousand.tsv +0 -0
  2188. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/number/ty.tsv +0 -0
  2189. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/ordinal/digit.tsv +0 -0
  2190. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/roman/README.md +0 -0
  2191. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/roman/female.tsv +0 -0
  2192. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/roman/key_word.tsv +0 -0
  2193. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/roman/male.tsv +0 -0
  2194. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/roman/roman_to_spoken.tsv +0 -0
  2195. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/suppletive.tsv +0 -0
  2196. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/telephone/ip_prompt.tsv +0 -0
  2197. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/telephone/ssn_prompt.tsv +0 -0
  2198. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/telephone/telephone_prompt.tsv +0 -0
  2199. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/time/__init__.py +0 -0
  2200. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/time/suffix.tsv +0 -0
  2201. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/time/zone.tsv +0 -0
  2202. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/whitelist/alternatives.tsv +0 -0
  2203. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/whitelist/alternatives_all_format.tsv +0 -0
  2204. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/whitelist/asr.tsv +0 -0
  2205. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/whitelist/asr_with_pc.tsv +0 -0
  2206. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/whitelist/ipa_symbols.tsv +0 -0
  2207. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/whitelist/lj_speech.tsv +0 -0
  2208. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/data/whitelist/tts.tsv +0 -0
  2209. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/cardinal_test.py +0 -0
  2210. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/cardinal.txt +0 -0
  2211. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/decimal.txt +0 -0
  2212. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/electronic.txt +0 -0
  2213. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/fraction.txt +0 -0
  2214. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/measure.txt +0 -0
  2215. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/money.txt +0 -0
  2216. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/ordinal.txt +0 -0
  2217. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/range.txt +0 -0
  2218. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/roman.txt +0 -0
  2219. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/telephone.txt +0 -0
  2220. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/time.txt +0 -0
  2221. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/whitelist.txt +0 -0
  2222. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/data/word.txt +0 -0
  2223. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/date_test.py +0 -0
  2224. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/decimal_test.py +0 -0
  2225. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/fraction_test.py +0 -0
  2226. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/measure_test.py +0 -0
  2227. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/money_test.py +0 -0
  2228. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/ordinal_test.py +0 -0
  2229. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/range_test.py +0 -0
  2230. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/roman_test.py +0 -0
  2231. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/telephone_test.py +0 -0
  2232. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/time_test.py +0 -0
  2233. {wetextprocessing-1.0.6.dev19/tn/japanese → wetextprocessing-1.3.0/tn/english}/test/utils.py +0 -0
  2234. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/whitelist_test.py +0 -0
  2235. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/english/test/word_test.py +0 -0
  2236. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/char/common_chinese_char.tsv +0 -0
  2237. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/char/fullwidth_to_halfwidth.tsv +0 -0
  2238. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/char/hiragana_and_katakana.tsv +0 -0
  2239. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/char/oov_tags.tsv +0 -0
  2240. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/char/punctuations_ja.tsv +0 -0
  2241. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/date/d.tsv +0 -0
  2242. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/date/date.tsv +0 -0
  2243. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/date/dd.tsv +0 -0
  2244. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/date/m.tsv +0 -0
  2245. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/date/mm.tsv +0 -0
  2246. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/default/whitelist.tsv +0 -0
  2247. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/math/operator.tsv +0 -0
  2248. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/measure/units_en.tsv +0 -0
  2249. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/measure/units_ja.tsv +0 -0
  2250. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/money/code.tsv +0 -0
  2251. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/money/symbol.tsv +0 -0
  2252. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/number/digit.tsv +0 -0
  2253. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/number/dot.tsv +0 -0
  2254. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/number/en_digit.tsv +0 -0
  2255. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/number/sign.tsv +0 -0
  2256. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/number/teen.tsv +0 -0
  2257. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/number/zero.tsv +0 -0
  2258. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/pyopenjtalk/transliteration.tsv +0 -0
  2259. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/sport/club.tsv +0 -0
  2260. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/sport/country.tsv +0 -0
  2261. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/time/hour.tsv +0 -0
  2262. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/time/minute.tsv +0 -0
  2263. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/time/noon.tsv +0 -0
  2264. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/data/time/second.tsv +0 -0
  2265. {wetextprocessing-1.0.6.dev19/itn → wetextprocessing-1.3.0/tn}/japanese/rules/char.py +0 -0
  2266. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/test/data/cardinal.txt +0 -0
  2267. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/test/data/char.txt +0 -0
  2268. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/test/data/date.txt +0 -0
  2269. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/test/data/fraction.txt +0 -0
  2270. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/test/data/math.txt +0 -0
  2271. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/test/data/measure.txt +0 -0
  2272. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/test/data/money.txt +0 -0
  2273. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/test/data/sport.txt +0 -0
  2274. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/test/data/time.txt +0 -0
  2275. {wetextprocessing-1.0.6.dev19 → wetextprocessing-1.3.0}/tn/japanese/test/data/whitelist.txt +0 -0
@@ -0,0 +1,91 @@
1
+ name: Android Build
2
+
3
+ on:
4
+ workflow_dispatch:
5
+ pull_request:
6
+ paths:
7
+ # Android project itself, plus the shared C++ runtime it compiles from source.
8
+ - "runtime/android/**"
9
+ - "runtime/bindings/android/**"
10
+ - "runtime/CMakeLists.txt"
11
+ - "runtime/CMakePresets.json"
12
+ - "runtime/processor/**"
13
+ - "runtime/utils/**"
14
+ - "runtime/cmake/**"
15
+ - ".github/workflows/android.yml"
16
+
17
+ jobs:
18
+ build-android:
19
+ name: build (android)
20
+ runs-on: ubuntu-latest
21
+ steps:
22
+ - uses: actions/checkout@v4
23
+
24
+ - uses: actions/setup-java@v4
25
+ with:
26
+ distribution: temurin
27
+ java-version: '21'
28
+
29
+ # Caches downloaded Gradle distribution and dependencies across runs.
30
+ - name: Setup Gradle
31
+ uses: gradle/actions/setup-gradle@v4
32
+ with:
33
+ build-root-directory: runtime/android
34
+
35
+ # Puts sdkmanager on PATH, accepts licenses, installs platform + build-tools.
36
+ - name: Setup Android SDK
37
+ uses: android-actions/setup-android@v3
38
+ with:
39
+ packages: 'platforms;android-36 build-tools;36.0.0'
40
+
41
+ # sdkmanager has no bare "ndk" package — must install "ndk;X.Y.Z". Pick the
42
+ # newest side-by-side NDK from the package list.
43
+ - name: Install NDK (latest)
44
+ run: |
45
+ NDK_PKG=$(sdkmanager --list 2>/dev/null \
46
+ | grep -oE 'ndk;[0-9]+\.[0-9]+\.[0-9]+' \
47
+ | sort -t';' -k2 -V \
48
+ | tail -1)
49
+ if [ -z "$NDK_PKG" ]; then
50
+ echo "ERROR: no ndk;* package found in sdkmanager --list" >&2
51
+ sdkmanager --list 2>/dev/null | grep -i ndk | head -20 >&2 || true
52
+ exit 1
53
+ fi
54
+ echo "Installing $NDK_PKG"
55
+ sdkmanager --install "$NDK_PKG"
56
+ NDK_VER="${NDK_PKG#ndk;}"
57
+ echo "Using NDK $NDK_VER"
58
+ echo "ANDROID_NDK_HOME=$ANDROID_HOME/ndk/$NDK_VER" >> "$GITHUB_ENV"
59
+
60
+ # Build the native lib with CMake presets and install it into jniLibs/<abi>/.
61
+ # Gradle does NOT drive CMake; it only packages the prebuilt .so.
62
+ - name: Build native lib
63
+ working-directory: runtime
64
+ run: |
65
+ cmake --preset android-arm64-v8a
66
+ cmake --build --preset android-arm64-v8a
67
+ cmake --install build/aarch64-linux-android --component jni
68
+
69
+ - uses: actions/setup-python@v5
70
+ with:
71
+ python-version: '3.x'
72
+
73
+ # Generate the four FST models straight into assets/ so the APK ships with
74
+ # them (the app loads zh_tn_* and zh_itn_* at runtime).
75
+ - name: Generate model assets
76
+ run: |
77
+ pip install pynini importlib_resources
78
+ assets=runtime/android/app/src/main/assets
79
+ python -m tn --language zh --overwrite_cache --cache_dir "$assets"
80
+ python -m itn --language zh --overwrite_cache --cache_dir "$assets"
81
+
82
+ - name: Build release APK
83
+ working-directory: runtime/android
84
+ run: ./gradlew :app:assembleRelease -PabiFilters=arm64-v8a --no-daemon
85
+
86
+ - name: Upload release APK
87
+ uses: actions/upload-artifact@v4
88
+ with:
89
+ name: app-release-arm64-v8a
90
+ path: runtime/android/app/build/outputs/apk/release/app-release.apk
91
+ if-no-files-found: error
@@ -0,0 +1,69 @@
1
+ name: Runtime Build
2
+
3
+ on:
4
+ workflow_dispatch:
5
+ pull_request:
6
+ paths:
7
+ - "runtime/**"
8
+ # Android has its own workflow; don't rebuild the desktop runtime for
9
+ # Android-only changes (shared processor/utils/cmake still match above).
10
+ - "!runtime/android/**"
11
+ - "!runtime/bindings/android/**"
12
+ - ".github/workflows/runtime.yml"
13
+
14
+ jobs:
15
+ build:
16
+ runs-on: ${{ matrix.os }}
17
+ strategy:
18
+ fail-fast: false
19
+ matrix:
20
+ os: [ubuntu-latest, macos-latest]
21
+ steps:
22
+ - uses: actions/checkout@v4
23
+ - name: Build runtime
24
+ run: |
25
+ cmake -B build -G Ninja -S runtime -DBUILD_TESTING=ON
26
+ cmake --build build --parallel
27
+ - uses: actions/setup-python@v5
28
+ if: matrix.os == 'ubuntu-latest'
29
+ with:
30
+ python-version: '3.x'
31
+ - name: Install dependencies & Generate TN models
32
+ if: matrix.os == 'ubuntu-latest'
33
+ run: |
34
+ pip install pynini importlib_resources
35
+ python -m tn --language zh --overwrite_cache
36
+ - name: Run tests
37
+ if: matrix.os == 'ubuntu-latest'
38
+ run: ctest --test-dir build/test --output-on-failure
39
+
40
+ build-windows:
41
+ name: build (windows-latest)
42
+ runs-on: windows-latest
43
+ defaults:
44
+ run:
45
+ shell: msys2 {0}
46
+ steps:
47
+ - uses: actions/checkout@v4
48
+
49
+ - uses: msys2/setup-msys2@v2
50
+ with:
51
+ msystem: MINGW64
52
+ update: false
53
+ cache: true
54
+ install: >-
55
+ mingw-w64-x86_64-cmake
56
+ mingw-w64-x86_64-gcc
57
+ mingw-w64-x86_64-ninja
58
+
59
+ - name: Build runtime
60
+ run: |
61
+ cmake --version
62
+ gcc --version
63
+
64
+ # Tests are skipped on Windows: processor_test needs zh_tn_*.fst, which
65
+ # are generated via pynini, and pynini has no Windows build.
66
+ cmake -S runtime -B build -G Ninja \
67
+ -DCMAKE_BUILD_TYPE=Release \
68
+ -DBUILD_TESTING=OFF
69
+ cmake --build build --parallel
@@ -0,0 +1,86 @@
1
+ name: UnitTest
2
+
3
+ on:
4
+ workflow_dispatch:
5
+ pull_request:
6
+ paths:
7
+ - '**.py'
8
+ - '**.tsv'
9
+ - '**.toml'
10
+ - '**.yml'
11
+ - '**.yaml'
12
+ - '.flake8'
13
+ - '.style.yapf'
14
+
15
+ jobs:
16
+ unit-test:
17
+ runs-on: ${{ matrix.os }}
18
+ strategy:
19
+ fail-fast: false
20
+ matrix:
21
+ os: ${{ github.event_name == 'pull_request' && fromJSON('["ubuntu-latest"]') || fromJSON('["ubuntu-latest", "macos-latest", "windows-latest"]') }}
22
+ python-version: ['3.9']
23
+ steps:
24
+ - uses: actions/checkout@v4
25
+ with:
26
+ fetch-depth: 0
27
+ - name: Set up Python ${{ matrix.python-version }} (non-Windows)
28
+ if: runner.os != 'Windows'
29
+ uses: actions/setup-python@v4
30
+ with:
31
+ python-version: ${{ matrix.python-version }}
32
+ - name: Cache Homebrew (macOS)
33
+ if: runner.os == 'macOS'
34
+ uses: actions/cache@v4
35
+ with:
36
+ path: |
37
+ /opt/homebrew/Cellar/openfst
38
+ /opt/homebrew/include/fst
39
+ /opt/homebrew/lib/libfst*
40
+ key: brew-openfst-${{ runner.os }}
41
+ - name: Install openfst (macOS)
42
+ if: runner.os == 'macOS'
43
+ run: |
44
+ brew install openfst || true
45
+ echo "CPLUS_INCLUDE_PATH=/opt/homebrew/include:$CPLUS_INCLUDE_PATH" >> $GITHUB_ENV
46
+ echo "LIBRARY_PATH=/opt/homebrew/lib:$LIBRARY_PATH" >> $GITHUB_ENV
47
+ - name: Cache pip (non-Windows)
48
+ if: runner.os != 'Windows'
49
+ uses: actions/cache@v4
50
+ with:
51
+ path: ~/.cache/pip
52
+ key: pip-${{ runner.os }}-${{ hashFiles('pyproject.toml') }}
53
+ - name: Install dependencies (non-Windows)
54
+ if: runner.os != 'Windows'
55
+ run: |
56
+ python -m pip install --upgrade pip
57
+ pip install .[test]
58
+ - name: Set up Conda (Windows)
59
+ if: runner.os == 'Windows'
60
+ uses: conda-incubator/setup-miniconda@v3
61
+ with:
62
+ python-version: ${{ matrix.python-version }}
63
+ activate-environment: test
64
+ use-only-tar-bz2: true
65
+ - name: Cache Conda (Windows)
66
+ if: runner.os == 'Windows'
67
+ uses: actions/cache@v4
68
+ with:
69
+ path: C:\Miniconda\envs\test
70
+ key: conda-${{ runner.os }}-${{ hashFiles('pyproject.toml') }}
71
+ - name: Install dependencies (Windows)
72
+ if: runner.os == 'Windows'
73
+ shell: bash -el {0}
74
+ run: |
75
+ conda install -c conda-forge pynini
76
+ python -m pip install --upgrade pip
77
+ pip install .[test] --no-deps
78
+ pip install flake8 pytest importlib_resources
79
+ - name: Lint with flake8
80
+ shell: bash -el {0}
81
+ run: |
82
+ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
83
+ flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
84
+ - name: Test with pytest
85
+ shell: bash -el {0}
86
+ run: pytest
@@ -0,0 +1,74 @@
1
+ name: Build Wheels
2
+
3
+ on:
4
+ workflow_dispatch:
5
+ inputs:
6
+ version:
7
+ description: 'Release version (optional, will be determined by setuptools_scm if not provided)'
8
+ required: false
9
+
10
+ jobs:
11
+ build:
12
+ runs-on: ubuntu-latest
13
+ strategy:
14
+ matrix:
15
+ python-version: [3.9]
16
+ steps:
17
+ - uses: actions/checkout@v4
18
+ with:
19
+ fetch-depth: 0
20
+ - name: Set up Python ${{ matrix.python-version }}
21
+ uses: actions/setup-python@v5
22
+ with:
23
+ python-version: ${{ matrix.python-version }}
24
+ - name: Install dependencies
25
+ run: |
26
+ python -m pip install --upgrade pip
27
+ pip install .[test]
28
+ pip install build twine
29
+
30
+ - name: Build Graph
31
+ run: |
32
+ python -m tn --text "2.5平方电线" --overwrite_cache --language "zh"
33
+ python -m tn --text "2010-03-21" --overwrite_cache --language "en"
34
+ python -m itn --text "二点五平方电线" --overwrite_cache
35
+ python -m itn --text "one two three" --overwrite_cache --language "en"
36
+
37
+ - name: Prepare Graph
38
+ run: |
39
+ mkdir graph
40
+ cp tn/*.fst graph
41
+ cp itn/*.fst graph
42
+ cp tn/*_cache.json graph
43
+ cp itn/*_cache.json graph
44
+
45
+ - name: Get version from setuptools_scm
46
+ id: scm_version
47
+ run: |
48
+ # Check if version is provided as input
49
+ if [ -n "${{ github.event.inputs.version }}" ]; then
50
+ echo "VERSION=${{ github.event.inputs.version }}" >> $GITHUB_OUTPUT
51
+ else
52
+ # Get version from setuptools_scm
53
+ pip install setuptools_scm
54
+ VERSION=$(python -m setuptools_scm | tail -1)
55
+ echo "VERSION=$VERSION" >> $GITHUB_OUTPUT
56
+ fi
57
+
58
+ - name: Upload Graph
59
+ uses: actions/upload-artifact@v4
60
+ with:
61
+ name: release-graph-v${{ steps.scm_version.outputs.VERSION }}
62
+ path: graph
63
+
64
+ - name: Build package
65
+ run: |
66
+ python -m build
67
+
68
+ - name: Publish on pypi.org
69
+ if: github.ref == 'refs/heads/master' && github.repository == 'wenet-e2e/WeTextProcessing'
70
+ env:
71
+ TWINE_USERNAME: __token__
72
+ TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
73
+ run: |
74
+ python -m twine upload dist/*
@@ -0,0 +1,48 @@
1
+ # Byte-compiled / optimized / DLL files
2
+ __pycache__/
3
+ *.py[cod]
4
+ *$py.class
5
+
6
+ # Visual Studio Code files
7
+ .vscode
8
+ .vs
9
+
10
+ # PyCharm files
11
+ .idea
12
+
13
+ # Eclipse Project settings
14
+ *.*project
15
+ .settings
16
+
17
+ # Sublime Text settings
18
+ *.sublime-workspace
19
+ *.sublime-project
20
+
21
+ # Editor temporaries
22
+ *.swn
23
+ *.swo
24
+ *.swp
25
+ *.swm
26
+ *~
27
+
28
+ # IPython notebook checkpoints
29
+ .ipynb_checkpoints
30
+
31
+ # macOS dir files
32
+ .DS_Store
33
+
34
+ # Clangd files
35
+ .cache
36
+ .venv/
37
+ compile_commands.json
38
+
39
+
40
+ # Setup files
41
+ WeTextProcessing.egg-info/
42
+ build/
43
+ dist/
44
+ tn/*.far
45
+ itn/*.far
46
+
47
+ # Version file(setuptools_scm generated)
48
+ tn/_version.py
@@ -0,0 +1,23 @@
1
+ exclude: '.*\.(txt|tsv)$'
2
+ repos:
3
+ - repo: https://github.com/pre-commit/pre-commit-hooks
4
+ rev: v4.5.0
5
+ hooks:
6
+ - id: trailing-whitespace
7
+ - repo: https://github.com/google/yapf
8
+ rev: 'v0.43.0'
9
+ hooks:
10
+ - id: yapf
11
+ - repo: https://github.com/pycqa/flake8
12
+ rev: '7.3.0'
13
+ hooks:
14
+ - id: flake8
15
+ - repo: https://github.com/pre-commit/mirrors-clang-format
16
+ rev: 'v17.0.6'
17
+ hooks:
18
+ - id: clang-format
19
+ exclude: '.*\.(json|java|js|m|mm|proto)'
20
+ - repo: https://github.com/cpplint/cpplint
21
+ rev: '1.6.1'
22
+ hooks:
23
+ - id: cpplint
@@ -0,0 +1,3 @@
1
+ [style]
2
+ based_on_style = pep8
3
+ column_limit = 127
@@ -0,0 +1,290 @@
1
+ Metadata-Version: 2.4
2
+ Name: WeTextProcessing
3
+ Version: 1.3.0
4
+ Summary: WeTextProcessing, including TN & ITN
5
+ Author-email: Zhendong Peng <pzd17@tsinghua.org.cn>, Xingchen Song <sxc19@tsinghua.org.cn>
6
+ Project-URL: Homepage, https://github.com/wenet-e2e/WeTextProcessing
7
+ Classifier: Programming Language :: Python :: 3
8
+ Classifier: Operating System :: OS Independent
9
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
10
+ Requires-Python: >=3.7
11
+ Description-Content-Type: text/markdown
12
+ License-File: LICENSE
13
+ Requires-Dist: pynini>=2.1.6
14
+ Requires-Dist: importlib_resources
15
+ Provides-Extra: test
16
+ Requires-Dist: pytest; extra == "test"
17
+ Requires-Dist: flake8; extra == "test"
18
+ Provides-Extra: dev
19
+ Requires-Dist: flake8; extra == "dev"
20
+ Requires-Dist: pre-commit==3.5.0; extra == "dev"
21
+ Requires-Dist: pytest; extra == "dev"
22
+ Dynamic: license-file
23
+
24
+ <div align="center">
25
+
26
+ # WeTextProcessing
27
+
28
+ **Rule-based text normalization and inverse text normalization for speech applications.**
29
+
30
+ [Quick start](#quick-start) · [Languages](#supported-languages) · [Python API](#python-api) · [Rule architecture](docs/python-rule-architecture.md) · [C++ runtime](runtime/README.md)
31
+
32
+ [![Unit tests](https://github.com/wenet-e2e/WeTextProcessing/actions/workflows/unittest.yml/badge.svg)](https://github.com/wenet-e2e/WeTextProcessing/actions/workflows/unittest.yml)
33
+
34
+ </div>
35
+
36
+ WeTextProcessing converts between written and spoken text using weighted finite-state transducers (WFSTs), built with [Pynini](https://www.openfst.org/twiki/bin/view/GRM/Pynini) and [OpenFst](https://www.openfst.org/twiki/bin/view/FST/WebHome).
37
+
38
+ - **TN and ITN** — normalize numbers, dates, times, currencies, units, and other structured expressions.
39
+ - **Multilingual rules** — language-specific grammars for every language in [Supported languages](#supported-languages).
40
+ - **Explainable output** — inspect raw tagged fields, trace input/output spans, and request unique n-best results.
41
+ - **Reusable graphs** — automatic, configuration-aware FST caching and a separate C++ deployment runtime.
42
+
43
+ For background and motivation, read [WeTextProcessing: Production First & Production Ready Text Processing Toolkit](https://mp.weixin.qq.com/s/q_11lck78qcjylHCi6wVsQ) (Chinese).
44
+
45
+ ## What are TN and ITN?
46
+
47
+ ### Text normalization
48
+
49
+ TN converts written expressions into their spoken forms, for example `12` → `十二`. It is commonly used before speech synthesis.
50
+
51
+ <div align=center><img src="https://user-images.githubusercontent.com/13466943/193439861-acfba531-13d1-4fca-b2f2-6e47fc10f195.png" alt="Cover" width="50%"/></div>
52
+
53
+ ### Inverse text normalization
54
+
55
+ ITN converts spoken expressions into written forms, for example `十二` → `12`. It is commonly used after speech recognition.
56
+
57
+ <div align=center><img src="https://user-images.githubusercontent.com/13466943/193439870-634c44a3-bd62-4311-bcf2-1427758d5f62.png" alt="Cover" width="50%"/></div>
58
+
59
+ ## Supported languages
60
+
61
+ | Language | CLI code | TN | ITN | Python package |
62
+ | --- | --- | --- | --- | --- |
63
+ | Arabic | `ar` | Yes | Yes | `arabic` |
64
+ | Armenian | `hy` | Yes | Yes | `armenian` |
65
+ | Chinese | `zh` | Yes | Yes | `chinese` |
66
+ | English | `en` | Yes | Yes | `english` |
67
+ | French | `fr` | Yes | Yes | `french` |
68
+ | German | `de` | Yes | Yes | `german` |
69
+ | Hebrew | `he` | Yes | Yes | `hebrew` |
70
+ | Hindi | `hi` | Yes | Yes | `hindi` |
71
+ | Hindi–English (code-mixed) | `hi_en` | No | Yes | `hindi_english` |
72
+ | Hungarian | `hu` | Yes | Yes | `hungarian` |
73
+ | Italian | `it` | Yes | Yes | `italian` |
74
+ | Japanese | `ja` | Yes | Yes | `japanese` |
75
+ | Kinyarwanda | `rw` | Yes | Yes | `kinyarwanda` |
76
+ | Korean | `ko` | Yes | Yes | `korean` |
77
+ | Marathi | `mr` | Yes | Yes | `marathi` |
78
+ | Portuguese | `pt` | Yes | Yes | `portuguese` |
79
+ | Russian | `ru` | Yes | Yes | `russian` |
80
+ | Spanish | `es` | Yes | Yes | `spanish` |
81
+ | Spanish–English (code-mixed) | `es_en` | No | Yes | `spanish_english` |
82
+ | Swedish | `sv` | Yes | Yes | `swedish` |
83
+ | Vietnamese | `vi` | Yes | Yes | `vietnamese` |
84
+
85
+ Chinese is the CLI default. Rule coverage and configurable options vary by language and direction.
86
+
87
+ The published package includes Chinese, English, and Japanese. The other languages are included in this source checkout; install from source to use them before the next package release.
88
+
89
+ ## Quick start
90
+
91
+ ### Install
92
+
93
+ Install the published package:
94
+
95
+ ```bash
96
+ pip install WeTextProcessing
97
+ ```
98
+
99
+ For the latest rules, including every language beyond Chinese, English, and Japanese, install from source:
100
+
101
+ ```bash
102
+ git clone https://github.com/wenet-e2e/WeTextProcessing.git
103
+ cd WeTextProcessing
104
+ pip install -e .
105
+ ```
106
+
107
+ Pynini/OpenFst must be available in your Python environment. If a compatible wheel is unavailable, follow the [Pynini installation instructions](https://www.openfst.org/twiki/bin/view/GRM/Pynini). On Windows, the project's CI uses `conda install -c conda-forge pynini` before installing WeTextProcessing.
108
+
109
+ ### Run the CLI
110
+
111
+ ```bash
112
+ wetn --text "2.5平方电线"
113
+ weitn --text "二点五平方电线"
114
+
115
+ wetn --language en --text "13 dollars"
116
+ weitn --language en --text "thirteen dollars"
117
+
118
+ wetn --language de --text "23"
119
+ weitn --language de --text "dreiundzwanzig"
120
+
121
+ wetn --language ja --text "12時"
122
+ weitn --language ja --text "十二時"
123
+
124
+ wetn --language ko --text "오후 3시 30분"
125
+ weitn --language ko --text "오천원"
126
+ ```
127
+
128
+ Each input produces two lines: the **tagged representation**, then the **verbalized result**. The tagger preserves raw field values; the verbalizer performs the conversion.
129
+
130
+ The same interfaces are available as `python -m tn` and `python -m itn`. Both accept `--text`, `--file PATH`, or UTF-8 text from standard input when neither option is supplied:
131
+
132
+ ```bash
133
+ python -m tn --language en --file input.txt
134
+ python -m itn --language ko < input.txt
135
+ ```
136
+
137
+ Only input line endings are removed before processing. Other spaces are passed to the selected language's rules.
138
+
139
+ ### Language-specific options
140
+
141
+ Inspect the options for the selected language and direction:
142
+
143
+ ```bash
144
+ wetn --language zh --help
145
+ weitn --language ko --help
146
+ ```
147
+
148
+ For example:
149
+
150
+ ```bash
151
+ wetn --language zh --no-remove-erhua --text "这儿"
152
+ weitn --language zh --enable-0-to-9 --text "一二三"
153
+ ```
154
+
155
+ Boolean options accept `--option` / `--no-option` and the legacy `--option True|False` form. Options from another language or direction are not accepted.
156
+
157
+ ## Python API
158
+
159
+ Import the normalizer for the language you need and reuse the instance:
160
+
161
+ ```python
162
+ from tn.chinese.normalizer import Normalizer
163
+ from itn.chinese.inverse_normalizer import InverseNormalizer
164
+
165
+ normalizer = Normalizer()
166
+ inverse_normalizer = InverseNormalizer()
167
+
168
+ print(normalizer.normalize("今天中午12点"))
169
+ print(inverse_normalizer.normalize("二点五平方电线"))
170
+ ```
171
+
172
+ Other languages use the same class names under their packages. For example, Korean:
173
+
174
+ ```python
175
+ from tn.korean.normalizer import Normalizer
176
+ from itn.korean.inverse_normalizer import InverseNormalizer
177
+
178
+ normalizer = Normalizer()
179
+ inverse_normalizer = InverseNormalizer()
180
+
181
+ print(normalizer.normalize("오후 3시 30분"))
182
+ print(inverse_normalizer.normalize("오천원"))
183
+ ```
184
+
185
+ ### Inspect tags
186
+
187
+ Tagging and verbalization are independently accessible:
188
+
189
+ ```python
190
+ tagged = normalizer.tag("오후 3시 30분")
191
+ print(tagged)
192
+ print(normalizer.verbalize(tagged))
193
+ ```
194
+
195
+ The hours and minutes remain written values in the tags, rather than being replaced with their Korean readings.
196
+
197
+ ### Input/output span mapping
198
+
199
+ `normalize_with_mapping()` returns normalized text and the spans matched by grammar rules:
200
+
201
+ ```python
202
+ from tn.chinese.normalizer import Normalizer
203
+
204
+ normalizer = Normalizer()
205
+ result = normalizer.normalize_with_mapping("今天中午12点")
206
+
207
+ print(result.output_text)
208
+ for mapping in result.mappings:
209
+ print(mapping.token_type, mapping.input_text, "=>", mapping.output_text)
210
+ ```
211
+
212
+ Output:
213
+
214
+ ```text
215
+ 今天中午十二点
216
+ math 12 => 十二
217
+ ```
218
+
219
+ Offsets are half-open Unicode character offsets, not UTF-8 byte offsets. Use `include_identity=True` to include unchanged tokens, or `result.as_dict()` to obtain a serializable representation. Mappings are traced through the WFSTs, not inferred from a text diff.
220
+
221
+ ### N-best results
222
+
223
+ ```python
224
+ outputs = normalizer.normalize("3/4", nbest=3)
225
+ results = normalizer.normalize_with_mapping("3/4", nbest=3)
226
+ ```
227
+
228
+ `nbest=1` returns a string or a single `NormalizationResult`; `nbest>1` returns a list of up to that many unique outputs. The mapping API follows the same output ranking.
229
+
230
+ ## FST caching
231
+
232
+ Graphs are cached automatically outside the source tree. Cache bundles record the pipeline's configuration and grammar-source fingerprint; changing either causes a new bundle to be built.
233
+
234
+ | Setting | Behavior |
235
+ | --- | --- |
236
+ | `cache_dir=None` | Use the platform's default user cache |
237
+ | `cache_dir="/path/to/cache"` | Choose an explicit cache root |
238
+ | `cache_dir=False` | Build in memory without persistent caching |
239
+ | `overwrite_cache=True` | Force rebuilding the selected bundle |
240
+
241
+ ```python
242
+ from tn.chinese.normalizer import Normalizer
243
+
244
+ normalizer = Normalizer(cache_dir="/path/to/wetext-cache")
245
+ uncached = Normalizer(cache_dir=False)
246
+ ```
247
+
248
+ The CLI exposes `--cache-dir PATH` and `--overwrite-cache`. Normal use does not require forced rebuilds.
249
+
250
+ ## Development and deployment
251
+
252
+ Install development dependencies and run the tests:
253
+
254
+ ```bash
255
+ pip install -e '.[dev]'
256
+ pre-commit install
257
+ pytest -q
258
+ ```
259
+
260
+ Before adding or changing a language rule, read the [Python rule architecture guide](docs/python-rule-architecture.md). Rules preserve raw fields in the tagger, apply semantic conversion in the verbalizer, and use pipeline-owned rule inventories and field-order schemas.
261
+
262
+ | Guide | Scope |
263
+ | --- | --- |
264
+ | [TN rules](tn/README.md) | Text normalization rules and examples |
265
+ | [ITN rules](itn/README.md) | Inverse text normalization rules and examples |
266
+ | [Python rule architecture](docs/python-rule-architecture.md) | Tagging, verbalization, registration, and span mapping contracts |
267
+ | [C++ runtime](runtime/README.md) | Native build and deployment instructions |
268
+
269
+ Python cache bundles are **not** C++ runtime exports. Follow the runtime's model and build instructions for native deployment.
270
+
271
+ ## Community
272
+
273
+ Use [GitHub Issues](https://github.com/wenet-e2e/WeTextProcessing/issues) for questions, bug reports, and feature requests.
274
+
275
+ For Chinese users, follow the WeNet official account using the QR code on the left. Scan the personal QR code on the right to contact the person who can invite you to the WeChat discussion group.
276
+
277
+ | <img src="https://github.com/robin1001/qr/blob/master/wenet.jpeg" width="250px"> | <img src="https://user-images.githubusercontent.com/13466943/203046432-f637180e-4c87-40cc-be05-ce48c65dd1ef.jpg" width="250px"> |
278
+ | ---- | ---- |
279
+
280
+ ## Acknowledgments
281
+
282
+ - [OpenFst](https://www.openfst.org/twiki/bin/view/FST/WebHome) and [Pynini](https://www.openfst.org/twiki/bin/view/GRM/Pynini) for the finite-state toolkit.
283
+ - The [NeMo Text Processing](https://github.com/NVIDIA/NeMo-text-processing) team and community for text-processing grammars and resources, including the [Chinese TN reference](https://github.com/NVIDIA/NeMo-text-processing/tree/main/nemo_text_processing/text_normalization/zh).
284
+ - [Zhenxiang Ma](https://github.com/mzxcpp), [Jiayu Du](https://github.com/dophist), and [SpeechColab](https://github.com/SpeechColab) for their contributions.
285
+ - [Pynini](https://github.com/kylebgorman/pynini) for reference implementations of FAR loading and shortest-path decoding in the C++ runtime.
286
+ - [chinese_text_normalization](https://github.com/speechio/chinese_text_normalization/tree/master/thrax/src/cn) for Chinese ITN grammar resources.
287
+
288
+ ## License
289
+
290
+ [Apache License 2.0](LICENSE).