tokenizers 0.2.1-x86_64-darwin → 0.2.2-x86_64-darwin

Sign up to get free protection for your applications and to get access to all the features.
Files changed (14) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +6 -0
  3. data/Cargo.lock +124 -1252
  4. data/Cargo.toml +0 -5
  5. data/LICENSE-THIRD-PARTY.txt +6585 -23511
  6. data/README.md +1 -1
  7. data/lib/tokenizers/2.7/tokenizers.bundle +0 -0
  8. data/lib/tokenizers/3.0/tokenizers.bundle +0 -0
  9. data/lib/tokenizers/3.1/tokenizers.bundle +0 -0
  10. data/lib/tokenizers/3.2/tokenizers.bundle +0 -0
  11. data/lib/tokenizers/from_pretrained.rb +119 -0
  12. data/lib/tokenizers/version.rb +1 -1
  13. data/lib/tokenizers.rb +2 -3
  14. metadata +3 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 7567ef43442e5678528c5af15bb8766f174fed057dcb78a675294764e125ab4a
4
- data.tar.gz: f78e79113cfd20842c31e7423983b822874d31e12bc1e8d86b7f2c19ef38d9de
3
+ metadata.gz: cd9f4e13c08d5a2bbc017d0a297de6be483f1f894e3089a8c2a87bcddf09eb92
4
+ data.tar.gz: 144af0a6041863212aa44a42729e9d721b78f09862943d339c2aef695728b88d
5
5
  SHA512:
6
- metadata.gz: c189c48041813827f7de41402999cda3db29c388aeec0213ac319114e89760f181ba33b3ecd9e14fef58416b934954fcd979464713884caca8acf0a0a299708f
7
- data.tar.gz: 6eddee9c6d0d27035e0a9fb66c4dde408912d923afb29c0a6cc15be1b17146a675412f6fa600f0fe0e2b256a11369e6137f13303e715ff123a4b83567f78e2f1
6
+ metadata.gz: 0ee668f662c8c30aea39f618745e97810ec7a2c017f6a8cf8022d595ca7edfc7a9eb03bcf5cecb23e5035c0d36f5c3d656af7e7b615827109b07917c8b7a340f
7
+ data.tar.gz: a3c1dbe246a323ae8ff3e6572fa610a0e1ccf025e178f49956bf54616511e776dfead2da6a36a6a064a27fdd81ad441f6601c755c415e10df3009f615274b5f3
data/CHANGELOG.md CHANGED
@@ -1,3 +1,9 @@
1
+ ## 0.2.2 (2022-01-15)
2
+
3
+ - Added precompiled gem for Linux ARM
4
+ - Added `from_file` method
5
+ - Fixed error with precompiled gem on Linux x86-64
6
+
1
7
  ## 0.2.1 (2022-01-12)
2
8
 
3
9
  - Added support for Ruby 3.2