tokenizers 0.2.1-arm64-darwin → 0.2.2-arm64-darwin

Sign up to get free protection for your applications and to get access to all the features.
Files changed (14) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +6 -0
  3. data/Cargo.lock +124 -1252
  4. data/Cargo.toml +0 -5
  5. data/LICENSE-THIRD-PARTY.txt +6585 -23511
  6. data/README.md +1 -1
  7. data/lib/tokenizers/2.7/tokenizers.bundle +0 -0
  8. data/lib/tokenizers/3.0/tokenizers.bundle +0 -0
  9. data/lib/tokenizers/3.1/tokenizers.bundle +0 -0
  10. data/lib/tokenizers/3.2/tokenizers.bundle +0 -0
  11. data/lib/tokenizers/from_pretrained.rb +119 -0
  12. data/lib/tokenizers/version.rb +1 -1
  13. data/lib/tokenizers.rb +2 -3
  14. metadata +3 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b63433567c1b129ad7bebac0cdd911f95afa44f376160fa85d29233968e2dcb4
4
- data.tar.gz: '00489683ead88d495b9bb3b49443f14420e79cd68444c2275c2ea46a9f6a7df4'
3
+ metadata.gz: 73ca7206fd54ae0198f9c843e7cca055538d317d039bf0451ab785b610168187
4
+ data.tar.gz: 46b6980dc05a23adc07fdc36cd99b5a342a421e9fc8498de51c17661914f763f
5
5
  SHA512:
6
- metadata.gz: 4f7fd3ae109c040f5b71b17eba9bb66a4ac3619c1c4ad94f88c7dd3d7c710bfc2ca95d7e80b2bb5f08eb0e0c799116acf5f643c616961934519d0f1601b47e3a
7
- data.tar.gz: 4b9ffb5ab007ad97166980f195006f5f4b3c845420cb4da960d2aa611361be8d051d97d245037cc04b539487b386e167dc9321520130e0c1ed6e195f31d65c35
6
+ metadata.gz: 1a009db58b4dcbd1bc4e171aa35ac24d33ad5808089c67fb8a366c657863d55eb1cbc23e022a64603f75fb8bc4f12e5bff1f8f743097216889baeb382b372af2
7
+ data.tar.gz: fd87078b0e929829b1ad6d746213e7d0c73d6716672fea48e9ac6365c7aca67544214bcae8d5d009db8b6654773ecb9ffe4cadeecd237dc3958ba2f257e64185
data/CHANGELOG.md CHANGED
@@ -1,3 +1,9 @@
1
+ ## 0.2.2 (2022-01-15)
2
+
3
+ - Added precompiled gem for Linux ARM
4
+ - Added `from_file` method
5
+ - Fixed error with precompiled gem on Linux x86-64
6
+
1
7
  ## 0.2.1 (2022-01-12)
2
8
 
3
9
  - Added support for Ruby 3.2