sd-encoder 0.1.2__tar.gz → 0.1.4__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/Cargo.lock +1 -1
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/Cargo.toml +1 -1
- sd_encoder-0.1.4/LICENSE.md +21 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/PKG-INFO +9 -10
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/README.md +6 -8
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/pyproject.toml +1 -1
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/.gitignore +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/.rustfmt.toml +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/Makefile +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/api_response +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/benches/load.rs +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/examples/load_profile.rs +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/python/sd_encoder/__init__.py +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/python/sd_encoder/_sd_encoder.pyi +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/python/sd_encoder/py.typed +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/scripts/test_local_build.sh +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/src/encoder.rs +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/src/lib.rs +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/src/py/config.rs +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/src/py/encoder.rs +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/src/py/enums.rs +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/src/py/mod.rs +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/src/py/output.rs +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/src/types.rs +0 -0
- {sd_encoder-0.1.2 → sd_encoder-0.1.4}/uv.lock +0 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Yanick Andrade
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -1,13 +1,14 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: sd_encoder
|
|
3
|
-
Version: 0.1.
|
|
3
|
+
Version: 0.1.4
|
|
4
4
|
Classifier: Programming Language :: Rust
|
|
5
5
|
Classifier: Programming Language :: Python :: Implementation :: CPython
|
|
6
6
|
Classifier: Programming Language :: Python :: Implementation :: PyPy
|
|
7
|
+
License-File: LICENSE.md
|
|
7
8
|
Summary: Rust-accelerated structured data encoder for LLM token compression
|
|
8
9
|
Keywords: Natural Language Compression,LLM Compressor,Compressed Language Model,CLM,NLP,Text Compression,Structure Data Compression,SDEncoder
|
|
9
10
|
Home-Page: https://yanickjair.github.io/cllm/crates/sd_encoder/
|
|
10
|
-
Author-email: Yanick Andrade <
|
|
11
|
+
Author-email: Yanick Andrade <info@clm-lang.com>
|
|
11
12
|
Requires-Python: >=3.8
|
|
12
13
|
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
|
|
13
14
|
|
|
@@ -21,7 +22,7 @@ Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
|
|
|
21
22
|
<p align="center">
|
|
22
23
|
<a href="https://github.com/YanickJair/cllm/actions"><img src="https://github.com/YanickJar/cllm/workflows/Test%20Suite/badge.svg" alt="Test Suite"></a>
|
|
23
24
|
<a href="https://pypi.org/project/sd-encoder/"><img src="https://img.shields.io/pypi/v/sd-encoder.svg" alt="PyPI"></a>
|
|
24
|
-
<a href="https://github.com/YanickJar/cllm/blob/main/LICENSE
|
|
25
|
+
<a href="https://github.com/YanickJar/cllm/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"></a>
|
|
25
26
|
</p>
|
|
26
27
|
|
|
27
28
|
<p align="center"><em>Compress structured data into compact token sequences — 40–85% fewer tokens, no model retraining, no heavy dependencies.</em></p>
|
|
@@ -227,10 +228,10 @@ encoder.encode_validated({"id": 1, "title": "Test", "internal_log": "...", "raw"
|
|
|
227
228
|
|
|
228
229
|
## Relationship to CLM
|
|
229
230
|
|
|
230
|
-
`sd-encoder` is the engine behind the Structured Data encoder in [CLM](https://pypi.org/project/clm-core/). If you need thread or system prompt encoding alongside structured data, install the full library
|
|
231
|
+
`sd-encoder` is the engine behind the Structured Data encoder in [CLM](https://pypi.org/project/clm-core/). If you need thread or system prompt encoding alongside structured data, install the full library with the `sd_encoder` extra:
|
|
231
232
|
|
|
232
233
|
```bash
|
|
233
|
-
pip install clm-core
|
|
234
|
+
pip install "clm-core[sd_encoder]"
|
|
234
235
|
```
|
|
235
236
|
|
|
236
237
|
`sd-encoder` is the right choice when:
|
|
@@ -243,16 +244,14 @@ pip install clm-core
|
|
|
243
244
|
|
|
244
245
|
## License
|
|
245
246
|
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
- **AGPL-3.0** — free for open source use ([LICENSE-AGPL](LICENSE-AGPL))
|
|
249
|
-
- **Commercial** — for proprietary products and SaaS ([contact](mailto:yanick.jair.ta@gmail.com))
|
|
247
|
+
- **MIT License** — License © 2025-PRESENT ([MIT License](LMIT License))
|
|
248
|
+
(mailto:info@clm-lang.com))
|
|
250
249
|
|
|
251
250
|
---
|
|
252
251
|
|
|
253
252
|
<p align="center">
|
|
254
253
|
<a href="https://github.com/YanickJair/cllm/issues">Issues</a> ·
|
|
255
254
|
<a href="https://github.com/YanickJair/cllm/discussions">Discussions</a> ·
|
|
256
|
-
<a href="mailto:
|
|
255
|
+
<a href="mailto:info@clm-lang.com">Contact</a>
|
|
257
256
|
</p>
|
|
258
257
|
|
|
@@ -8,7 +8,7 @@
|
|
|
8
8
|
<p align="center">
|
|
9
9
|
<a href="https://github.com/YanickJair/cllm/actions"><img src="https://github.com/YanickJar/cllm/workflows/Test%20Suite/badge.svg" alt="Test Suite"></a>
|
|
10
10
|
<a href="https://pypi.org/project/sd-encoder/"><img src="https://img.shields.io/pypi/v/sd-encoder.svg" alt="PyPI"></a>
|
|
11
|
-
<a href="https://github.com/YanickJar/cllm/blob/main/LICENSE
|
|
11
|
+
<a href="https://github.com/YanickJar/cllm/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"></a>
|
|
12
12
|
</p>
|
|
13
13
|
|
|
14
14
|
<p align="center"><em>Compress structured data into compact token sequences — 40–85% fewer tokens, no model retraining, no heavy dependencies.</em></p>
|
|
@@ -214,10 +214,10 @@ encoder.encode_validated({"id": 1, "title": "Test", "internal_log": "...", "raw"
|
|
|
214
214
|
|
|
215
215
|
## Relationship to CLM
|
|
216
216
|
|
|
217
|
-
`sd-encoder` is the engine behind the Structured Data encoder in [CLM](https://pypi.org/project/clm-core/). If you need thread or system prompt encoding alongside structured data, install the full library
|
|
217
|
+
`sd-encoder` is the engine behind the Structured Data encoder in [CLM](https://pypi.org/project/clm-core/). If you need thread or system prompt encoding alongside structured data, install the full library with the `sd_encoder` extra:
|
|
218
218
|
|
|
219
219
|
```bash
|
|
220
|
-
pip install clm-core
|
|
220
|
+
pip install "clm-core[sd_encoder]"
|
|
221
221
|
```
|
|
222
222
|
|
|
223
223
|
`sd-encoder` is the right choice when:
|
|
@@ -230,15 +230,13 @@ pip install clm-core
|
|
|
230
230
|
|
|
231
231
|
## License
|
|
232
232
|
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
- **AGPL-3.0** — free for open source use ([LICENSE-AGPL](LICENSE-AGPL))
|
|
236
|
-
- **Commercial** — for proprietary products and SaaS ([contact](mailto:yanick.jair.ta@gmail.com))
|
|
233
|
+
- **MIT License** — License © 2025-PRESENT ([MIT License](LMIT License))
|
|
234
|
+
(mailto:info@clm-lang.com))
|
|
237
235
|
|
|
238
236
|
---
|
|
239
237
|
|
|
240
238
|
<p align="center">
|
|
241
239
|
<a href="https://github.com/YanickJair/cllm/issues">Issues</a> ·
|
|
242
240
|
<a href="https://github.com/YanickJair/cllm/discussions">Discussions</a> ·
|
|
243
|
-
<a href="mailto:
|
|
241
|
+
<a href="mailto:info@clm-lang.com">Contact</a>
|
|
244
242
|
</p>
|
|
@@ -12,7 +12,7 @@ classifiers = [
|
|
|
12
12
|
"Programming Language :: Python :: Implementation :: PyPy",
|
|
13
13
|
]
|
|
14
14
|
authors = [
|
|
15
|
-
{ name = "Yanick Andrade", email = "
|
|
15
|
+
{ name = "Yanick Andrade", email = "info@clm-lang.com" },
|
|
16
16
|
]
|
|
17
17
|
keywords = [
|
|
18
18
|
"Natural Language Compression",
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|