sie-server 0.8.2__tar.gz → 0.8.3__tar.gz

{sie_server-0.8.2 → sie_server-0.8.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: sie-server
-Version: 0.8.2
+Version: 0.8.3
 Summary: Search Inference Engine - GPU inference server for search workloads
 License: Apache-2.0
 License-File: LICENSE
@@ -10,9 +10,10 @@ Requires-Dist: docling<2.100,>=2.99
 Requires-Dist: einops<1,>=0.8
 Requires-Dist: fastapi<1,>=0.115
 Requires-Dist: flagembedding<2,>=1.3
-Requires-Dist: gliclass<1,>=0.1
+Requires-Dist: gliclass<1,>=0.1.17
+Requires-Dist: gliformer==0.1.2
 Requires-Dist: gliner2<2,>=1.3.1
-Requires-Dist: gliner<1,>=0.2
+Requires-Dist: gliner<1,>=0.2.26
 Requires-Dist: glirel<2,>=1.0
 Requires-Dist: httpx>=0.28.1
 Requires-Dist: huggingface-hub<2,>=0.26

sie_server-0.8.3/README.md

@@ -0,0 +1,273 @@
+# SIE Server
+
+GPU inference server for embeddings, reranking, and entity extraction.
+
+## Features
+
+- Multi-model serving with LRU eviction
+- Token-based dynamic batching
+- Hot reload model configs without restart
+- Unified API: `encode()`, `score()`, `extract()`
+- Prometheus metrics and OpenTelemetry tracing
+- Reactive GPU OOM recovery + proactive idle eviction
+
+## Installation
+
+```bash
+pip install sie-server
+```
+
+`sie-server` ships several model bundles, and a single environment can only hold one
+`transformers` version — install the one your bundle needs:
+
+- **Default bundle** (embeddings, reranking, extraction) — verified on `transformers` 4.x:
+
+  ```bash
+  pip install sie-server "transformers<5"
+  ```
+
+- **Transformers 5 bundle** (LightOnOCR, GLM-OCR, GLiGuard, and the GLiNER2.5-Decide models) — requires
+  `transformers` 5.x, and is served with `-b transformers5`. The GLiNER2.5-Decide models also need `gliner2`
+  2.x. `sie-server` itself asks for `gliner2<2`, which the default bundle's GLiNER2 models need, so pip
+  reports that conflict when the second command below installs 2.x; the transformers5 bundle's GLiNER2
+  models are verified on 2.0.0:
+
+  ```bash
+  pip install sie-server "transformers>=5,<6"
+  pip install "gliner2==2.0.0"  # only for the GLiNER2.5-Decide models
+  sie-server serve -b transformers5
+  ```
+
+## Quick Start
+
+```bash
+sie-server serve --port 8080 --device cuda:0
+```
+
+### TensorRT-LLM generation
+
+TensorRT-LLM buffers completion token IDs and emits the final text together only
+after generation finishes and the terminal stream is verified, even with
+`stream=True`. This preserves context-sensitive spacing and punctuation and
+removes matched stop sequences consistently from returned text, completion-token
+usage, and logprobs. Long completions delay the first visible text; existing
+request timeouts still apply. Other adapters are unaffected.
+
+### GLiClass usage
+
+For GLiClass classification, `usage.input_tokens` counts each item's document
+tokens plus the instruction and few-shot example texts sent with the request,
+because the model encodes that text again for every item. Label names,
+including the labels attached to examples, are not counted. With an
+instruction or examples, an item's count is capped at the model's
+`max_sequence_length` minus the label prompt, unless the document count alone
+is already higher. An item refused because its document pushes the labels out
+of the window returns a per-item `INPUT_TOO_LONG` error and counts nothing. A
+request that sends no instruction or examples is counted exactly as before.
+
+The instruction and each example text may be at most 2,048 characters, and
+together with the example labels at most 8,192 characters. Up to 32 examples
+are accepted, and they must leave room for the document in the model window.
+Label names are refused when their total length exceeds 16 characters per
+token of the window (8,192 characters for a 512-token model), more than any
+label prompt can fit.
+
+### GLiClass CUDA graphs
+
+A GLiClass forward on a DeBERTa encoder launches about a thousand small GPU
+kernels, so at small batch sizes the CPU that launches them is the bottleneck.
+A CUDA graph records the kernel launches of one input shape once; a replay
+launches them all in one call. Graphs are an operator setting, fixed when the
+model loads, in the model profile:
+
+```yaml
+profiles:
+  default:
+    adapter_options:
+      loadtime:
+        cuda_graphs: bucketed
+```
+
+| Value | Shapes recorded | Scores |
+|--|--|--|
+| `off` (default) | none | eager |
+| `exact` | each (batch, sequence length, label slots) seen twice | bit-identical to eager |
+| `bucketed` | sequence lengths padded up to a multiple of 32 tokens (64 on 1,024-token models) | padding moves fp16 probabilities (see below) |
+
+A request can send `options={"cuda_graphs": "off"}` to run eagerly on a model
+loaded with graphs. It cannot turn graphs on: any other value is refused.
+
+Padding is masked, but a longer sequence rounds fp16 sums differently, the same
+kind of change batching requests together makes. We compared `bucketed` with
+eager execution on the 384 CVE descriptions from `examples/typed-decisions`,
+three questions each, asked one at a time, as separate groups and as joint
+groups, plus 65 long documents. Each input was sent three times (long
+documents twice), so that graphs were recorded and then replayed: 11,538
+answers per model. A small change can still flip a near tie between the top
+two labels:
+
+| Model | Largest probability change | Top label changed | Shipped profile |
+|--|--|--|--|
+| `gliclass-small-v1.0` | 0.0039 | 3 answers | `off` |
+| `gliclass-base-v1.0` | 0.0049 | none | `bucketed` |
+| `gliclass-large-v1.0` | 0.0056 | none | `bucketed` |
+| `gliclass-base-v3.0` | 0.0054 | 3 | `off` |
+| `gliclass-large-v3.0` | 0.0093 | 3 | `off` |
+| `gliclass-instruct-base-v1.0` | 0.0076 | 12 | `off` |
+| `gliclass-instruct-large-v1.0` | 0.0098 | 18 | `off` |
+| `opir-multitask-large-v1.0` | 0.0144 | none | `bucketed` |
+| `gliclass-multilang-mini` (100 descriptions, no joint groups: 2,016 answers) | 0.0227 | 3 | `off` |
+
+`exact` changed nothing.
+
+The shipped profiles load with `bucketed` graphs only where no top label
+changed: `gliclass-base-v1.0`, `gliclass-large-v1.0` and
+`opir-multitask-large-v1.0`. Their probabilities can differ from eager
+execution by up to the amounts above. To run one of them eagerly, set
+`cuda_graphs: off` in its profile, or send `options={"cuda_graphs": "off"}` with
+a request. The other models load with `off`.
+
+Graphs apply on CUDA to the DeBERTa-based GLiClass models: the v1.0 models,
+`gliclass-base-v3.0` and `gliclass-large-v3.0`, the base and large instruct
+models, the Opir multitask models and `gliclass-multilang-mini`. The
+ModernBERT-based models (the edge models, `gliclass-multilang-edge` and the
+Opir edge models), CPU and MPS run eagerly with any value, and the load logs a
+warning.
+
+Nothing is recorded at load. A shape is recorded the first time a request
+needs it (the second time in `exact` mode), and that request takes a little
+over two eager forwards (76 ms against 33 ms on `gliclass-large-v1.0` on an
+L4). A model keeps at most 64 graphs, least recently used first
+out. A graph holds at most 2,048 tokens (batch times padded length); larger
+forwards are bound by the GPU rather than by kernel launches and run eagerly.
+
+**Memory, and other models on the same GPU.** Graph memory counts as device
+memory in use, but it is not attributed to the model: under memory pressure
+the server evicts whole models, least recently used first, which may be
+another model. A model's graphs share one memory pool, and the driver keeps a
+copy of each graph: about 8 MB for a `gliclass-large-v1.0` forward. On
+`gliclass-large-v1.0` in `bucketed` mode, the 31 graphs recorded for the CVE
+descriptions above took 555 MB of device memory: 490 MB for the pool and
+the graphs, 60 MB cached on the recording stream (its cuBLAS workspace and one
+warm-up row) and 5 MB of relative-position tables. The pool keeps what evicted
+graphs used, so traffic with many shapes keeps growing it. The runner therefore
+adds up the device memory its graphs hold (what each recording took, plus the
+tables and buffers they read), and past 4% of the device's memory (900 MB on
+an L4) it drops every graph, returns their memory to the device and records
+again. Graphs are also released when the model unloads, and
+when one of its forwards runs out of memory.
+
+While a graph records, PyTorch's caching allocator does not free cached blocks
+to satisfy other allocations, so another model on the same GPU that needs
+memory in that window (about one forward) can run out of memory where it
+otherwise would not. Recording is kept rare to limit this: one recording at a
+time in the process, none while less than a tenth of the device's memory is
+free, and per model at most 16 recordings at once, then one per 2 seconds. If a
+recording itself runs out of memory, the request still gets its eager answer;
+the model drops its graphs and records nothing for a minute.
+
+Both limits are approximate. The free-memory check reads the device once,
+before recording, so a model loading at the same moment can still meet one
+recording. The memory budget is checked after each recording, so a model's
+graphs can exceed it by one recording: up to about 330 MB, one graph of the
+largest shape on `gliclass-large-v1.0`. On a GPU shared with other models,
+leave memory headroom, or enable graphs only where the model has the GPU to
+itself. Usage and billing do not change.
+
+### GLiNER2.5-Decide usage and limits
+
+The GLiNER2.5-Decide models (`fastino/GLiNER2.5-Decide`, `GLiNER2.5-multi-Decide`,
+`GLiNER2.5-Decide-1B`) run on `gliner2` 2.x, which the transformers5 bundle
+pins (the `transformers5` image, or a native install as described above). Each
+item is one encoder row: every question's (or label group's) name,
+instruction, and labels, then the document. One forward pass answers them all,
+so the questions of a request are not independent: adding or changing one can
+change another's probabilities and score. `usage.input_tokens` counts the
+document tokens the model reads plus the tokens of the instructions and label
+descriptions (criteria) sent with the item, as Laya and GLiClass count
+instructions and criteria. Question ids, group names, and label names are not
+counted, and an item that returns an error counts nothing.
+
+A request takes at most 64 questions or label groups, 64 options per question,
+and 1,024 options in total. Question ids and group names may have 128
+characters, labels 256, and each instruction or description 2,048, with 65,536
+characters in all. Strings that contain one of the model's prompt markers
+(`[L]`, `[P]`, `[DESCRIPTION]`, ...) are refused. The questions may take at most
+512 tokens, or half the model's window when that is less: 256 of
+`GLiNER2.5-Decide`'s 512 tokens, 512 of the others' 2,048. This bounds the
+uncounted question and label tokens read with each item; a request needing more
+fails with `INPUT_TOO_LONG`. The
+document is read up to the whole words that fit in the rest of the window; a
+word longer than 4,096 characters, text past 64 characters per token of the
+window, or 4 words per token of the window also ends what is read. Words are
+split as gliner2 splits them, in linear time. A conversation (a list state) is
+read from its newest turn back; a run of more than 4,096 characters without a
+space is read only in its last 4,096 characters, and reading stops there. An item none of whose words fits, or that does
+not fit whole with `options={"overflow_policy": "error"}`, returns a per-item
+`INPUT_TOO_LONG` error while the other items succeed.
+
+## Configuration
+
+`sie-server` reads its config from `SIE_*` environment variables (Pydantic
+`BaseSettings`). Common knobs:
+
+### Memory & OOM resilience
+
+| Env var | Default | Effect |
+|--|--|--|
+| `SIE_MEMORY_PRESSURE_THRESHOLD_PERCENT` | `95` | VRAM utilisation that triggers reactive LRU eviction by the pressure monitor. |
+| `SIE_OOM_RECOVERY__ENABLED` | `true` | Master switch for reactive OOM recovery in the worker dispatch path (`cache_clear → evict_lru → split_batch`). |
+| `SIE_OOM_RECOVERY__STRATEGY` | `cache_clear,evict_lru,split_batch` | Ordered recovery actions. Earlier actions tried first. |
+| `SIE_OOM_RECOVERY__MAX_SPLIT_DEPTH` | `4` | Cap on recursive batch halving (≤16 sub-batches). |
+| `SIE_OOM_RECOVERY__EVICTION_LOCK_TIMEOUT_S` | `5.0` | Soft timeout when waiting for the registry's load-lock during recovery eviction. |
+| `SIE_OOM_RECOVERY__RETRY_AFTER_S` | `5` | `Retry-After` header value on `RESOURCE_EXHAUSTED` responses. |
+| `SIE_DISABLE_OOM_RECOVERY` | unset | Convenience kill switch (`1`/`true`/`yes`) for incident triage. Wins over `SIE_OOM_RECOVERY__ENABLED=true`. |
+| `SIE_IDLE_EVICT_S` | unset (disabled) | Unload models that have been idle longer than this (seconds). Additive to the pressure monitor; helps free cold weights before pressure builds. |
+| `SIE_OOM_NAK_DELAY_S` | `10.0` | Queue-mode only. NAK delay (seconds) for `RESOURCE_EXHAUSTED` work items so JetStream redelivers them after memory pressure has had a chance to clear. |
+
+When OOM recovery is exhausted on a request, the server returns
+`HTTP 503 RESOURCE_EXHAUSTED` with `Retry-After`. The Python SDK
+auto-retries; see `packages/sie_sdk/README.md` for client-side controls.
+
+### Batching & request handling
+
+| Env var | Default | Effect |
+|--|--|--|
+| `SIE_MAX_BATCH_REQUESTS` | `64` | Maximum number of items per batched inference call. |
+| `SIE_MAX_BATCH_WAIT_MS` | `15.0` | Initial value for the adaptive first-request batch timeout. At runtime the PI batching controller steers it between `SIE_ADAPTIVE_BATCHING__MIN_WAIT_MS` and `SIE_ADAPTIVE_BATCHING__MAX_WAIT_MS`, so it is a starting point rather than a fixed wait. |
+| `SIE_MAX_CONCURRENT_REQUESTS` | `512` | Per-worker queue size; admission control returns `QUEUE_FULL` above this. |
+| `SIE_MAX_LORAS_PER_MODEL` | `10` | Maximum concurrent LoRA adapters per base model. |
+| `SIE_MAX_ITEM_TEXT_BYTES` | `2097152` (2 MiB) | Most bytes of UTF-8 one encode, score, or extract item may carry in its `text` and `metadata` together. A larger item is rejected with `INVALID_INPUT` (HTTP 400) before it is tokenized. Generation prompts are not items and are not affected. |
+
+### Compute & precision
+
+| Env var | Default | Effect |
+|--|--|--|
+| `SIE_DEFAULT_COMPUTE_PRECISION` | `float16` | One of `float16`, `bfloat16`, `float32`. |
+| `SIE_ATTENTION_BACKEND` | `auto` | One of `auto`, `flash_attention_2`, `sdpa`, `eager`. |
+
+### Diagnostics
+
+| Env var | Default | Effect |
+|--|--|--|
+| `SIE_GRAMMAR_PREFLIGHT_DEBUG` | unset (off) | Enables the legacy worker-side Outlines preflight compile before each structured-output request. Off by default because SGLang is the production grammar authority. Use for diagnosing schema-rejection problems or slow compiles in a controlled environment; not recommended for production traffic. |
+
+For nested settings (any field with `__`), the env-var format is
+`SIE_<TOP>__<NESTED>=value`. The complete schema is in
+`packages/sie_server/src/sie_server/config/engine.py`.
+
+## Observability
+
+The server emits the checked-in worker telemetry contract once through
+OpenTelemetry/OTLP. The regional collector owns Prometheus exposition and
+optional remote OTLP routing; the application does not expose `/metrics` or
+maintain a second Prometheus registry. See `telemetry/contract.yaml` for exact
+instrument names, dimensions, ownership, and histogram bounds.
+
+## API
+
+See the [API documentation](https://sie.dev/docs) for details.
+
+## License
+
+Apache 2.0

{sie_server-0.8.2 → sie_server-0.8.3}/bundles/default.yaml

@@ -41,6 +41,8 @@ adapters:
 - sie_server.adapters.gliner_bi
 - sie_server.adapters.glirel
 - sie_server.adapters.gliclass
+- sie_server.adapters.laya.adapter
+- sie_server.adapters.gliformer.adapter
 - sie_server.adapters.donut
 - sie_server.adapters.florence2
 - sie_server.adapters.docling.adapter
@@ -72,14 +74,20 @@ deps:
   pillow: '==12.3.0'
   # grounding_dino
   requests: ''
-  # gliner, gliner_bi
-  gliner: '>=0.2,<1'
-  # gliner2
+  # gliner, gliner_bi (0.2.26: first release that loads the relex checkpoints)
+  gliner: '>=0.2.26,<1'
+  # gliner2. Held at 1.x: gliner2 2.0 changes gliner2-base/large-v1 outputs
+  # (every repeated NER mention is returned, and structured extraction is
+  # rescored). The GLiNER2.5-Decide models, which need 2.x, are served from
+  # the transformers5 bundle.
   gliner2: '>=1.3.1,<2'
   # glirel
   glirel: '>=1.0,<2'
-  # gliclass
-  gliclass: '>=0.1,<1'
+  # gliclass (0.1.17: cross-attention scorer for the multilingual checkpoints;
+  # 0.1.18+ needs transformers 5)
+  gliclass: '>=0.1.17,<1'
+  # gliformer: exact, the adapter uses package internals (see pyproject.toml)
+  gliformer: '==0.1.2'
   # gliner/glirel/gliclass shared dep
   loguru: '>=0.7,<1'
   # donut, florence2

{sie_server-0.8.2 → sie_server-0.8.3}/bundles/transformers5.yaml

@@ -7,6 +7,7 @@ adapters:
 - sie_server.adapters.lighton_ocr.adapter
 - sie_server.adapters.glm_ocr
 - sie_server.adapters.gliner2.classification
+- sie_server.adapters.gliner2.decide
 - sie_server.adapters.st_sparse_vision.adapter
 deps:
   # Transformers 5.x for models requiring newer architecture support
@@ -30,5 +31,10 @@ deps:
   # unsatisfiable against that pin and broke `uv run` resolution.
   # Let transformers drive the version.
   pillow: ''
-  # gliguard-LLMGuardrails-300M uses GLiNER2 classification APIs.
-  gliner2: '>=1.3.1,<2'
+  # gliguard-LLMGuardrails-300M uses GLiNER2 classification APIs, and the
+  # GLiNER2.5-Decide models need gliner2 2.x (AutoExtractor, boundary
+  # architecture). Exact: the Decide adapter builds the task prompt with the
+  # package's processor. GLiGuard's outputs are unchanged from 1.3.2. gliner2
+  # 2.0 declares transformers<5 only for its `local` extra, which is not used;
+  # the multilingual and 1B Decide checkpoints were saved with transformers 5.
+  gliner2: '==2.0.0'

sie_server-0.8.3/models/convaiinnovations__laya-multilingual.yaml

@@ -0,0 +1,24 @@
+sie_id: convaiinnovations/laya-multilingual
+hf_id: convaiinnovations/laya-multilingual
+hf_revision: 82d57fc4f2d1be3d2caac494045f2ec51d0842f3
+# This checkpoint ships no calibration temperatures (rl_agent_config.json has
+# temperature [1, 1, 1] and no per-option buckets), so its probabilities and
+# confidences are the raw softmax of the decision head: uncalibrated.
+inputs:
+  text: true
+  image: false
+  audio: false
+  video: false
+tasks:
+  encode: null
+  score: null
+  extract: {}
+max_sequence_length: 1024
+profiles:
+  default:
+    max_batch_tokens: 16384
+    compute_precision: bfloat16
+    adapter_path: sie_server.adapters.laya.adapter:LayaAdapter
+    adapter_options:
+      loadtime: {}
+      runtime: {}

sie_server-0.8.3/models/convaiinnovations__laya-typed-decisions.yaml

@@ -0,0 +1,21 @@
+sie_id: convaiinnovations/laya-typed-decisions
+hf_id: convaiinnovations/laya-typed-decisions
+hf_revision: bc76315b568af04bc19f133c8bca9a2b3a2d9905
+inputs:
+  text: true
+  image: false
+  audio: false
+  video: false
+tasks:
+  encode: null
+  score: null
+  extract: {}
+max_sequence_length: 1024
+profiles:
+  default:
+    max_batch_tokens: 16384
+    compute_precision: bfloat16
+    adapter_path: sie_server.adapters.laya.adapter:LayaAdapter
+    adapter_options:
+      loadtime: {}
+      runtime: {}

sie_server-0.8.3/models/convaiinnovations__laya.yaml

@@ -0,0 +1,21 @@
+sie_id: convaiinnovations/laya
+hf_id: convaiinnovations/laya
+hf_revision: aa8c91ca088ec597df95a0d1c76b3063cb2ae5e8
+inputs:
+  text: true
+  image: false
+  audio: false
+  video: false
+tasks:
+  encode: null
+  score: null
+  extract: {}
+max_sequence_length: 512
+profiles:
+  default:
+    max_batch_tokens: 16384
+    compute_precision: bfloat16
+    adapter_path: sie_server.adapters.laya.adapter:LayaAdapter
+    adapter_options:
+      loadtime: {}
+      runtime: {}

sie_server-0.8.3/models/fastino__GLiNER2.5-Decide-1B.yaml

@@ -0,0 +1,38 @@
+# GLiNER2.5-Decide-1B: typed-decision classifier on the Ettin 1B encoder
+# (jhu-clsp/ettin-enc-from-dec-1b, ModernBERT; span architecture), loaded through
+# gliner2 2.x AutoExtractor. Served from the transformers5 bundle, which pins gliner2 2.x.
+#
+# Checkpoint caveat: the weights at this revision are the step-285000 checkpoint
+# (upload commit eac5ac1e). The 59.6% fast-decisions score on the model card was
+# measured on an earlier step-195000 checkpoint (stated in that revision's card);
+# the uploaded weights had not been re-evaluated.
+#
+# The encoder config stores its RoPE bases only in the transformers-5
+# rope_parameters form (160000 for full and sliding-window layers). The adapter
+# checks the bases the loaded encoder actually uses, and under transformers 4
+# writes them where that version reads them (it would otherwise run the
+# sliding-window layers at 10000).
+sie_id: fastino/GLiNER2.5-Decide-1B
+hf_id: fastino/GLiNER2.5-Decide-1B
+hf_revision: 52c94d3b698bf6d2619df9d898bdc1523ea3f1ca
+inputs:
+  text: true
+  image: false
+  audio: false
+  video: false
+tasks:
+  encode: null
+  score: null
+  extract: {}
+# Tokens per item: every question's prompt and labels, then the document. The
+# Ettin encoder config sets max_position_embeddings 7999 and the tokenizer
+# model_max_length 8192; 2048 tokens stays within both.
+max_sequence_length: 2048
+profiles:
+  default:
+    max_batch_tokens: 16384
+    compute_precision: float16
+    adapter_path: sie_server.adapters.gliner2.decide:GLiNER2DecideAdapter
+    adapter_options:
+      loadtime: {}
+      runtime: {}

sie_server-0.8.3/models/fastino__GLiNER2.5-Decide.yaml

@@ -0,0 +1,28 @@
+# GLiNER2.5-Decide: English typed-decision classifier (DeBERTa-v3-large encoder,
+# span architecture), loaded through gliner2 2.x AutoExtractor. Served from the
+# transformers5 bundle, which pins gliner2 2.x.
+sie_id: fastino/GLiNER2.5-Decide
+hf_id: fastino/GLiNER2.5-Decide
+hf_revision: 7ee5da4c2415e32259bcdc0b1a7367c32ce8d6f6
+inputs:
+  text: true
+  image: false
+  audio: false
+  video: false
+tasks:
+  encode: null
+  score: null
+  extract: {}
+# Tokens per item: every question's prompt and labels, then the document. 512,
+# as for fastino/gliner2-large-v1, this model's base: its DeBERTa-v3-large encoder
+# was pretrained on 512 positions (max_position_embeddings), and neither the model
+# card nor the checkpoint config (max_len null) gives a longer length.
+max_sequence_length: 512
+profiles:
+  default:
+    max_batch_tokens: 16384
+    compute_precision: float16
+    adapter_path: sie_server.adapters.gliner2.decide:GLiNER2DecideAdapter
+    adapter_options:
+      loadtime: {}
+      runtime: {}

sie_server-0.8.3/models/fastino__GLiNER2.5-multi-Decide.yaml

@@ -0,0 +1,27 @@
+# GLiNER2.5-multi-Decide: multilingual typed-decision classifier (mDeBERTa-v3-base
+# encoder, boundary architecture; trained with max_len 4096 words), loaded through
+# gliner2 2.x AutoExtractor. Served from the transformers5 bundle, which pins gliner2 2.x.
+sie_id: fastino/GLiNER2.5-multi-Decide
+hf_id: fastino/GLiNER2.5-multi-Decide
+hf_revision: 6bc1d43d201b0691e733626389af8c57eea3ea68
+inputs:
+  text: true
+  image: false
+  audio: false
+  video: false
+tasks:
+  encode: null
+  score: null
+  extract: {}
+# Tokens per item: every question's prompt and labels, then the document. The
+# checkpoint config sets max_len 4096 (gliner2 counts it in words), and the
+# tokenizer sets no length limit; 2048 tokens stays within both.
+max_sequence_length: 2048
+profiles:
+  default:
+    max_batch_tokens: 16384
+    compute_precision: float16
+    adapter_path: sie_server.adapters.gliner2.decide:GLiNER2DecideAdapter
+    adapter_options:
+      loadtime: {}
+      runtime: {}

{sie_server-0.8.2 → sie_server-0.8.3}/models/knowledgator__gliclass-base-v1.0.yaml

@@ -19,5 +19,8 @@ profiles:
     adapter_options:
       loadtime:
         classification_type: single-label
+        # Replay forwards as CUDA graphs over bucketed lengths; "off" runs them eagerly.
+        # See "GLiClass CUDA graphs" in packages/sie_server/README.md.
+        cuda_graphs: bucketed
       runtime:
         threshold: 0.0

sie_server-0.8.3/models/knowledgator__gliclass-base-v3.0.yaml

@@ -0,0 +1,29 @@
+sie_id: knowledgator/gliclass-base-v3.0
+hf_id: knowledgator/gliclass-base-v3.0
+hf_revision: 77a70e6cd52e602ed18184ef37d18bdd3741e3d5
+inputs:
+  text: true
+  image: false
+  audio: false
+  video: false
+tasks:
+  encode: null
+  score: null
+  extract: {}
+max_sequence_length: 512
+profiles:
+  default:
+    max_batch_tokens: 16384
+    compute_precision: null
+    adapter_path: sie_server.adapters.gliclass:GLiClassAdapter
+    adapter_options:
+      loadtime:
+        classification_type: single-label
+      runtime:
+        threshold: 0.0
+  multi-label:
+    extends: default
+    adapter_options:
+      runtime:
+        classification_type: multi-label
+        threshold: 0.0

sie_server-0.8.3/models/knowledgator__gliclass-edge-v3.0.yaml

@@ -0,0 +1,29 @@
+sie_id: knowledgator/gliclass-edge-v3.0
+hf_id: knowledgator/gliclass-edge-v3.0
+hf_revision: df03993a2ed98e5e4a0d2dd7efbbd105abe874cf
+inputs:
+  text: true
+  image: false
+  audio: false
+  video: false
+tasks:
+  encode: null
+  score: null
+  extract: {}
+max_sequence_length: 512
+profiles:
+  default:
+    max_batch_tokens: 16384
+    compute_precision: null
+    adapter_path: sie_server.adapters.gliclass:GLiClassAdapter
+    adapter_options:
+      loadtime:
+        classification_type: single-label
+      runtime:
+        threshold: 0.0
+  multi-label:
+    extends: default
+    adapter_options:
+      runtime:
+        classification_type: multi-label
+        threshold: 0.0

sie_server-0.8.3/models/knowledgator__gliclass-instruct-base-v1.0.yaml

@@ -0,0 +1,29 @@
+sie_id: knowledgator/gliclass-instruct-base-v1.0
+hf_id: knowledgator/gliclass-instruct-base-v1.0
+hf_revision: 4f6a108b08a5537f395521d19b5073e197923dd3
+inputs:
+  text: true
+  image: false
+  audio: false
+  video: false
+tasks:
+  encode: null
+  score: null
+  extract: {}
+max_sequence_length: 512
+profiles:
+  default:
+    max_batch_tokens: 16384
+    compute_precision: null
+    adapter_path: sie_server.adapters.gliclass:GLiClassAdapter
+    adapter_options:
+      loadtime:
+        classification_type: single-label
+      runtime:
+        threshold: 0.0
+  multi-label:
+    extends: default
+    adapter_options:
+      runtime:
+        classification_type: multi-label
+        threshold: 0.0

sie_server-0.8.3/models/knowledgator__gliclass-instruct-edge-v1.0.yaml

@@ -0,0 +1,29 @@
+sie_id: knowledgator/gliclass-instruct-edge-v1.0
+hf_id: knowledgator/gliclass-instruct-edge-v1.0
+hf_revision: 727be8a417f6a7718e591b025e07054c146d8139
+inputs:
+  text: true
+  image: false
+  audio: false
+  video: false
+tasks:
+  encode: null
+  score: null
+  extract: {}
+max_sequence_length: 512
+profiles:
+  default:
+    max_batch_tokens: 16384
+    compute_precision: null
+    adapter_path: sie_server.adapters.gliclass:GLiClassAdapter
+    adapter_options:
+      loadtime:
+        classification_type: single-label
+      runtime:
+        threshold: 0.0
+  multi-label:
+    extends: default
+    adapter_options:
+      runtime:
+        classification_type: multi-label
+        threshold: 0.0

sie_server-0.8.3/models/knowledgator__gliclass-instruct-large-v1.0.yaml

@@ -0,0 +1,29 @@
+sie_id: knowledgator/gliclass-instruct-large-v1.0
+hf_id: knowledgator/gliclass-instruct-large-v1.0
+hf_revision: 825e5478c1bf4bffbf297690517097ccbdb2e006
+inputs:
+  text: true
+  image: false
+  audio: false
+  video: false
+tasks:
+  encode: null
+  score: null
+  extract: {}
+max_sequence_length: 512
+profiles:
+  default:
+    max_batch_tokens: 16384
+    compute_precision: null
+    adapter_path: sie_server.adapters.gliclass:GLiClassAdapter
+    adapter_options:
+      loadtime:
+        classification_type: single-label
+      runtime:
+        threshold: 0.0
+  multi-label:
+    extends: default
+    adapter_options:
+      runtime:
+        classification_type: multi-label
+        threshold: 0.0