gliner-haystack 0.1.0__tar.gz → 0.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,226 @@
1
+ Metadata-Version: 2.3
2
+ Name: gliner-haystack
3
+ Version: 0.2.0
4
+ Summary: Haystack entity extractors backed by GLiNER 2.5
5
+ Author: Max Swain
6
+ Requires-Dist: haystack-ai>=3.0,<4
7
+ Requires-Dist: httpx2>=0.1
8
+ Requires-Python: >=3.11
9
+ Description-Content-Type: text/markdown
10
+
11
+ # gliner-haystack
12
+
13
+ [Haystack](https://haystack.deepset.ai/) components for extracting named entities with
14
+ [GLiNER2 and GLiNER2.5](https://github.com/fastino-ai/GLiNER2). Extract entities from text
15
+ or enrich Haystack `Document` metadata, with synchronous and asynchronous pipeline support.
16
+
17
+ The Python package is a **client** for a compatible GLiNER server exposing vLLM Factory's
18
+ `POST /pooling` endpoint; it does not download or run a model in the Python process.
19
+ You can connect to an existing server or use the [Docker images](#serving-gliner-with-docker)
20
+ provided by this repository for a production-ready, higher-throughput inference backend.
21
+
22
+ ## Install
23
+
24
+ ```sh
25
+ pip install gliner-haystack
26
+ # Or: uv add gliner-haystack
27
+ ```
28
+
29
+ Requires Python 3.11+ and Haystack 3.x. Start a compatible GLiNER2 or GLiNER2.5 server before
30
+ running the examples below. The default endpoint is `http://localhost:8000` and the
31
+ model name used in requests is `/models/current` (matching the supplied Compose files).
32
+
33
+ ## Import
34
+
35
+ ```python
36
+ from haystack_integrations.components.extractors.gliner import GLiNERDocumentExtractor, GLiNERTextExtractor
37
+ ```
38
+
39
+ ## Usage
40
+
41
+ ### Extract entities from text
42
+
43
+ ```python
44
+ from haystack_integrations.components.extractors.gliner import GLiNERTextExtractor
45
+
46
+ extractor = GLiNERTextExtractor(api_base_url="http://localhost:8000")
47
+ result = extractor.run(
48
+ text="Alice works at Acme in Paris.",
49
+ labels=["person", "organization", "location"],
50
+ )
51
+ print(result["entities"])
52
+ extractor.close()
53
+ ```
54
+
55
+ `entities` is a mapping from label to a list of records. For example (confidence values
56
+ are illustrative):
57
+
58
+ ```python
59
+ {
60
+ "person": [{"text": "Alice", "start": 0, "end": 5, "confidence": 0.98}],
61
+ "organization": [{"text": "Acme", "start": 15, "end": 19, "confidence": 0.94}],
62
+ "location": [{"text": "Paris", "start": 23, "end": 28, "confidence": 0.96}],
63
+ }
64
+ ```
65
+
66
+ Labels are **run inputs**, so you can reuse the same component with different labels.
67
+ The extractor requests character spans and confidence from the server; returned records
68
+ reflect the server's actual detections.
69
+
70
+ ### Enrich Haystack documents
71
+
72
+ ```python
73
+ from haystack import Document
74
+ from haystack_integrations.components.extractors.gliner import GLiNERDocumentExtractor
75
+
76
+ extractor = GLiNERDocumentExtractor(api_base_url="http://localhost:8000")
77
+ result = extractor.run(
78
+ documents=[Document(content="Alice works at Acme in Paris.", meta={"source": "example"})],
79
+ labels=["person", "organization", "location"],
80
+ )
81
+ enriched = result["documents"][0]
82
+ print(enriched.meta["entities"])
83
+ print(enriched.meta["source"]) # "example"
84
+ extractor.close()
85
+ ```
86
+
87
+ The component returns new documents, preserves their existing metadata, and adds
88
+ `meta["entities"]`; it does not change the input documents or their text.
89
+
90
+ ### Use in a Haystack pipeline
91
+
92
+ ```python
93
+ from haystack import Pipeline
94
+ from haystack_integrations.components.extractors.gliner import GLiNERTextExtractor
95
+
96
+ extractor = GLiNERTextExtractor()
97
+ pipeline = Pipeline()
98
+ pipeline.add_component("entities", extractor)
99
+ result = pipeline.run({
100
+ "entities": {
101
+ "text": "Alice works at Acme in Paris.",
102
+ "labels": ["person", "organization", "location"],
103
+ }
104
+ })
105
+ print(result["entities"]["entities"])
106
+ extractor.close()
107
+ ```
108
+
109
+ In Haystack 3, `Pipeline.run_async(...)` calls the components' `run_async(...)` methods.
110
+ To process multiple documents concurrently, use `GLiNERDocumentExtractor.run_async`:
111
+
112
+ ```python
113
+ import asyncio
114
+ from haystack import Document
115
+ from haystack_integrations.components.extractors.gliner import GLiNERDocumentExtractor
116
+
117
+ async def main():
118
+ extractor = GLiNERDocumentExtractor(max_concurrency=5)
119
+ result = await extractor.run_async(
120
+ documents=[Document(content="Alice works at Acme."), Document(content="Bob lives in Paris.")],
121
+ labels=["person", "organization", "location"],
122
+ )
123
+ await extractor.close_async()
124
+ return result["documents"]
125
+
126
+ documents = asyncio.run(main())
127
+ ```
128
+
129
+ The async document extractor uses a shared HTTP client and a semaphore to limit
130
+ in-flight requests to `max_concurrency` (default: 5), preserving document order.
131
+ Synchronous `run` sends requests sequentially. Clients are initialized lazily; call
132
+ `warm_up()` or `warm_up_async()` to prepare them ahead of time, and `close()` or
133
+ `close_async()` when finished.
134
+
135
+ ### Configuration
136
+
137
+ Both components accept `api_base_url` (default `http://localhost:8000`), `model`
138
+ (default `/models/current`), `threshold` (default `0.5`), and `http_client_kwargs`
139
+ for [httpx2](https://github.com/pydantic/httpx2) client options. For example:
140
+
141
+ ```python
142
+ extractor = GLiNERTextExtractor(
143
+ api_base_url="https://inference.example.com",
144
+ model="/models/current",
145
+ threshold=0.3,
146
+ http_client_kwargs={"timeout": 120.0},
147
+ )
148
+ ```
149
+
150
+ Without `http_client_kwargs`, the client timeout is 30 seconds. The document component
151
+ also accepts `max_concurrency` for async extraction. Both components require nonempty
152
+ `labels` on every `run` / `run_async` call.
153
+
154
+ ### Compatible model families
155
+
156
+ The components use vLLM Factory's **schema-based entity extraction** contract. They
157
+ work with GLiNER2.5 boundary models (for example, `fastino/gliner2.5-multi-v1`
158
+ with `deberta_gliner25_io`) and GLiNER2 schema models (for example,
159
+ `fastino/gliner2-large-v1` with `deberta_gliner2_io`). The `model` constructor
160
+ argument must match the model name exposed by your server; the supplied Compose files
161
+ serve either model as `/models/current`. Both plugins accept the components' entity
162
+ schema, threshold, and span/confidence options. The GLiNER2 plugin ignores the
163
+ GLiNER2.5-specific `runtime="classic"` request field.
164
+
165
+ vLLM Factory also serves older span-based GLiNER models, but those plugins take a
166
+ **different** `labels` request and return flat entity lists; these two components do
167
+ not support that response format. They are entity extractors, not wrappers for other
168
+ GLiNER2 tasks such as relation extraction or classification.
169
+
170
+ ## Serving GLiNER with Docker
171
+
172
+ For production inference, this repository publishes vLLM Factory-based images
173
+ configured for GLiNER2.5 by default, for NVIDIA GPUs and x86-64 CPUs. Both serve `POST /pooling` on port 8000.
174
+ The GPU image is intended for higher-throughput deployments; the CPU image also
175
+ allows development and integration testing on machines without a supported GPU.
176
+
177
+ | Image | Compose file | Requirements |
178
+ | --- | --- | --- |
179
+ | `ghcr.io/maxdswain/vllm-gliner:latest` | `compose.yaml` | Supported NVIDIA GPU and NVIDIA Container Toolkit |
180
+ | `ghcr.io/maxdswain/vllm-gliner:latest-cpu` | `compose.cpu.yaml` | x86-64 CPU |
181
+
182
+ ```sh
183
+ # From a clone of this repository, choose one:
184
+ docker compose up -d # GPU
185
+ # docker compose -f compose.cpu.yaml up -d # CPU instead
186
+ ```
187
+
188
+ The first startup downloads and prepares `fastino/gliner2.5-multi-v1` in a persistent
189
+ Hugging Face cache; wait until the server is ready before sending requests. With either
190
+ Compose file, the Python examples above work without changes. To serve GLiNER2 instead,
191
+ change the Compose file's environment entries to `MODEL_ID: fastino/gliner2-large-v1`,
192
+ `MODEL_PLUGIN: deberta_gliner2`, and `IO_PLUGIN: deberta_gliner2_io` (keep the
193
+ `/models/current` path). To build an image yourself,
194
+ initialize the submodule and build with `docker build -t vllm-gliner .`; the Dockerfile
195
+ supports a `VLLM_BASE_IMAGE` build argument for selecting the CPU base image.
196
+
197
+ You can also call the server directly:
198
+
199
+ ```sh
200
+ curl http://localhost:8000/pooling \
201
+ -H 'Content-Type: application/json' \
202
+ -d '{"model":"/models/current","data":{"text":"Alice works at Acme in Paris.","schema":{"entities":{"person":"","organization":"","location":""}},"runtime":"classic","include_spans":true,"include_confidence":true}}'
203
+ ```
204
+
205
+ ## Development
206
+
207
+ ```sh
208
+ uv sync --group dev --group test
209
+ uv run --group dev --group test poe check
210
+ ```
211
+
212
+ `poe check` runs offline pytest tests, Ruff lint/format checks (120-character lines),
213
+ and Pyrefly type checks. For integration tests against a running GLiNER server:
214
+
215
+ ```sh
216
+ GLINER_INTEGRATION_URL=http://localhost:8000 uv run --group dev --group test poe test-integration
217
+ ```
218
+
219
+ Without `GLINER_INTEGRATION_URL`, integration tests skip. CPU-only serving works for
220
+ these tests; a GPU is not required.
221
+
222
+ Python package checks and publishing are defined in `.github/workflows/publish-package.yml`.
223
+ After checks pass on `main`, releasable Conventional Commits drive git-cliff release
224
+ notes, a version bump, a GitHub Release, and PyPI Trusted Publishing via the `pypi`
225
+ GitHub environment. Docker image publishing is a separate workflow that rebuilds only
226
+ when image inputs change.
@@ -0,0 +1,216 @@
1
+ # gliner-haystack
2
+
3
+ [Haystack](https://haystack.deepset.ai/) components for extracting named entities with
4
+ [GLiNER2 and GLiNER2.5](https://github.com/fastino-ai/GLiNER2). Extract entities from text
5
+ or enrich Haystack `Document` metadata, with synchronous and asynchronous pipeline support.
6
+
7
+ The Python package is a **client** for a compatible GLiNER server exposing vLLM Factory's
8
+ `POST /pooling` endpoint; it does not download or run a model in the Python process.
9
+ You can connect to an existing server or use the [Docker images](#serving-gliner-with-docker)
10
+ provided by this repository for a production-ready, higher-throughput inference backend.
11
+
12
+ ## Install
13
+
14
+ ```sh
15
+ pip install gliner-haystack
16
+ # Or: uv add gliner-haystack
17
+ ```
18
+
19
+ Requires Python 3.11+ and Haystack 3.x. Start a compatible GLiNER2 or GLiNER2.5 server before
20
+ running the examples below. The default endpoint is `http://localhost:8000` and the
21
+ model name used in requests is `/models/current` (matching the supplied Compose files).
22
+
23
+ ## Import
24
+
25
+ ```python
26
+ from haystack_integrations.components.extractors.gliner import GLiNERDocumentExtractor, GLiNERTextExtractor
27
+ ```
28
+
29
+ ## Usage
30
+
31
+ ### Extract entities from text
32
+
33
+ ```python
34
+ from haystack_integrations.components.extractors.gliner import GLiNERTextExtractor
35
+
36
+ extractor = GLiNERTextExtractor(api_base_url="http://localhost:8000")
37
+ result = extractor.run(
38
+ text="Alice works at Acme in Paris.",
39
+ labels=["person", "organization", "location"],
40
+ )
41
+ print(result["entities"])
42
+ extractor.close()
43
+ ```
44
+
45
+ `entities` is a mapping from label to a list of records. For example (confidence values
46
+ are illustrative):
47
+
48
+ ```python
49
+ {
50
+ "person": [{"text": "Alice", "start": 0, "end": 5, "confidence": 0.98}],
51
+ "organization": [{"text": "Acme", "start": 15, "end": 19, "confidence": 0.94}],
52
+ "location": [{"text": "Paris", "start": 23, "end": 28, "confidence": 0.96}],
53
+ }
54
+ ```
55
+
56
+ Labels are **run inputs**, so you can reuse the same component with different labels.
57
+ The extractor requests character spans and confidence from the server; returned records
58
+ reflect the server's actual detections.
59
+
60
+ ### Enrich Haystack documents
61
+
62
+ ```python
63
+ from haystack import Document
64
+ from haystack_integrations.components.extractors.gliner import GLiNERDocumentExtractor
65
+
66
+ extractor = GLiNERDocumentExtractor(api_base_url="http://localhost:8000")
67
+ result = extractor.run(
68
+ documents=[Document(content="Alice works at Acme in Paris.", meta={"source": "example"})],
69
+ labels=["person", "organization", "location"],
70
+ )
71
+ enriched = result["documents"][0]
72
+ print(enriched.meta["entities"])
73
+ print(enriched.meta["source"]) # "example"
74
+ extractor.close()
75
+ ```
76
+
77
+ The component returns new documents, preserves their existing metadata, and adds
78
+ `meta["entities"]`; it does not change the input documents or their text.
79
+
80
+ ### Use in a Haystack pipeline
81
+
82
+ ```python
83
+ from haystack import Pipeline
84
+ from haystack_integrations.components.extractors.gliner import GLiNERTextExtractor
85
+
86
+ extractor = GLiNERTextExtractor()
87
+ pipeline = Pipeline()
88
+ pipeline.add_component("entities", extractor)
89
+ result = pipeline.run({
90
+ "entities": {
91
+ "text": "Alice works at Acme in Paris.",
92
+ "labels": ["person", "organization", "location"],
93
+ }
94
+ })
95
+ print(result["entities"]["entities"])
96
+ extractor.close()
97
+ ```
98
+
99
+ In Haystack 3, `Pipeline.run_async(...)` calls the components' `run_async(...)` methods.
100
+ To process multiple documents concurrently, use `GLiNERDocumentExtractor.run_async`:
101
+
102
+ ```python
103
+ import asyncio
104
+ from haystack import Document
105
+ from haystack_integrations.components.extractors.gliner import GLiNERDocumentExtractor
106
+
107
+ async def main():
108
+ extractor = GLiNERDocumentExtractor(max_concurrency=5)
109
+ result = await extractor.run_async(
110
+ documents=[Document(content="Alice works at Acme."), Document(content="Bob lives in Paris.")],
111
+ labels=["person", "organization", "location"],
112
+ )
113
+ await extractor.close_async()
114
+ return result["documents"]
115
+
116
+ documents = asyncio.run(main())
117
+ ```
118
+
119
+ The async document extractor uses a shared HTTP client and a semaphore to limit
120
+ in-flight requests to `max_concurrency` (default: 5), preserving document order.
121
+ Synchronous `run` sends requests sequentially. Clients are initialized lazily; call
122
+ `warm_up()` or `warm_up_async()` to prepare them ahead of time, and `close()` or
123
+ `close_async()` when finished.
124
+
125
+ ### Configuration
126
+
127
+ Both components accept `api_base_url` (default `http://localhost:8000`), `model`
128
+ (default `/models/current`), `threshold` (default `0.5`), and `http_client_kwargs`
129
+ for [httpx2](https://github.com/pydantic/httpx2) client options. For example:
130
+
131
+ ```python
132
+ extractor = GLiNERTextExtractor(
133
+ api_base_url="https://inference.example.com",
134
+ model="/models/current",
135
+ threshold=0.3,
136
+ http_client_kwargs={"timeout": 120.0},
137
+ )
138
+ ```
139
+
140
+ Without `http_client_kwargs`, the client timeout is 30 seconds. The document component
141
+ also accepts `max_concurrency` for async extraction. Both components require nonempty
142
+ `labels` on every `run` / `run_async` call.
143
+
144
+ ### Compatible model families
145
+
146
+ The components use vLLM Factory's **schema-based entity extraction** contract. They
147
+ work with GLiNER2.5 boundary models (for example, `fastino/gliner2.5-multi-v1`
148
+ with `deberta_gliner25_io`) and GLiNER2 schema models (for example,
149
+ `fastino/gliner2-large-v1` with `deberta_gliner2_io`). The `model` constructor
150
+ argument must match the model name exposed by your server; the supplied Compose files
151
+ serve either model as `/models/current`. Both plugins accept the components' entity
152
+ schema, threshold, and span/confidence options. The GLiNER2 plugin ignores the
153
+ GLiNER2.5-specific `runtime="classic"` request field.
154
+
155
+ vLLM Factory also serves older span-based GLiNER models, but those plugins take a
156
+ **different** `labels` request and return flat entity lists; these two components do
157
+ not support that response format. They are entity extractors, not wrappers for other
158
+ GLiNER2 tasks such as relation extraction or classification.
159
+
160
+ ## Serving GLiNER with Docker
161
+
162
+ For production inference, this repository publishes vLLM Factory-based images
163
+ configured for GLiNER2.5 by default, for NVIDIA GPUs and x86-64 CPUs. Both serve `POST /pooling` on port 8000.
164
+ The GPU image is intended for higher-throughput deployments; the CPU image also
165
+ allows development and integration testing on machines without a supported GPU.
166
+
167
+ | Image | Compose file | Requirements |
168
+ | --- | --- | --- |
169
+ | `ghcr.io/maxdswain/vllm-gliner:latest` | `compose.yaml` | Supported NVIDIA GPU and NVIDIA Container Toolkit |
170
+ | `ghcr.io/maxdswain/vllm-gliner:latest-cpu` | `compose.cpu.yaml` | x86-64 CPU |
171
+
172
+ ```sh
173
+ # From a clone of this repository, choose one:
174
+ docker compose up -d # GPU
175
+ # docker compose -f compose.cpu.yaml up -d # CPU instead
176
+ ```
177
+
178
+ The first startup downloads and prepares `fastino/gliner2.5-multi-v1` in a persistent
179
+ Hugging Face cache; wait until the server is ready before sending requests. With either
180
+ Compose file, the Python examples above work without changes. To serve GLiNER2 instead,
181
+ change the Compose file's environment entries to `MODEL_ID: fastino/gliner2-large-v1`,
182
+ `MODEL_PLUGIN: deberta_gliner2`, and `IO_PLUGIN: deberta_gliner2_io` (keep the
183
+ `/models/current` path). To build an image yourself,
184
+ initialize the submodule and build with `docker build -t vllm-gliner .`; the Dockerfile
185
+ supports a `VLLM_BASE_IMAGE` build argument for selecting the CPU base image.
186
+
187
+ You can also call the server directly:
188
+
189
+ ```sh
190
+ curl http://localhost:8000/pooling \
191
+ -H 'Content-Type: application/json' \
192
+ -d '{"model":"/models/current","data":{"text":"Alice works at Acme in Paris.","schema":{"entities":{"person":"","organization":"","location":""}},"runtime":"classic","include_spans":true,"include_confidence":true}}'
193
+ ```
194
+
195
+ ## Development
196
+
197
+ ```sh
198
+ uv sync --group dev --group test
199
+ uv run --group dev --group test poe check
200
+ ```
201
+
202
+ `poe check` runs offline pytest tests, Ruff lint/format checks (120-character lines),
203
+ and Pyrefly type checks. For integration tests against a running GLiNER server:
204
+
205
+ ```sh
206
+ GLINER_INTEGRATION_URL=http://localhost:8000 uv run --group dev --group test poe test-integration
207
+ ```
208
+
209
+ Without `GLINER_INTEGRATION_URL`, integration tests skip. CPU-only serving works for
210
+ these tests; a GPU is not required.
211
+
212
+ Python package checks and publishing are defined in `.github/workflows/publish-package.yml`.
213
+ After checks pass on `main`, releasable Conventional Commits drive git-cliff release
214
+ notes, a version bump, a GitHub Release, and PyPI Trusted Publishing via the `pypi`
215
+ GitHub environment. Docker image publishing is a separate workflow that rebuilds only
216
+ when image inputs change.
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "gliner-haystack"
3
- version = "0.1.0"
3
+ version = "0.2.0"
4
4
  description = "Haystack entity extractors backed by GLiNER 2.5"
5
5
  readme = "README.md"
6
6
  requires-python = ">=3.11"
@@ -36,6 +36,18 @@ check = [
36
36
  [tool.ruff]
37
37
  line-length = 120
38
38
 
39
+ [tool.ruff.lint]
40
+ extend-select = [
41
+ "D",
42
+ "D417",
43
+ ]
44
+
45
+ [tool.ruff.lint.pydocstyle]
46
+ convention = "numpy"
47
+
48
+ [tool.ruff.lint.per-file-ignores]
49
+ "tests/**" = ["D"]
50
+
39
51
  [tool.pytest.ini_options]
40
52
  testpaths = ["tests"]
41
53
  markers = ["integration: requires a live vLLM GLiNER2.5 server (CPU or GPU)"]
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "gliner-haystack"
3
- version = "0.1.0"
3
+ version = "0.2.0"
4
4
  description = "Haystack entity extractors backed by GLiNER 2.5"
5
5
  readme = "README.md"
6
6
  authors = [
@@ -27,6 +27,15 @@ check = ["lint", "format", "types", "test"]
27
27
  [tool.ruff]
28
28
  line-length = 120
29
29
 
30
+ [tool.ruff.lint]
31
+ extend-select = ["D", "D417"]
32
+
33
+ [tool.ruff.lint.pydocstyle]
34
+ convention = "numpy"
35
+
36
+ [tool.ruff.lint.per-file-ignores]
37
+ "tests/**" = ["D"]
38
+
30
39
  [tool.pytest.ini_options]
31
40
  testpaths = ["tests"]
32
41
  markers = ["integration: requires a live vLLM GLiNER2.5 server (CPU or GPU)"]
@@ -0,0 +1,431 @@
1
+ """Provide Haystack entity extractors for vLLM Factory's GLiNER2 and GLiNER2.5 plugins."""
2
+
3
+ import asyncio
4
+ from dataclasses import replace
5
+ from typing import Any
6
+
7
+ import httpx2
8
+ from haystack import Document, component
9
+
10
+
11
+ def _validate_labels(labels: list[str]) -> None:
12
+ """Reject empty or invalid entity labels before sending a request.
13
+
14
+ Parameters
15
+ ----------
16
+ labels : list[str]
17
+ Entity types to extract from the input text.
18
+
19
+ Raises
20
+ ------
21
+ ValueError
22
+ If no nonempty string labels are supplied.
23
+ """
24
+ if not labels or any(not isinstance(label, str) or not label.strip() for label in labels):
25
+ raise ValueError("labels must be a non-empty list of non-empty strings")
26
+
27
+
28
+ @component
29
+ class GLiNERTextExtractor:
30
+ """Extract labeled entities from text using a remote GLiNER2 or GLiNER2.5 server.
31
+
32
+ Connect to vLLM Factory's ``POST /pooling`` endpoint. The server must be
33
+ running separately; this component does not load a model locally. Pass
34
+ entity labels to each call, so the same instance can extract different
35
+ types of entities from different texts. The ``entities`` output maps each
36
+ label to records containing text, character offsets, and confidence when
37
+ the server detects that label.
38
+
39
+ Examples
40
+ --------
41
+ Run against the GLiNER2.5 server at the default URL::
42
+
43
+ from haystack_integrations.components.extractors.gliner import GLiNERTextExtractor
44
+
45
+ extractor = GLiNERTextExtractor()
46
+ result = extractor.run(
47
+ text="Alice works at Acme.",
48
+ labels=["person", "organization"],
49
+ )
50
+ entities = result["entities"]
51
+ extractor.close()
52
+
53
+ Use ``await extractor.run_async(text=..., labels=...)`` and
54
+ ``await extractor.close_async()`` inside an async application or a
55
+ Haystack 3 ``Pipeline.run_async`` call.
56
+ """
57
+
58
+ def __init__(
59
+ self,
60
+ api_base_url: str = "http://localhost:8000",
61
+ model: str = "/models/current",
62
+ threshold: float = 0.5,
63
+ http_client_kwargs: dict[str, Any] | None = None,
64
+ ) -> None:
65
+ """Configure the remote GLiNER text extractor.
66
+
67
+ Parameters
68
+ ----------
69
+ api_base_url : str
70
+ Base URL of the vLLM Factory server, without the ``/pooling`` path.
71
+ Defaults to ``http://localhost:8000``.
72
+ model : str
73
+ Model identifier advertised by the server. The supplied Docker
74
+ image uses ``/models/current``.
75
+ threshold : float
76
+ Minimum entity confidence, between 0 and 1, passed to the server.
77
+ http_client_kwargs : dict[str, Any] | None
78
+ Options passed to both ``httpx2.Client`` and ``httpx2.AsyncClient``.
79
+ Defaults to a 30-second timeout; specify, for example,
80
+ ``{"timeout": 120.0}`` to override it.
81
+
82
+ Raises
83
+ ------
84
+ ValueError
85
+ If ``threshold`` is outside the interval from 0 to 1.
86
+ """
87
+ if not 0 <= threshold <= 1:
88
+ raise ValueError("threshold must be between 0 and 1")
89
+ self.api_base_url = api_base_url.rstrip("/")
90
+ self.model = model
91
+ self.threshold = threshold
92
+ self.http_client_kwargs = dict(http_client_kwargs) if http_client_kwargs is not None else {"timeout": 30.0}
93
+ self.client: httpx2.Client | None = None
94
+ self.async_client: httpx2.AsyncClient | None = None
95
+
96
+ def warm_up(self) -> None:
97
+ """Initialize a reusable synchronous HTTP client, if needed.
98
+
99
+ The first ``run`` call also warms up the component automatically.
100
+ """
101
+ if self.client is None:
102
+ self.client = httpx2.Client(**self.http_client_kwargs)
103
+
104
+ async def warm_up_async(self) -> None:
105
+ """Initialize a reusable asynchronous HTTP client, if needed.
106
+
107
+ The first ``run_async`` call also warms up the component automatically.
108
+ """
109
+ if self.async_client is None:
110
+ self.async_client = httpx2.AsyncClient(**self.http_client_kwargs)
111
+
112
+ def close(self) -> None:
113
+ """Close the synchronous HTTP client and release its connections.
114
+
115
+ Subsequent ``run`` calls create a new client as needed.
116
+ """
117
+ if self.client is not None:
118
+ self.client.close()
119
+ self.client = None
120
+
121
+ async def close_async(self) -> None:
122
+ """Close the asynchronous HTTP client and release its connections.
123
+
124
+ Subsequent ``run_async`` calls create a new client as needed.
125
+ """
126
+ if self.async_client is not None:
127
+ await self.async_client.aclose()
128
+ self.async_client = None
129
+
130
+ def _payload(self, text: str, labels: list[str]) -> dict[str, Any]:
131
+ """Build a schema-based pooling request for text and entity labels.
132
+
133
+ Parameters
134
+ ----------
135
+ text : str
136
+ Source text to analyze.
137
+ labels : list[str]
138
+ Entity types to extract.
139
+
140
+ Returns
141
+ -------
142
+ dict[str, Any]
143
+ JSON request body with the model, schema, and extraction options.
144
+ """
145
+ _validate_labels(labels)
146
+ return {
147
+ "model": self.model,
148
+ "data": {
149
+ "text": text,
150
+ "schema": {"entities": {label: "" for label in labels}},
151
+ "runtime": "classic",
152
+ "threshold": self.threshold,
153
+ "include_spans": True,
154
+ "include_confidence": True,
155
+ },
156
+ }
157
+
158
+ @staticmethod
159
+ def _entities(response: httpx2.Response) -> dict[str, list[dict[str, Any]]]:
160
+ """Validate a pooling response and return its labeled entity records.
161
+
162
+ Parameters
163
+ ----------
164
+ response : httpx2.Response
165
+ HTTP response from a compatible GLiNER ``/pooling`` endpoint.
166
+
167
+ Returns
168
+ -------
169
+ dict[str, list[dict[str, Any]]]
170
+ Entity records grouped by label.
171
+
172
+ Raises
173
+ ------
174
+ httpx2.HTTPStatusError
175
+ If the server returns an error status.
176
+ ValueError
177
+ If the response does not contain a labeled entity mapping.
178
+ """
179
+ response.raise_for_status()
180
+ payload = response.json()
181
+ if not isinstance(payload, dict) or not isinstance(payload.get("data"), dict):
182
+ raise ValueError("Invalid vLLM GLiNER pooling response: expected a data object") # noqa: TRY004
183
+ entities = payload["data"].get("entities")
184
+ if not isinstance(entities, dict) or any(
185
+ not isinstance(label, str)
186
+ or not isinstance(records, list)
187
+ or any(not isinstance(record, dict) for record in records)
188
+ for label, records in entities.items()
189
+ ):
190
+ raise ValueError("Invalid vLLM GLiNER pooling response: expected labeled entity lists")
191
+ return entities
192
+
193
+ @component.output_types(entities=dict[str, list[dict[str, Any]]])
194
+ def run(self, text: str, labels: list[str]) -> dict[str, dict[str, list[dict[str, Any]]]]:
195
+ """Extract entities from text synchronously.
196
+
197
+ Parameters
198
+ ----------
199
+ text : str
200
+ Text to analyze.
201
+ labels : list[str]
202
+ Nonempty names of entity types to find, such as ``["person"]``.
203
+
204
+ Returns
205
+ -------
206
+ dict[str, dict[str, list[dict[str, Any]]]]
207
+ ``{"entities": {label: [records]}}`` with the server's spans
208
+ and confidence values for each detected entity.
209
+
210
+ Raises
211
+ ------
212
+ ValueError
213
+ If the labels are invalid or the server's response is malformed.
214
+ httpx2.HTTPStatusError
215
+ If the server returns an error status.
216
+ """
217
+ payload = self._payload(text, labels)
218
+ self.warm_up()
219
+ assert self.client is not None
220
+ response = self.client.post(f"{self.api_base_url}/pooling", json=payload)
221
+ return {"entities": self._entities(response)}
222
+
223
+ @component.output_types(entities=dict[str, list[dict[str, Any]]])
224
+ async def run_async(self, text: str, labels: list[str]) -> dict[str, dict[str, list[dict[str, Any]]]]:
225
+ """Extract entities from text without blocking the event loop.
226
+
227
+ This is the asynchronous counterpart of ``run`` and returns the same
228
+ ``entities`` mapping. Use ``await`` to retrieve the result.
229
+
230
+ Parameters
231
+ ----------
232
+ text : str
233
+ Text to analyze.
234
+ labels : list[str]
235
+ Nonempty names of entity types to find, such as ``["person"]``.
236
+
237
+ Returns
238
+ -------
239
+ dict[str, dict[str, list[dict[str, Any]]]]
240
+ ``{"entities": {label: [records]}}`` with the server's spans
241
+ and confidence values for each detected entity.
242
+
243
+ Raises
244
+ ------
245
+ ValueError
246
+ If the labels are invalid or the server's response is malformed.
247
+ httpx2.HTTPStatusError
248
+ If the server returns an error status.
249
+ """
250
+ payload = self._payload(text, labels)
251
+ await self.warm_up_async()
252
+ assert self.async_client is not None
253
+ response = await self.async_client.post(f"{self.api_base_url}/pooling", json=payload)
254
+ return {"entities": self._entities(response)}
255
+
256
+
257
+ @component
258
+ class GLiNERDocumentExtractor:
259
+ """Enrich copies of Haystack documents with GLiNER2 or GLiNER2.5 entities.
260
+
261
+ The component sends each document's text to a vLLM Factory ``/pooling``
262
+ server and adds the labeled entity mapping to ``meta["entities"]`` of
263
+ each returned document. Existing metadata and document content are
264
+ preserved, and input documents are not modified. Use ``run_async`` to
265
+ process a batch concurrently; ``max_concurrency`` limits simultaneous
266
+ HTTP requests.
267
+
268
+ Examples
269
+ --------
270
+ Add person and organization entities to document metadata::
271
+
272
+ from haystack import Document
273
+ from haystack_integrations.components.extractors.gliner import GLiNERDocumentExtractor
274
+
275
+ extractor = GLiNERDocumentExtractor(max_concurrency=5)
276
+ result = extractor.run(
277
+ documents=[Document(content="Alice works at Acme.")],
278
+ labels=["person", "organization"],
279
+ )
280
+ documents = result["documents"]
281
+ entities = documents[0].meta["entities"]
282
+ extractor.close()
283
+
284
+ Use ``await extractor.run_async(documents=..., labels=...)`` inside an
285
+ async application or a Haystack 3 ``Pipeline.run_async`` call.
286
+ """
287
+
288
+ def __init__(
289
+ self,
290
+ api_base_url: str = "http://localhost:8000",
291
+ model: str = "/models/current",
292
+ threshold: float = 0.5,
293
+ http_client_kwargs: dict[str, Any] | None = None,
294
+ max_concurrency: int = 5,
295
+ ) -> None:
296
+ """Configure the remote GLiNER document extractor.
297
+
298
+ Parameters
299
+ ----------
300
+ api_base_url : str
301
+ Base URL of the vLLM Factory server, without the ``/pooling`` path.
302
+ Defaults to ``http://localhost:8000``.
303
+ model : str
304
+ Model identifier advertised by the server. The supplied Docker
305
+ image uses ``/models/current``.
306
+ threshold : float
307
+ Minimum entity confidence, between 0 and 1, passed to the server.
308
+ http_client_kwargs : dict[str, Any] | None
309
+ Options passed to both ``httpx2.Client`` and ``httpx2.AsyncClient``.
310
+ Defaults to a 30-second timeout.
311
+ max_concurrency : int
312
+ Maximum number of concurrent HTTP requests in ``run_async``;
313
+ ``run`` always processes documents sequentially. Defaults to 5.
314
+
315
+ Raises
316
+ ------
317
+ ValueError
318
+ If ``max_concurrency`` is not a positive integer or ``threshold``
319
+ is outside the interval from 0 to 1.
320
+ """
321
+ if isinstance(max_concurrency, bool) or not isinstance(max_concurrency, int) or max_concurrency < 1:
322
+ raise ValueError("max_concurrency must be a positive integer")
323
+ self.text_extractor = GLiNERTextExtractor(api_base_url, model, threshold, http_client_kwargs)
324
+ self.max_concurrency = max_concurrency
325
+
326
+ def warm_up(self) -> None:
327
+ """Initialize a reusable synchronous HTTP client for document requests.
328
+
329
+ The first ``run`` call also warms up the component automatically.
330
+ """
331
+ self.text_extractor.warm_up()
332
+
333
+ async def warm_up_async(self) -> None:
334
+ """Initialize a reusable asynchronous HTTP client for document requests.
335
+
336
+ The first nonempty ``run_async`` call also warms up the component.
337
+ """
338
+ await self.text_extractor.warm_up_async()
339
+
340
+ def close(self) -> None:
341
+ """Close the synchronous HTTP client after processing documents."""
342
+ self.text_extractor.close()
343
+
344
+ async def close_async(self) -> None:
345
+ """Close the asynchronous HTTP client after processing documents."""
346
+ await self.text_extractor.close_async()
347
+
348
+ @component.output_types(documents=list[Document])
349
+ def run(self, documents: list[Document], labels: list[str]) -> dict[str, list[Document]]:
350
+ """Add entities to copies of documents, processing one at a time.
351
+
352
+ Parameters
353
+ ----------
354
+ documents : list[Document]
355
+ Text documents to analyze in input order.
356
+ labels : list[str]
357
+ Nonempty names of entity types to extract from every document.
358
+
359
+ Returns
360
+ -------
361
+ dict[str, list[Document]]
362
+ ``{"documents": enriched_documents}`` with entities stored in
363
+ each document's ``meta["entities"]``.
364
+
365
+ Raises
366
+ ------
367
+ ValueError
368
+ If labels are invalid, a document has no text content, or a
369
+ response is malformed.
370
+ httpx2.HTTPStatusError
371
+ If the server returns an error status.
372
+ """
373
+ _validate_labels(labels)
374
+ result = []
375
+ for document in documents:
376
+ if document.content is None:
377
+ raise ValueError("GLiNERDocumentExtractor requires text document content")
378
+ entities = self.text_extractor.run(document.content, labels)["entities"]
379
+ result.append(replace(document, meta={**document.meta, "entities": entities}))
380
+ return {"documents": result}
381
+
382
+ @component.output_types(documents=list[Document])
383
+ async def run_async(self, documents: list[Document], labels: list[str]) -> dict[str, list[Document]]:
384
+ """Add entities to document copies with bounded concurrent requests.
385
+
386
+ The shared async HTTP client sends at most ``max_concurrency`` requests
387
+ simultaneously. Returned documents retain the input order.
388
+
389
+ Parameters
390
+ ----------
391
+ documents : list[Document]
392
+ Text documents to analyze in input order.
393
+ labels : list[str]
394
+ Nonempty names of entity types to extract from every document.
395
+
396
+ Returns
397
+ -------
398
+ dict[str, list[Document]]
399
+ ``{"documents": enriched_documents}`` with entities stored in
400
+ each document's ``meta["entities"]``.
401
+
402
+ Raises
403
+ ------
404
+ ValueError
405
+ If labels are invalid, a document has no text content, or a
406
+ response is malformed.
407
+ httpx2.HTTPStatusError
408
+ If the server returns an error status.
409
+ """
410
+ _validate_labels(labels)
411
+ for document in documents:
412
+ if document.content is None:
413
+ raise ValueError("GLiNERDocumentExtractor requires text document content")
414
+
415
+ semaphore = asyncio.Semaphore(self.max_concurrency)
416
+ extractor = self.text_extractor
417
+ if not documents:
418
+ return {"documents": []}
419
+ await self.warm_up_async()
420
+ assert extractor.async_client is not None
421
+ client = extractor.async_client
422
+
423
+ async def extract(document: Document) -> Document:
424
+ async with semaphore:
425
+ response = await client.post(
426
+ f"{extractor.api_base_url}/pooling", json=extractor._payload(document.content or "", labels)
427
+ )
428
+ entities = extractor._entities(response)
429
+ return replace(document, meta={**document.meta, "entities": entities})
430
+
431
+ return {"documents": list(await asyncio.gather(*(extract(document) for document in documents)))}
@@ -1,85 +0,0 @@
1
- Metadata-Version: 2.3
2
- Name: gliner-haystack
3
- Version: 0.1.0
4
- Summary: Haystack entity extractors backed by GLiNER 2.5
5
- Author: Max Swain
6
- Requires-Dist: haystack-ai>=3.0,<4
7
- Requires-Dist: httpx2>=0.1
8
- Requires-Python: >=3.11
9
- Description-Content-Type: text/markdown
10
-
11
- # gliner-haystack
12
-
13
- Serve GLiNER2.5 with vLLM Factory using Docker Compose. GPU and CPU images are available:
14
-
15
- | Image | Example Compose file | Requirements |
16
- | --- | --- | --- |
17
- | `ghcr.io/maxdswain/vllm-gliner:latest` (GPU) | [compose.yaml](compose.yaml) | NVIDIA GPU and NVIDIA Container Toolkit |
18
- | `ghcr.io/maxdswain/vllm-gliner:latest-cpu` (x86-64 CPU) | [compose.cpu.yaml](compose.cpu.yaml) | x86-64 CPU |
19
-
20
- ## Haystack components
21
-
22
- Install with `uv sync --group dev --group test`. The text component returns labeled entity spans;
23
- the document component stores them in copied documents' `meta["entities"]`:
24
-
25
- ```python
26
- from haystack import Document
27
- from haystack_integrations.components.extractors.gliner import GLiNERDocumentExtractor, GLiNERTextExtractor
28
-
29
- labels = ["person", "organization", "location"]
30
- text_extractor = GLiNERTextExtractor(api_base_url="http://localhost:8000", http_client_kwargs={"timeout": 30.0})
31
- document_extractor = GLiNERDocumentExtractor(max_concurrency=5)
32
- try:
33
- entities = text_extractor.run("Alice works at Acme in Paris.", labels)["entities"]
34
- documents = document_extractor.run([Document(content="Alice works at Acme in Paris.")], labels)["documents"]
35
- finally:
36
- text_extractor.close()
37
- document_extractor.close()
38
- ```
39
-
40
- Both components support `run_async` via Haystack 3's `Pipeline.run_async`. Document `run_async` sends requests
41
- concurrently, with a semaphore enforcing `max_concurrency`; synchronous `run` processes documents sequentially.
42
- Call `warm_up()` / `warm_up_async()` to prepare reusable HTTP clients if desired (runs also warm up lazily).
43
- Call `close()` / `close_async()` to release them when done. HTTP client options such as `timeout` can be passed
44
- via `http_client_kwargs` in either constructor.
45
-
46
- Offline tests, lint, formatting and type checks: `uv run --group dev --group test poe check`.
47
- To run tests against a live server:
48
- `GLINER_INTEGRATION_URL=http://localhost:8000 uv run --group dev --group test poe test-integration`.
49
- Integration tests skip when the variable is unset.
50
-
51
- ## Python package releases
52
-
53
- `.github/workflows/publish-package.yml` runs offline checks on pull requests and pushes to `main`
54
- (Python 3.11 and 3.13). After successful checks on `main`, git-cliff finds releasable
55
- Conventional Commits (`feat`, `fix`, `perf`, or breaking changes). If there are any, the
56
- workflow updates `pyproject.toml` and `uv.lock`, builds distributions, commits the version,
57
- creates a `vX.Y.Z` tag and GitHub Release with generated notes, then publishes those
58
- artifacts to PyPI using the `pypi` GitHub environment and PyPI Trusted Publishing. The
59
- initial release is `v0.1.0`; Docker-scoped commits do not release the Python package.
60
- The release job needs permission to push its version commit and tag to `main` (check branch
61
- protection rules). GitHub's `GITHUB_TOKEN` does not trigger another workflow when it creates
62
- a release, so the PyPI publish job runs in the same workflow after the release job succeeds.
63
-
64
- ## Quick start
65
-
66
- Install Docker with Compose, then:
67
-
68
- ```sh
69
- git clone https://github.com/maxdswain/gliner-haystack.git
70
- cd gliner-haystack
71
-
72
- # GPU (default)
73
- docker compose up
74
-
75
- # Or CPU
76
- docker compose -f compose.cpu.yaml up
77
- ```
78
-
79
- The first start downloads the model. Once the server is ready, try:
80
-
81
- ```sh
82
- curl http://localhost:8000/pooling \
83
- -H 'Content-Type: application/json' \
84
- -d '{"model":"/models/current","data":{"text":"Alice works at Acme in Paris.","schema":{"entities":{"person":"","organization":"","location":""}},"runtime":"classic","include_spans":true}}'
85
- ```
@@ -1,75 +0,0 @@
1
- # gliner-haystack
2
-
3
- Serve GLiNER2.5 with vLLM Factory using Docker Compose. GPU and CPU images are available:
4
-
5
- | Image | Example Compose file | Requirements |
6
- | --- | --- | --- |
7
- | `ghcr.io/maxdswain/vllm-gliner:latest` (GPU) | [compose.yaml](compose.yaml) | NVIDIA GPU and NVIDIA Container Toolkit |
8
- | `ghcr.io/maxdswain/vllm-gliner:latest-cpu` (x86-64 CPU) | [compose.cpu.yaml](compose.cpu.yaml) | x86-64 CPU |
9
-
10
- ## Haystack components
11
-
12
- Install with `uv sync --group dev --group test`. The text component returns labeled entity spans;
13
- the document component stores them in copied documents' `meta["entities"]`:
14
-
15
- ```python
16
- from haystack import Document
17
- from haystack_integrations.components.extractors.gliner import GLiNERDocumentExtractor, GLiNERTextExtractor
18
-
19
- labels = ["person", "organization", "location"]
20
- text_extractor = GLiNERTextExtractor(api_base_url="http://localhost:8000", http_client_kwargs={"timeout": 30.0})
21
- document_extractor = GLiNERDocumentExtractor(max_concurrency=5)
22
- try:
23
- entities = text_extractor.run("Alice works at Acme in Paris.", labels)["entities"]
24
- documents = document_extractor.run([Document(content="Alice works at Acme in Paris.")], labels)["documents"]
25
- finally:
26
- text_extractor.close()
27
- document_extractor.close()
28
- ```
29
-
30
- Both components support `run_async` via Haystack 3's `Pipeline.run_async`. Document `run_async` sends requests
31
- concurrently, with a semaphore enforcing `max_concurrency`; synchronous `run` processes documents sequentially.
32
- Call `warm_up()` / `warm_up_async()` to prepare reusable HTTP clients if desired (runs also warm up lazily).
33
- Call `close()` / `close_async()` to release them when done. HTTP client options such as `timeout` can be passed
34
- via `http_client_kwargs` in either constructor.
35
-
36
- Offline tests, lint, formatting and type checks: `uv run --group dev --group test poe check`.
37
- To run tests against a live server:
38
- `GLINER_INTEGRATION_URL=http://localhost:8000 uv run --group dev --group test poe test-integration`.
39
- Integration tests skip when the variable is unset.
40
-
41
- ## Python package releases
42
-
43
- `.github/workflows/publish-package.yml` runs offline checks on pull requests and pushes to `main`
44
- (Python 3.11 and 3.13). After successful checks on `main`, git-cliff finds releasable
45
- Conventional Commits (`feat`, `fix`, `perf`, or breaking changes). If there are any, the
46
- workflow updates `pyproject.toml` and `uv.lock`, builds distributions, commits the version,
47
- creates a `vX.Y.Z` tag and GitHub Release with generated notes, then publishes those
48
- artifacts to PyPI using the `pypi` GitHub environment and PyPI Trusted Publishing. The
49
- initial release is `v0.1.0`; Docker-scoped commits do not release the Python package.
50
- The release job needs permission to push its version commit and tag to `main` (check branch
51
- protection rules). GitHub's `GITHUB_TOKEN` does not trigger another workflow when it creates
52
- a release, so the PyPI publish job runs in the same workflow after the release job succeeds.
53
-
54
- ## Quick start
55
-
56
- Install Docker with Compose, then:
57
-
58
- ```sh
59
- git clone https://github.com/maxdswain/gliner-haystack.git
60
- cd gliner-haystack
61
-
62
- # GPU (default)
63
- docker compose up
64
-
65
- # Or CPU
66
- docker compose -f compose.cpu.yaml up
67
- ```
68
-
69
- The first start downloads the model. Once the server is ready, try:
70
-
71
- ```sh
72
- curl http://localhost:8000/pooling \
73
- -H 'Content-Type: application/json' \
74
- -d '{"model":"/models/current","data":{"text":"Alice works at Acme in Paris.","schema":{"entities":{"person":"","organization":"","location":""}},"runtime":"classic","include_spans":true}}'
75
- ```
@@ -1,180 +0,0 @@
1
- """Haystack entity extractors backed by vLLM Factory's GLiNER2.5 pooling API."""
2
-
3
- import asyncio
4
- from dataclasses import replace
5
- from typing import Any
6
-
7
- import httpx2
8
- from haystack import Document, component
9
-
10
-
11
- def _validate_labels(labels: list[str]) -> None:
12
- if not labels or any(not isinstance(label, str) or not label.strip() for label in labels):
13
- raise ValueError("labels must be a non-empty list of non-empty strings")
14
-
15
-
16
- @component
17
- class GLiNERTextExtractor:
18
- """Extract labeled entities from text using the vLLM GLiNER2.5 server.
19
-
20
- The ``entities`` mapping has one list per label. Each record includes
21
- ``text``, ``start``, ``end``, and ``confidence`` when provided by the server.
22
- """
23
-
24
- def __init__(
25
- self,
26
- api_base_url: str = "http://localhost:8000",
27
- model: str = "/models/current",
28
- threshold: float = 0.5,
29
- http_client_kwargs: dict[str, Any] | None = None,
30
- ) -> None:
31
- if not 0 <= threshold <= 1:
32
- raise ValueError("threshold must be between 0 and 1")
33
- self.api_base_url = api_base_url.rstrip("/")
34
- self.model = model
35
- self.threshold = threshold
36
- self.http_client_kwargs = dict(http_client_kwargs) if http_client_kwargs is not None else {"timeout": 30.0}
37
- self.client: httpx2.Client | None = None
38
- self.async_client: httpx2.AsyncClient | None = None
39
-
40
- def warm_up(self) -> None:
41
- """Create and retain the synchronous HTTP client."""
42
- if self.client is None:
43
- self.client = httpx2.Client(**self.http_client_kwargs)
44
-
45
- async def warm_up_async(self) -> None:
46
- """Create and retain the asynchronous HTTP client."""
47
- if self.async_client is None:
48
- self.async_client = httpx2.AsyncClient(**self.http_client_kwargs)
49
-
50
- def close(self) -> None:
51
- """Release the synchronous client's connections."""
52
- if self.client is not None:
53
- self.client.close()
54
- self.client = None
55
-
56
- async def close_async(self) -> None:
57
- """Release the asynchronous client's connections."""
58
- if self.async_client is not None:
59
- await self.async_client.aclose()
60
- self.async_client = None
61
-
62
- def _payload(self, text: str, labels: list[str]) -> dict[str, Any]:
63
- _validate_labels(labels)
64
- return {
65
- "model": self.model,
66
- "data": {
67
- "text": text,
68
- "schema": {"entities": {label: "" for label in labels}},
69
- "runtime": "classic",
70
- "threshold": self.threshold,
71
- "include_spans": True,
72
- "include_confidence": True,
73
- },
74
- }
75
-
76
- @staticmethod
77
- def _entities(response: httpx2.Response) -> dict[str, list[dict[str, Any]]]:
78
- response.raise_for_status()
79
- payload = response.json()
80
- if not isinstance(payload, dict) or not isinstance(payload.get("data"), dict):
81
- raise ValueError("Invalid vLLM GLiNER pooling response: expected a data object") # noqa: TRY004
82
- entities = payload["data"].get("entities")
83
- if not isinstance(entities, dict) or any(
84
- not isinstance(label, str)
85
- or not isinstance(records, list)
86
- or any(not isinstance(record, dict) for record in records)
87
- for label, records in entities.items()
88
- ):
89
- raise ValueError("Invalid vLLM GLiNER pooling response: expected labeled entity lists")
90
- return entities
91
-
92
- @component.output_types(entities=dict[str, list[dict[str, Any]]])
93
- def run(self, text: str, labels: list[str]) -> dict[str, dict[str, list[dict[str, Any]]]]:
94
- """Call ``POST /pooling`` synchronously."""
95
- payload = self._payload(text, labels)
96
- self.warm_up()
97
- assert self.client is not None
98
- response = self.client.post(f"{self.api_base_url}/pooling", json=payload)
99
- return {"entities": self._entities(response)}
100
-
101
- @component.output_types(entities=dict[str, list[dict[str, Any]]])
102
- async def run_async(self, text: str, labels: list[str]) -> dict[str, dict[str, list[dict[str, Any]]]]:
103
- """Call ``POST /pooling`` without blocking the event loop."""
104
- payload = self._payload(text, labels)
105
- await self.warm_up_async()
106
- assert self.async_client is not None
107
- response = await self.async_client.post(f"{self.api_base_url}/pooling", json=payload)
108
- return {"entities": self._entities(response)}
109
-
110
-
111
- @component
112
- class GLiNERDocumentExtractor:
113
- """Return copies of documents with labeled entities in ``meta['entities']``."""
114
-
115
- def __init__(
116
- self,
117
- api_base_url: str = "http://localhost:8000",
118
- model: str = "/models/current",
119
- threshold: float = 0.5,
120
- http_client_kwargs: dict[str, Any] | None = None,
121
- max_concurrency: int = 5,
122
- ) -> None:
123
- if isinstance(max_concurrency, bool) or not isinstance(max_concurrency, int) or max_concurrency < 1:
124
- raise ValueError("max_concurrency must be a positive integer")
125
- self.text_extractor = GLiNERTextExtractor(api_base_url, model, threshold, http_client_kwargs)
126
- self.max_concurrency = max_concurrency
127
-
128
- def warm_up(self) -> None:
129
- """Prepare the synchronous HTTP client."""
130
- self.text_extractor.warm_up()
131
-
132
- async def warm_up_async(self) -> None:
133
- """Prepare the asynchronous HTTP client."""
134
- await self.text_extractor.warm_up_async()
135
-
136
- def close(self) -> None:
137
- """Close the synchronous HTTP client."""
138
- self.text_extractor.close()
139
-
140
- async def close_async(self) -> None:
141
- """Close the asynchronous HTTP client."""
142
- await self.text_extractor.close_async()
143
-
144
- @component.output_types(documents=list[Document])
145
- def run(self, documents: list[Document], labels: list[str]) -> dict[str, list[Document]]:
146
- """Extract entities synchronously per document, without mutating inputs."""
147
- _validate_labels(labels)
148
- result = []
149
- for document in documents:
150
- if document.content is None:
151
- raise ValueError("GLiNERDocumentExtractor requires text document content")
152
- entities = self.text_extractor.run(document.content, labels)["entities"]
153
- result.append(replace(document, meta={**document.meta, "entities": entities}))
154
- return {"documents": result}
155
-
156
- @component.output_types(documents=list[Document])
157
- async def run_async(self, documents: list[Document], labels: list[str]) -> dict[str, list[Document]]:
158
- """Extract concurrently, bounding in-flight requests with a semaphore."""
159
- _validate_labels(labels)
160
- for document in documents:
161
- if document.content is None:
162
- raise ValueError("GLiNERDocumentExtractor requires text document content")
163
-
164
- semaphore = asyncio.Semaphore(self.max_concurrency)
165
- extractor = self.text_extractor
166
- if not documents:
167
- return {"documents": []}
168
- await self.warm_up_async()
169
- assert extractor.async_client is not None
170
- client = extractor.async_client
171
-
172
- async def extract(document: Document) -> Document:
173
- async with semaphore:
174
- response = await client.post(
175
- f"{extractor.api_base_url}/pooling", json=extractor._payload(document.content or "", labels)
176
- )
177
- entities = extractor._entities(response)
178
- return replace(document, meta={**document.meta, "entities": entities})
179
-
180
- return {"documents": list(await asyncio.gather(*(extract(document) for document in documents)))}