deepl-haystack 0.1.1__tar.gz → 0.2.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (8) hide show
  1. {deepl_haystack-0.1.1 → deepl_haystack-0.2.1}/PKG-INFO +111 -11
  2. deepl_haystack-0.2.1/README.md +154 -0
  3. deepl_haystack-0.2.1/deepl_haystack/__init__.py +12 -0
  4. deepl_haystack-0.1.1/deepl_haystack/components/translators/deepl/translator.py → deepl_haystack-0.2.1/deepl_haystack/components.py +15 -13
  5. {deepl_haystack-0.1.1 → deepl_haystack-0.2.1}/pyproject.toml +8 -3
  6. deepl_haystack-0.1.1/README.md +0 -55
  7. deepl_haystack-0.1.1/deepl_haystack/__init__.py +0 -10
  8. {deepl_haystack-0.1.1 → deepl_haystack-0.2.1}/LICENSE +0 -0
{deepl_haystack-0.1.1 → deepl_haystack-0.2.1}/PKG-INFO RENAMED
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: deepl-haystack
3
- Version: 0.1.1
3
+ Version: 0.2.1
4
4
  Summary: Haystack integration with DeepL translation services provider.
5
5
  Home-page: https://github.com/dribia/deepl-haystack
6
6
  License: Apache-2.0
@@ -22,6 +22,7 @@ Classifier: Programming Language :: Python :: 3.12
22
22
  Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
23
23
  Requires-Dist: deepl (>=1.18.0,<2.0.0)
24
24
  Requires-Dist: haystack-ai (>=2.3.0,<3.0.0)
25
+ Requires-Dist: pytest-dotenv (>=0.5.2,<0.6.0)
25
26
  Project-URL: Bug Tracker, https://github.com/dribia/deepl-haystack/issues
26
27
  Project-URL: Repository, https://github.com/dribia/deepl-haystack
27
28
  Description-Content-Type: text/markdown
@@ -35,19 +36,103 @@ DeepL Haystack Integration
35
36
  | Package | [![PyPI](https://img.shields.io/pypi/v/deepl-haystack)](https://pypi.org/project/deepl-haystack/) ![PyPI - Downloads](https://img.shields.io/pypi/dm/deepl-haystack?color=blue&logo=pypi&logoColor=gold) ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/deepl-haystack?logo=python&logoColor=gold) [![GitHub](https://img.shields.io/github/license/dribia/deepl-haystack?color=blue)](LICENSE) |
36
37
  ---
37
38
 
38
- **Table of Contents**
39
-
40
- - [deepl-haystack](#deepl-haystack-integration)
41
- - [Installation](#installation)
42
- - [Contributing](#contributing)
43
- - [License](#license)
44
-
45
39
  ## Installation
46
40
 
41
+ This project resides in the Python Package Index (PyPI), so it can easily be installed with `pip`:
42
+
47
43
  ```console
48
44
  pip install deepl-haystack
49
45
  ```
50
46
 
47
+ ## Usage
48
+
49
+ The DeepL Haystack integration provides two Haystack components: `DeepLTextTranslator`
50
+ and `DeepLDocumentTranslator`. These components can be used to translate text and documents,
51
+ respectively, using the DeepL API.
52
+
53
+ ## Examples
54
+
55
+ To run these examples you'll need a working DeepL API key.
56
+ You can get one by signing up at the [DeepL API website](https://www.deepl.com/pro#developer).
57
+
58
+ ### Standalone Text Translation
59
+
60
+ ```python
61
+ from haystack.utils import Secret
62
+
63
+ from deepl_haystack import DeepLTextTranslator
64
+
65
+ translator = DeepLTextTranslator(
66
+ api_key=Secret.from_token("your_api_key_here"), source_lang="EN", target_lang="ES"
67
+ )
68
+
69
+ translated_text = translator.run("Hello, world!")
70
+ print(translated_text)
71
+ # {'translation': '¡Hola, mundo!', 'meta': {'source_lang': 'EN', 'target_lang': 'ES'}}
72
+ ```
73
+
74
+ ### Standalone Document Translation
75
+
76
+ ```python
77
+ from haystack.dataclasses import Document
78
+ from haystack.utils import Secret
79
+
80
+ from deepl_haystack import DeepLDocumentTranslator
81
+
82
+ translator = DeepLDocumentTranslator(
83
+ api_key=Secret.from_token("your_api_key_here"), source_lang="EN", target_lang="ES"
84
+ )
85
+
86
+ documents_to_translate = [
87
+ Document(content="Hello, world!"),
88
+ Document(content="Goodbye, Joe!", meta={"name": "Joe"}),
89
+ ]
90
+
91
+ translated_documents = translator.run(documents_to_translate)
92
+ print("\n".join([f"{doc.content}, {doc.meta}" for doc in translated_documents]))
93
+ # ¡Hola, mundo!, {'source_lang': 'EN', 'target_lang': 'ES'}
94
+ # ¡Adiós, Joe!, {'name': 'Joe', 'source_lang': 'EN', 'target_lang': 'ES'}
95
+ ```
96
+
97
+ ### Haystack Pipeline Integration
98
+
99
+ > [!TIP]
100
+ > To run this example, you'll need to have the `MarkdownToDocument` requirements installed:
101
+ > ```shell
102
+ > pip install markdown-it-py, mdit-plain
103
+ > ```
104
+
105
+ ```python
106
+ from haystack import Pipeline
107
+ from haystack.components.converters import MarkdownToDocument
108
+ from haystack.components.writers import DocumentWriter
109
+ from haystack.dataclasses.byte_stream import ByteStream
110
+ from haystack.document_stores.in_memory import InMemoryDocumentStore
111
+ from haystack.utils import Secret
112
+
113
+ from deepl_haystack import DeepLDocumentTranslator
114
+
115
+ document_store = InMemoryDocumentStore()
116
+
117
+ pipeline = Pipeline()
118
+ pipeline.add_component(instance=MarkdownToDocument(), name="converter")
119
+ pipeline.add_component(
120
+ instance=DeepLDocumentTranslator(
121
+ api_key=Secret.from_token("f9b24fba-2267-463a-97b1-6f211ad6197a:fx"),
122
+ target_lang="ES",
123
+ ),
124
+ name="translator",
125
+ )
126
+ pipeline.add_component(
127
+ instance=DocumentWriter(document_store=document_store), name="document_store"
128
+ )
129
+ pipeline.connect("converter", "translator")
130
+ pipeline.connect("translator", "document_store")
131
+ pipeline.run({"converter": {"sources": [ByteStream.from_string("# Hello world!")]}})
132
+ print(document_store.filter_documents())
133
+ # [Document(id=..., content: '¡Hola, mundo!', meta: {'source_lang': 'EN', 'language': 'ES'})]
134
+ ```
135
+
51
136
  ## Contributing
52
137
 
53
138
  [Poetry](https://python-poetry.org) is the best way to interact with this project, to install it,
@@ -59,13 +144,13 @@ With `poetry` installed, one can install the project dependencies with:
59
144
  poetry install
60
145
  ```
61
146
 
62
- Then, to run the tests:
147
+ Then, to run the project unit tests:
63
148
 
64
149
  ```shell
65
- make test
150
+ make test-unit
66
151
  ```
67
152
 
68
- To run the linters `ruff` and `mypy`:
153
+ To run the linters (`ruff` and `mypy`):
69
154
 
70
155
  ```shell
71
156
  make lint
@@ -77,8 +162,23 @@ To apply all code formatting:
77
162
  make format
78
163
  ```
79
164
 
165
+ And finally, to run the project integration tests (which actually use the DeepL API),
166
+ you should either have the `DEEPL_API_KEY` environment variable set,
167
+ or create a `.env` file:
168
+
169
+ ```dotenv
170
+ DEEPL_API_KEY=your_api_key_here
171
+ ```
172
+
173
+ And run:
174
+
175
+ ```shell
176
+ make test-integration
177
+ ```
178
+
80
179
  ## License
81
180
 
82
181
  `deepl-haystack` is distributed under the terms of the
83
182
  [MIT](https://opensource.org/license/mit) license.
183
+ Check the [LICENSE](./LICENSE) file for further details.
84
184
 
deepl_haystack-0.2.1/README.md ADDED
@@ -0,0 +1,154 @@
1
+ DeepL Haystack Integration
2
+ ==========================
3
+
4
+ | | |
5
+ |---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
6
+ | CI/CD | [![Tests](https://github.com/dribia/deepl-haystack/actions/workflows/test.yml/badge.svg)](https://github.com/dribia/deepl-haystack/actions/workflows/test.yml) [![Coverage Status](https://img.shields.io/codecov/c/github/dribia/driconfig)](https://codecov.io/gh/dribia/driconfig) [![Tests](https://github.com/dribia/deepl-haystack/actions/workflows/lint.yml/badge.svg)](https://github.com/dribia/deepl-haystack/actions/workflows/lint.yml) [![types - Mypy](https://img.shields.io/badge/types-Mypy-blue.svg)](https://github.com/python/mypy) [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) |
7
+ | Package | [![PyPI](https://img.shields.io/pypi/v/deepl-haystack)](https://pypi.org/project/deepl-haystack/) ![PyPI - Downloads](https://img.shields.io/pypi/dm/deepl-haystack?color=blue&logo=pypi&logoColor=gold) ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/deepl-haystack?logo=python&logoColor=gold) [![GitHub](https://img.shields.io/github/license/dribia/deepl-haystack?color=blue)](LICENSE) |
8
+ ---
9
+
10
+ ## Installation
11
+
12
+ This project resides in the Python Package Index (PyPI), so it can easily be installed with `pip`:
13
+
14
+ ```console
15
+ pip install deepl-haystack
16
+ ```
17
+
18
+ ## Usage
19
+
20
+ The DeepL Haystack integration provides two Haystack components: `DeepLTextTranslator`
21
+ and `DeepLDocumentTranslator`. These components can be used to translate text and documents,
22
+ respectively, using the DeepL API.
23
+
24
+ ## Examples
25
+
26
+ To run these examples you'll need a working DeepL API key.
27
+ You can get one by signing up at the [DeepL API website](https://www.deepl.com/pro#developer).
28
+
29
+ ### Standalone Text Translation
30
+
31
+ ```python
32
+ from haystack.utils import Secret
33
+
34
+ from deepl_haystack import DeepLTextTranslator
35
+
36
+ translator = DeepLTextTranslator(
37
+ api_key=Secret.from_token("your_api_key_here"), source_lang="EN", target_lang="ES"
38
+ )
39
+
40
+ translated_text = translator.run("Hello, world!")
41
+ print(translated_text)
42
+ # {'translation': '¡Hola, mundo!', 'meta': {'source_lang': 'EN', 'target_lang': 'ES'}}
43
+ ```
44
+
45
+ ### Standalone Document Translation
46
+
47
+ ```python
48
+ from haystack.dataclasses import Document
49
+ from haystack.utils import Secret
50
+
51
+ from deepl_haystack import DeepLDocumentTranslator
52
+
53
+ translator = DeepLDocumentTranslator(
54
+ api_key=Secret.from_token("your_api_key_here"), source_lang="EN", target_lang="ES"
55
+ )
56
+
57
+ documents_to_translate = [
58
+ Document(content="Hello, world!"),
59
+ Document(content="Goodbye, Joe!", meta={"name": "Joe"}),
60
+ ]
61
+
62
+ translated_documents = translator.run(documents_to_translate)
63
+ print("\n".join([f"{doc.content}, {doc.meta}" for doc in translated_documents]))
64
+ # ¡Hola, mundo!, {'source_lang': 'EN', 'target_lang': 'ES'}
65
+ # ¡Adiós, Joe!, {'name': 'Joe', 'source_lang': 'EN', 'target_lang': 'ES'}
66
+ ```
67
+
68
+ ### Haystack Pipeline Integration
69
+
70
+ > [!TIP]
71
+ > To run this example, you'll need to have the `MarkdownToDocument` requirements installed:
72
+ > ```shell
73
+ > pip install markdown-it-py, mdit-plain
74
+ > ```
75
+
76
+ ```python
77
+ from haystack import Pipeline
78
+ from haystack.components.converters import MarkdownToDocument
79
+ from haystack.components.writers import DocumentWriter
80
+ from haystack.dataclasses.byte_stream import ByteStream
81
+ from haystack.document_stores.in_memory import InMemoryDocumentStore
82
+ from haystack.utils import Secret
83
+
84
+ from deepl_haystack import DeepLDocumentTranslator
85
+
86
+ document_store = InMemoryDocumentStore()
87
+
88
+ pipeline = Pipeline()
89
+ pipeline.add_component(instance=MarkdownToDocument(), name="converter")
90
+ pipeline.add_component(
91
+ instance=DeepLDocumentTranslator(
92
+ api_key=Secret.from_token("f9b24fba-2267-463a-97b1-6f211ad6197a:fx"),
93
+ target_lang="ES",
94
+ ),
95
+ name="translator",
96
+ )
97
+ pipeline.add_component(
98
+ instance=DocumentWriter(document_store=document_store), name="document_store"
99
+ )
100
+ pipeline.connect("converter", "translator")
101
+ pipeline.connect("translator", "document_store")
102
+ pipeline.run({"converter": {"sources": [ByteStream.from_string("# Hello world!")]}})
103
+ print(document_store.filter_documents())
104
+ # [Document(id=..., content: '¡Hola, mundo!', meta: {'source_lang': 'EN', 'language': 'ES'})]
105
+ ```
106
+
107
+ ## Contributing
108
+
109
+ [Poetry](https://python-poetry.org) is the best way to interact with this project, to install it,
110
+ follow the official [Poetry installation guide](https://python-poetry.org/docs/#installation).
111
+
112
+ With `poetry` installed, one can install the project dependencies with:
113
+
114
+ ```shell
115
+ poetry install
116
+ ```
117
+
118
+ Then, to run the project unit tests:
119
+
120
+ ```shell
121
+ make test-unit
122
+ ```
123
+
124
+ To run the linters (`ruff` and `mypy`):
125
+
126
+ ```shell
127
+ make lint
128
+ ```
129
+
130
+ To apply all code formatting:
131
+
132
+ ```shell
133
+ make format
134
+ ```
135
+
136
+ And finally, to run the project integration tests (which actually use the DeepL API),
137
+ you should either have the `DEEPL_API_KEY` environment variable set,
138
+ or create a `.env` file:
139
+
140
+ ```dotenv
141
+ DEEPL_API_KEY=your_api_key_here
142
+ ```
143
+
144
+ And run:
145
+
146
+ ```shell
147
+ make test-integration
148
+ ```
149
+
150
+ ## License
151
+
152
+ `deepl-haystack` is distributed under the terms of the
153
+ [MIT](https://opensource.org/license/mit) license.
154
+ Check the [LICENSE](./LICENSE) file for further details.
deepl_haystack-0.2.1/deepl_haystack/__init__.py ADDED
@@ -0,0 +1,12 @@
1
+ """DeepL Haystack integration."""
2
+
3
+ from importlib.metadata import PackageNotFoundError, version
4
+
5
+ from .components import DeepLDocumentTranslator, DeepLTextTranslator
6
+
7
+ try:
8
+ __version__ = version(__name__)
9
+ except PackageNotFoundError:
10
+ __version__ = "unknown"
11
+
12
+ __all__ = ["DeepLDocumentTranslator", "DeepLTextTranslator"]
deepl_haystack-0.1.1/deepl_haystack/components/translators/deepl/translator.py → deepl_haystack-0.2.1/deepl_haystack/components.py RENAMED
@@ -1,4 +1,4 @@
1
- """DeepL translator component."""
1
+ """DeepL translator components."""
2
2
 
3
3
  from typing import Any, Dict, List, Optional, Union
4
4
 
@@ -69,7 +69,7 @@ class DeepLTextTranslator:
69
69
  assert isinstance(translation, TextResult)
70
70
  meta = {
71
71
  "source_lang": translation.detected_source_lang,
72
- "target_lang": self.target_lang,
72
+ "language": self.target_lang,
73
73
  }
74
74
  return {"translation": translation.text, "meta": meta}
75
75
 
@@ -163,17 +163,19 @@ class DeepLDocumentTranslator:
163
163
  assert isinstance(translations, list)
164
164
  for _translation in translations:
165
165
  pass
166
- return [
167
- Document(
168
- content=translation.text,
169
- meta=dict(
170
- **document.meta,
171
- source_lang=translation.detected_source_lang,
172
- target_lang=self.target_lang,
173
- ),
174
- )
175
- for document, translation in zip(documents, translations)
176
- ]
166
+ return {
167
+ "documents": [
168
+ Document(
169
+ content=translation.text,
170
+ meta=dict(
171
+ **document.meta,
172
+ source_lang=translation.detected_source_lang,
173
+ language=self.target_lang,
174
+ ),
175
+ )
176
+ for document, translation in zip(documents, translations)
177
+ ]
178
+ }
177
179
 
178
180
  def to_dict(self) -> Dict[str, Any]:
179
181
  """Serializes the component to a dictionary.
{deepl_haystack-0.1.1 → deepl_haystack-0.2.1}/pyproject.toml RENAMED
@@ -1,6 +1,6 @@
1
1
  [tool.poetry]
2
2
  name = "deepl-haystack"
3
- version = "0.1.1"
3
+ version = "0.2.1"
4
4
  description = "Haystack integration with DeepL translation services provider."
5
5
  authors = ["Albert Iribarne <iribarne@dribia.com>"]
6
6
  maintainers = ["Albert Iribarne <iribarne@dribia.com>", "Dribia Data Research <code@dribia.com>"]
@@ -23,6 +23,7 @@ keywords = ["haystack", "deepl", "translation", "nlp"]
23
23
  python = ">=3.9,<3.13"
24
24
  deepl = "^1.18.0"
25
25
  haystack-ai = "^2.3.0"
26
+ pytest-dotenv = "^0.5.2"
26
27
 
27
28
  [tool.poetry.group.test.dependencies]
28
29
  pytest = "^8.3.1"
@@ -35,6 +36,8 @@ mypy = "1.11.0"
35
36
 
36
37
  [tool.poetry.group.dev.dependencies]
37
38
  pre-commit = "^3.7.1"
39
+ markdown-it-py = "^3.0.0"
40
+ mdit-plain = "^1.0.1"
38
41
 
39
42
  [tool.ruff]
40
43
 
@@ -84,10 +87,12 @@ addopts = "--verbose"
84
87
  filterwarnings = 'error'
85
88
  xfail_strict = true
86
89
  testpaths = ["tests"]
87
- markers = []
90
+ markers = [
91
+ "integration: integration tests",
92
+ ]
88
93
 
89
94
  [tool.coverage.run]
90
- source = ["deepl_haystack", "tests"]
95
+ source = ["deepl_haystack"]
91
96
  branch = true
92
97
 
93
98
  [tool.coverage.report]
deepl_haystack-0.1.1/README.md DELETED
@@ -1,55 +0,0 @@
1
- DeepL Haystack Integration
2
- ==========================
3
-
4
- | | |
5
- |---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
6
- | CI/CD | [![Tests](https://github.com/dribia/deepl-haystack/actions/workflows/test.yml/badge.svg)](https://github.com/dribia/deepl-haystack/actions/workflows/test.yml) [![Coverage Status](https://img.shields.io/codecov/c/github/dribia/driconfig)](https://codecov.io/gh/dribia/driconfig) [![Tests](https://github.com/dribia/deepl-haystack/actions/workflows/lint.yml/badge.svg)](https://github.com/dribia/deepl-haystack/actions/workflows/lint.yml) [![types - Mypy](https://img.shields.io/badge/types-Mypy-blue.svg)](https://github.com/python/mypy) [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) |
7
- | Package | [![PyPI](https://img.shields.io/pypi/v/deepl-haystack)](https://pypi.org/project/deepl-haystack/) ![PyPI - Downloads](https://img.shields.io/pypi/dm/deepl-haystack?color=blue&logo=pypi&logoColor=gold) ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/deepl-haystack?logo=python&logoColor=gold) [![GitHub](https://img.shields.io/github/license/dribia/deepl-haystack?color=blue)](LICENSE) |
8
- ---
9
-
10
- **Table of Contents**
11
-
12
- - [deepl-haystack](#deepl-haystack-integration)
13
- - [Installation](#installation)
14
- - [Contributing](#contributing)
15
- - [License](#license)
16
-
17
- ## Installation
18
-
19
- ```console
20
- pip install deepl-haystack
21
- ```
22
-
23
- ## Contributing
24
-
25
- [Poetry](https://python-poetry.org) is the best way to interact with this project, to install it,
26
- follow the official [Poetry installation guide](https://python-poetry.org/docs/#installation).
27
-
28
- With `poetry` installed, one can install the project dependencies with:
29
-
30
- ```shell
31
- poetry install
32
- ```
33
-
34
- Then, to run the tests:
35
-
36
- ```shell
37
- make test
38
- ```
39
-
40
- To run the linters `ruff` and `mypy`:
41
-
42
- ```shell
43
- make lint
44
- ```
45
-
46
- To apply all code formatting:
47
-
48
- ```shell
49
- make format
50
- ```
51
-
52
- ## License
53
-
54
- `deepl-haystack` is distributed under the terms of the
55
- [MIT](https://opensource.org/license/mit) license.
deepl_haystack-0.1.1/deepl_haystack/__init__.py DELETED
@@ -1,10 +0,0 @@
1
- """DeepL Haystack integration."""
2
-
3
- from .components.translators.deepl.translator import (
4
- DeepLDocumentTranslator,
5
- DeepLTextTranslator,
6
- )
7
-
8
- __version__ = "0.1.1"
9
-
10
- __all__ = ["DeepLDocumentTranslator", "DeepLTextTranslator"]
{deepl_haystack-0.1.1 → deepl_haystack-0.2.1}/LICENSE RENAMED
File without changes