attachments 0.2.0__tar.gz → 0.2.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {attachments-0.2.0/src/attachments.egg-info → attachments-0.2.1}/PKG-INFO +49 -2
- {attachments-0.2.0 → attachments-0.2.1}/README.md +48 -1
- {attachments-0.2.0 → attachments-0.2.1}/pyproject.toml +1 -1
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments/renderers.py +33 -19
- {attachments-0.2.0 → attachments-0.2.1/src/attachments.egg-info}/PKG-INFO +49 -2
- {attachments-0.2.0 → attachments-0.2.1}/LICENSE +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/setup.cfg +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments/__init__.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments/audio_processing.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments/config.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments/core.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments/detectors.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments/exceptions.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments/image_processing.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments/office_contact_sheet.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments/parsers.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments/utils.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments.egg-info/SOURCES.txt +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments.egg-info/dependency_links.txt +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments.egg-info/requires.txt +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/src/attachments.egg-info/top_level.txt +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/tests/test_audio_processing.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/tests/test_base.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/tests/test_core_functionality.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/tests/test_document_parsing.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/tests/test_html_parsing.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/tests/test_image_processing.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/tests/test_indexing_slicing.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/tests/test_markdown_rendering.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/tests/test_office_contact_sheet.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/tests/test_pdf_parsing.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/tests/test_pptx_parsing.py +0 -0
- {attachments-0.2.0 → attachments-0.2.1}/tests/test_simple.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: attachments
|
|
3
|
-
Version: 0.2.
|
|
3
|
+
Version: 0.2.1
|
|
4
4
|
Summary: A Python library to handle various file types for LLMs.
|
|
5
5
|
Author-email: Maxim Rivest <mrive052@gmail.com>
|
|
6
6
|
License-Expression: MIT
|
|
@@ -38,7 +38,54 @@ Requires-Dist: filelock; extra == "dev"
|
|
|
38
38
|
Dynamic: license-file
|
|
39
39
|
|
|
40
40
|
# Attachments
|
|
41
|
-
##
|
|
41
|
+
## Easiest way to give context to LLMs
|
|
42
|
+
|
|
43
|
+
Attachments has the ambition to be the general funnel for any files to be transformed into images+text for large language models context. For this to be true we, has a community, have to stop implementing this funnel (file -> images+text) each on our own. If you have a detector + parser + renderer contibute it! This will help you and us. Don't let your code alone in a dark corner of you computer.
|
|
44
|
+
|
|
45
|
+
## Very quickstart
|
|
46
|
+
|
|
47
|
+
```python
|
|
48
|
+
from attachments import Attachments
|
|
49
|
+
|
|
50
|
+
a = Attachment("/the/path/or/url/to/your/to/the/content/you/want/to/give/the/llm.xlsx", "another.pdf", "another.pptx"...)
|
|
51
|
+
|
|
52
|
+
prompt_ready_context = str(a)
|
|
53
|
+
images_ready_for_llm = a.images
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
That is really just it!
|
|
57
|
+
|
|
58
|
+
You can print it `print(a)`, you can interpolate it `f"the content is {a}"` you can string it `str(a)`. This will give you something very good to give the llm so that the AI can consider you content.
|
|
59
|
+
|
|
60
|
+
Nowadays, most genAI models comes with the ability to see images too. So you also have all attachments in images forms by using `a.images`.
|
|
61
|
+
This is a list of base64 encoded images, this is the fundamental format the most llm provider accept.
|
|
62
|
+
|
|
63
|
+
The simplest way to use and think about attachments is that if you want to put you best foot forward and up you chances that the llm grok your content you should pass all of the text in `a` to the prompt and all of the images in `a.images` in the image input. We aim for making those two as 'prompt engineered' as possible. *Attachments* is young but already very powerful and used in production. The api will not change. Maybe advanced feature and syntax will be added but the core will stay the same. Mostly we will support more file types and we will have better rendering for better performance and with less extreneous tokens.
|
|
64
|
+
|
|
65
|
+
## How to give attachments to openai llms?
|
|
66
|
+
|
|
67
|
+
```python
|
|
68
|
+
from openai import OpenAI
|
|
69
|
+
from attachments import Attachments
|
|
70
|
+
|
|
71
|
+
pdf_attachment = Attachments("https://github.com/microsoft/markitdown/raw/refs/heads/main/packages/markitdown/tests/test_files/test.pdf")
|
|
72
|
+
|
|
73
|
+
prompt = f"""
|
|
74
|
+
Analyze the following documents:
|
|
75
|
+
{pdf_attachment}
|
|
76
|
+
"""
|
|
77
|
+
|
|
78
|
+
content = [{"type": "input_image","image_url": image} for image in pdf_attachment.images] + \
|
|
79
|
+
[{"type": "input_text", "text": prompt}]
|
|
80
|
+
|
|
81
|
+
response = OpenAI().responses.create(model="gpt-4.1-nano", input=[{"role": "user", "content": content}])
|
|
82
|
+
response.output_text
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
|
|
86
|
+
|
|
87
|
+
|
|
88
|
+
|
|
42
89
|
A Python library designed to seamlessly handle various file types (local or URLs), process them, and present them in formats suitable for Large Language Models (LLMs).
|
|
43
90
|
|
|
44
91
|
It is meant to be very minimal.
|
|
@@ -1,5 +1,52 @@
|
|
|
1
1
|
# Attachments
|
|
2
|
-
##
|
|
2
|
+
## Easiest way to give context to LLMs
|
|
3
|
+
|
|
4
|
+
Attachments has the ambition to be the general funnel for any files to be transformed into images+text for large language models context. For this to be true we, has a community, have to stop implementing this funnel (file -> images+text) each on our own. If you have a detector + parser + renderer contibute it! This will help you and us. Don't let your code alone in a dark corner of you computer.
|
|
5
|
+
|
|
6
|
+
## Very quickstart
|
|
7
|
+
|
|
8
|
+
```python
|
|
9
|
+
from attachments import Attachments
|
|
10
|
+
|
|
11
|
+
a = Attachment("/the/path/or/url/to/your/to/the/content/you/want/to/give/the/llm.xlsx", "another.pdf", "another.pptx"...)
|
|
12
|
+
|
|
13
|
+
prompt_ready_context = str(a)
|
|
14
|
+
images_ready_for_llm = a.images
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
That is really just it!
|
|
18
|
+
|
|
19
|
+
You can print it `print(a)`, you can interpolate it `f"the content is {a}"` you can string it `str(a)`. This will give you something very good to give the llm so that the AI can consider you content.
|
|
20
|
+
|
|
21
|
+
Nowadays, most genAI models comes with the ability to see images too. So you also have all attachments in images forms by using `a.images`.
|
|
22
|
+
This is a list of base64 encoded images, this is the fundamental format the most llm provider accept.
|
|
23
|
+
|
|
24
|
+
The simplest way to use and think about attachments is that if you want to put you best foot forward and up you chances that the llm grok your content you should pass all of the text in `a` to the prompt and all of the images in `a.images` in the image input. We aim for making those two as 'prompt engineered' as possible. *Attachments* is young but already very powerful and used in production. The api will not change. Maybe advanced feature and syntax will be added but the core will stay the same. Mostly we will support more file types and we will have better rendering for better performance and with less extreneous tokens.
|
|
25
|
+
|
|
26
|
+
## How to give attachments to openai llms?
|
|
27
|
+
|
|
28
|
+
```python
|
|
29
|
+
from openai import OpenAI
|
|
30
|
+
from attachments import Attachments
|
|
31
|
+
|
|
32
|
+
pdf_attachment = Attachments("https://github.com/microsoft/markitdown/raw/refs/heads/main/packages/markitdown/tests/test_files/test.pdf")
|
|
33
|
+
|
|
34
|
+
prompt = f"""
|
|
35
|
+
Analyze the following documents:
|
|
36
|
+
{pdf_attachment}
|
|
37
|
+
"""
|
|
38
|
+
|
|
39
|
+
content = [{"type": "input_image","image_url": image} for image in pdf_attachment.images] + \
|
|
40
|
+
[{"type": "input_text", "text": prompt}]
|
|
41
|
+
|
|
42
|
+
response = OpenAI().responses.create(model="gpt-4.1-nano", input=[{"role": "user", "content": content}])
|
|
43
|
+
response.output_text
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
|
|
47
|
+
|
|
48
|
+
|
|
49
|
+
|
|
3
50
|
A Python library designed to seamlessly handle various file types (local or URLs), process them, and present them in formats suitable for Large Language Models (LLMs).
|
|
4
51
|
|
|
5
52
|
It is meant to be very minimal.
|
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
from abc import ABC, abstractmethod
|
|
4
4
|
from typing import List, Dict, Any
|
|
5
5
|
import xml.etree.ElementTree as ET
|
|
6
|
+
import re
|
|
6
7
|
|
|
7
8
|
class BaseRenderer(ABC):
|
|
8
9
|
"""Abstract base class for content renderers."""
|
|
@@ -51,12 +52,35 @@ class RendererRegistry:
|
|
|
51
52
|
|
|
52
53
|
class DefaultXMLRenderer(BaseRenderer):
|
|
53
54
|
"""Renders parsed content into a default XML-like string format."""
|
|
55
|
+
def _sanitize_xml_attr(self, value):
|
|
56
|
+
"""Ensure value is a string and escape/replace invalid XML attribute characters."""
|
|
57
|
+
if value is None:
|
|
58
|
+
return ""
|
|
59
|
+
value = str(value)
|
|
60
|
+
# Remove control characters except tab, newline, carriage return
|
|
61
|
+
value = re.sub(r'[\x00-\x08\x0B\x0C\x0E-\x1F\x7F]', '', value)
|
|
62
|
+
# Replace invalid XML attribute chars (", <, >, &, ')
|
|
63
|
+
value = value.replace('&', '&').replace('<', '<').replace('>', '>')
|
|
64
|
+
value = value.replace('"', '"').replace("'", ''')
|
|
65
|
+
return value
|
|
66
|
+
|
|
67
|
+
def _sanitize_xml_text(self, value):
|
|
68
|
+
"""Ensure value is a string and escape/replace invalid XML text characters."""
|
|
69
|
+
if value is None:
|
|
70
|
+
return ""
|
|
71
|
+
value = str(value)
|
|
72
|
+
# Remove control characters except tab, newline, carriage return
|
|
73
|
+
value = re.sub(r'[\x00-\x08\x0B\x0C\x0E-\x1F\x7F]', '', value)
|
|
74
|
+
# ElementTree will escape &, <, > automatically in text, but we can pre-escape for safety
|
|
75
|
+
value = value.replace('&', '&').replace('<', '<').replace('>', '>')
|
|
76
|
+
return value
|
|
77
|
+
|
|
54
78
|
def render(self, attachments_data: List[Dict[str, Any]]) -> str:
|
|
55
79
|
root = ET.Element("attachments")
|
|
56
80
|
for item_data in attachments_data:
|
|
57
|
-
item_id = item_data.get("id", "unknown")
|
|
58
|
-
item_type = item_data.get("type", "unknown")
|
|
59
|
-
original_path = item_data.get("original_path_str", item_data.get("file_path", "N/A"))
|
|
81
|
+
item_id = self._sanitize_xml_attr(item_data.get("id", "unknown"))
|
|
82
|
+
item_type = self._sanitize_xml_attr(item_data.get("type", "unknown"))
|
|
83
|
+
original_path = self._sanitize_xml_attr(item_data.get("original_path_str", item_data.get("file_path", "N/A")))
|
|
60
84
|
|
|
61
85
|
attachment_element = ET.SubElement(root, "attachment")
|
|
62
86
|
attachment_element.set("id", item_id)
|
|
@@ -64,31 +88,21 @@ class DefaultXMLRenderer(BaseRenderer):
|
|
|
64
88
|
attachment_element.set("original_path", original_path)
|
|
65
89
|
|
|
66
90
|
content_text = item_data.get("text", "")
|
|
67
|
-
|
|
68
91
|
content_element = ET.SubElement(attachment_element, "content")
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
# Actual CDATA wrapping might need to happen during serialization if the library doesn't do it.
|
|
72
|
-
if content_text: # Only add text if it's not empty
|
|
73
|
-
content_element.text = content_text # Store as regular text; XML serializer should handle escaping.
|
|
74
|
-
# For literal CDATA, one might need a custom serializer or to build the string manually.
|
|
92
|
+
if content_text:
|
|
93
|
+
content_element.text = self._sanitize_xml_text(content_text)
|
|
75
94
|
else:
|
|
76
|
-
# If there's no text content, we can leave the content tag empty
|
|
77
|
-
# or add a comment, or omit it. For now, an empty tag is fine.
|
|
78
95
|
pass
|
|
79
96
|
|
|
80
|
-
# Serialize to string with pretty print
|
|
81
|
-
# ET.indent(root) # Available in Python 3.9+
|
|
82
97
|
xml_str = ET.tostring(root, encoding="utf-8").decode("utf-8")
|
|
83
|
-
|
|
84
|
-
# Basic pretty printing for older Python versions or if ET.indent is not sufficient
|
|
85
98
|
try:
|
|
86
99
|
import xml.dom.minidom
|
|
87
100
|
dom = xml.dom.minidom.parseString(xml_str)
|
|
88
101
|
return dom.toprettyxml(indent=" ")
|
|
89
|
-
except
|
|
90
|
-
|
|
91
|
-
|
|
102
|
+
except Exception as e:
|
|
103
|
+
print("[attachments] XML rendering error. Invalid XML string was:\n", xml_str)
|
|
104
|
+
print("Exception:", e)
|
|
105
|
+
raise
|
|
92
106
|
|
|
93
107
|
class PlainTextRenderer(BaseRenderer):
|
|
94
108
|
"""Renders parsed content into a simple plain text string,
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: attachments
|
|
3
|
-
Version: 0.2.
|
|
3
|
+
Version: 0.2.1
|
|
4
4
|
Summary: A Python library to handle various file types for LLMs.
|
|
5
5
|
Author-email: Maxim Rivest <mrive052@gmail.com>
|
|
6
6
|
License-Expression: MIT
|
|
@@ -38,7 +38,54 @@ Requires-Dist: filelock; extra == "dev"
|
|
|
38
38
|
Dynamic: license-file
|
|
39
39
|
|
|
40
40
|
# Attachments
|
|
41
|
-
##
|
|
41
|
+
## Easiest way to give context to LLMs
|
|
42
|
+
|
|
43
|
+
Attachments has the ambition to be the general funnel for any files to be transformed into images+text for large language models context. For this to be true we, has a community, have to stop implementing this funnel (file -> images+text) each on our own. If you have a detector + parser + renderer contibute it! This will help you and us. Don't let your code alone in a dark corner of you computer.
|
|
44
|
+
|
|
45
|
+
## Very quickstart
|
|
46
|
+
|
|
47
|
+
```python
|
|
48
|
+
from attachments import Attachments
|
|
49
|
+
|
|
50
|
+
a = Attachment("/the/path/or/url/to/your/to/the/content/you/want/to/give/the/llm.xlsx", "another.pdf", "another.pptx"...)
|
|
51
|
+
|
|
52
|
+
prompt_ready_context = str(a)
|
|
53
|
+
images_ready_for_llm = a.images
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
That is really just it!
|
|
57
|
+
|
|
58
|
+
You can print it `print(a)`, you can interpolate it `f"the content is {a}"` you can string it `str(a)`. This will give you something very good to give the llm so that the AI can consider you content.
|
|
59
|
+
|
|
60
|
+
Nowadays, most genAI models comes with the ability to see images too. So you also have all attachments in images forms by using `a.images`.
|
|
61
|
+
This is a list of base64 encoded images, this is the fundamental format the most llm provider accept.
|
|
62
|
+
|
|
63
|
+
The simplest way to use and think about attachments is that if you want to put you best foot forward and up you chances that the llm grok your content you should pass all of the text in `a` to the prompt and all of the images in `a.images` in the image input. We aim for making those two as 'prompt engineered' as possible. *Attachments* is young but already very powerful and used in production. The api will not change. Maybe advanced feature and syntax will be added but the core will stay the same. Mostly we will support more file types and we will have better rendering for better performance and with less extreneous tokens.
|
|
64
|
+
|
|
65
|
+
## How to give attachments to openai llms?
|
|
66
|
+
|
|
67
|
+
```python
|
|
68
|
+
from openai import OpenAI
|
|
69
|
+
from attachments import Attachments
|
|
70
|
+
|
|
71
|
+
pdf_attachment = Attachments("https://github.com/microsoft/markitdown/raw/refs/heads/main/packages/markitdown/tests/test_files/test.pdf")
|
|
72
|
+
|
|
73
|
+
prompt = f"""
|
|
74
|
+
Analyze the following documents:
|
|
75
|
+
{pdf_attachment}
|
|
76
|
+
"""
|
|
77
|
+
|
|
78
|
+
content = [{"type": "input_image","image_url": image} for image in pdf_attachment.images] + \
|
|
79
|
+
[{"type": "input_text", "text": prompt}]
|
|
80
|
+
|
|
81
|
+
response = OpenAI().responses.create(model="gpt-4.1-nano", input=[{"role": "user", "content": content}])
|
|
82
|
+
response.output_text
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
|
|
86
|
+
|
|
87
|
+
|
|
88
|
+
|
|
42
89
|
A Python library designed to seamlessly handle various file types (local or URLs), process them, and present them in formats suitable for Large Language Models (LLMs).
|
|
43
90
|
|
|
44
91
|
It is meant to be very minimal.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|