streamdown 0.25.0__tar.gz → 0.27.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- streamdown-0.27.0/.aider.chat.history.md +1379 -0
- streamdown-0.27.0/.aider.input.history +69 -0
- streamdown-0.27.0/.aider.tags.cache.v4/cache.db +0 -0
- streamdown-0.27.0/.vimrc +1 -0
- streamdown-0.27.0/24-bit-color.sh +99 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/PKG-INFO +1 -5
- {streamdown-0.25.0 → streamdown-0.27.0}/README.md +0 -4
- streamdown-0.27.0/assets/logo.png +0 -0
- streamdown-0.27.0/assets/logo.svg +116 -0
- streamdown-0.27.0/configurable.png +0 -0
- streamdown-0.27.0/copyable.png +0 -0
- streamdown-0.27.0/dunder.png +0 -0
- streamdown-0.27.0/fucking-garbage.md +19 -0
- streamdown-0.27.0/newdir/file_0.py +22 -0
- streamdown-0.27.0/newdir/file_1.rb +43 -0
- streamdown-0.27.0/newdir/file_2.jl +23 -0
- streamdown-0.27.0/output.txt +19 -0
- streamdown-0.27.0/passthrough.py +60 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/pyproject.toml +1 -1
- streamdown-0.27.0/python-go.png +0 -0
- streamdown-0.27.0/somelog.txt +401 -0
- streamdown-0.27.0/ss-new.py +45 -0
- streamdown-0.27.0/ss.py +73 -0
- streamdown-0.27.0/streamdown/__init__.py +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/streamdown/sd.py +66 -34
- streamdown-0.27.0/streamdown/ss +1 -0
- streamdown-0.27.0/streamdown/ss1 +42 -0
- streamdown-0.27.0/table.png +0 -0
- streamdown-0.27.0/temp.py +75 -0
- streamdown-0.27.0/test.py +13 -0
- streamdown-0.27.0/test_input.md +8 -0
- streamdown-0.27.0/tester.py +29 -0
- streamdown-0.27.0/tests/bg-messed-up.md +8 -0
- streamdown-0.27.0/tests/chinese.md +4 -0
- streamdown-0.27.0/tests/cjj.mv +4 -0
- streamdown-0.27.0/tests/rerun.zsh +3 -0
- streamdown-0.27.0/tools/deploy.sh +17 -0
- streamdown-0.25.0/tests/rerun.zsh +0 -3
- streamdown-0.25.0/tools/deploy.sh +0 -7
- {streamdown-0.25.0 → streamdown-0.27.0}/.gitignore +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/LICENSE.MIT +0 -0
- /streamdown-0.25.0/streamdown/__init__.py → /streamdown-0.27.0/error.txt +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/requirements.txt +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/streamdown/plugins/README.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/streamdown/plugins/latex.py +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/README.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/backtick-with-post-spaces.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/block.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/bold_reset_with_link.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/broken-code.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/broken-example.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/chunk-buffer.sh +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/cjk-table.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/cjk-wrap.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/code.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/example.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/fizzbuzz.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/inline.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/jimmy_webb.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/line-buffer.sh +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/line-wrap.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/links.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/managerie.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/mandlebrot.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/markdown.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/nested-example.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/outline.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/pvgo_512.jpg +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/pythonvgo.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/qwen3.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/slash.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/strip-chunks.sh +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/table-break.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/table_test.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/test.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/test_input.md +0 -0
- {streamdown-0.25.0 → streamdown-0.27.0}/tests/wm.md +0 -0
|
@@ -0,0 +1,1379 @@
|
|
|
1
|
+
|
|
2
|
+
# aider chat started at 2025-03-11 17:24:49
|
|
3
|
+
|
|
4
|
+
> Newer aider version v0.76.2 is available.
|
|
5
|
+
> /home/chris/.local/share/uv/tools/aider-chat/bin/python -m pip install --upgrade --upgrade-strategy only-if-needed aider-chat
|
|
6
|
+
> Run pip install? (Y)es/(N)o [Yes]: y
|
|
7
|
+
> Re-run aider to use new version.
|
|
8
|
+
|
|
9
|
+
# aider chat started at 2025-03-11 17:25:22
|
|
10
|
+
|
|
11
|
+
> No git repo found, create one to track aider's changes (recommended)? (Y)es/(N)o [Yes]: y
|
|
12
|
+
> Added .aider* to .gitignore
|
|
13
|
+
> Git repository created in /home/chris/code/stream
|
|
14
|
+
> /home/chris/.local/bin/aider
|
|
15
|
+
> Found OPENROUTER_API_KEY so using openrouter/anthropic/claude-3.7-sonnet since no --model was specified.
|
|
16
|
+
> Aider v0.76.2
|
|
17
|
+
> Main model: openrouter/anthropic/claude-3.7-sonnet with diff edit format, infinite output
|
|
18
|
+
> Weak model: openrouter/anthropic/claude-3-5-haiku
|
|
19
|
+
> Git repo: .git with 0 files
|
|
20
|
+
> Repo-map: using 4096 tokens, auto refresh
|
|
21
|
+
> https://aider.chat/HISTORY.html#release-notes
|
|
22
|
+
> Would you like to see what's new in this version? (Y)es/(N)o [Yes]: n
|
|
23
|
+
|
|
24
|
+
#### let's make a streaming markdown parser. For now let's support h1-h6, ul, ol, and li.
|
|
25
|
+
|
|
26
|
+
I'll create a streaming markdown parser that supports h1-h6, ul, ol, and li elements. Let's create a new Python module for this.
|
|
27
|
+
|
|
28
|
+
First, I'll create a basic structure with a streaming parser that processes markdown content incrementally.
|
|
29
|
+
|
|
30
|
+
markdown_parser.py
|
|
31
|
+
```python
|
|
32
|
+
<<<<<<< SEARCH
|
|
33
|
+
=======
|
|
34
|
+
"""
|
|
35
|
+
Streaming Markdown Parser
|
|
36
|
+
|
|
37
|
+
A simple streaming parser for Markdown that supports:
|
|
38
|
+
- Headers (h1-h6)
|
|
39
|
+
- Unordered lists (ul)
|
|
40
|
+
- Ordered lists (ol)
|
|
41
|
+
- List items (li)
|
|
42
|
+
"""
|
|
43
|
+
|
|
44
|
+
class MarkdownParser:
|
|
45
|
+
def __init__(self):
|
|
46
|
+
self.in_list = False
|
|
47
|
+
self.list_type = None
|
|
48
|
+
self.list_level = 0
|
|
49
|
+
self.buffer = []
|
|
50
|
+
|
|
51
|
+
def parse_line(self, line):
|
|
52
|
+
"""Parse a single line of markdown and return HTML."""
|
|
53
|
+
line = line.rstrip()
|
|
54
|
+
|
|
55
|
+
# Empty line
|
|
56
|
+
if not line:
|
|
57
|
+
if self.in_list:
|
|
58
|
+
# Close any open list when encountering empty line
|
|
59
|
+
html = self._close_list()
|
|
60
|
+
self.in_list = False
|
|
61
|
+
self.list_type = None
|
|
62
|
+
self.list_level = 0
|
|
63
|
+
return html
|
|
64
|
+
return ""
|
|
65
|
+
|
|
66
|
+
# Headers (h1-h6)
|
|
67
|
+
if line.startswith('#'):
|
|
68
|
+
# Count the number of # at the beginning
|
|
69
|
+
level = 0
|
|
70
|
+
for char in line:
|
|
71
|
+
if char == '#':
|
|
72
|
+
level += 1
|
|
73
|
+
else:
|
|
74
|
+
break
|
|
75
|
+
|
|
76
|
+
if level >= 1 and level <= 6 and (len(line) == level or line[level] == ' '):
|
|
77
|
+
# Close any open list
|
|
78
|
+
html = ""
|
|
79
|
+
if self.in_list:
|
|
80
|
+
html = self._close_list()
|
|
81
|
+
self.in_list = False
|
|
82
|
+
self.list_type = None
|
|
83
|
+
self.list_level = 0
|
|
84
|
+
|
|
85
|
+
# Extract header content (skip the # and any leading space)
|
|
86
|
+
content = line[level:].lstrip()
|
|
87
|
+
return f"{html}<h{level}>{content}</h{level}>"
|
|
88
|
+
|
|
89
|
+
# Unordered list
|
|
90
|
+
if line.lstrip().startswith(('- ', '* ', '+ ')):
|
|
91
|
+
indent = len(line) - len(line.lstrip())
|
|
92
|
+
content = line.lstrip()[2:] # Skip the list marker and space
|
|
93
|
+
|
|
94
|
+
return self._handle_list_item('ul', indent, content)
|
|
95
|
+
|
|
96
|
+
# Ordered list
|
|
97
|
+
if line.lstrip().startswith(tuple(f"{i}. " for i in range(1, 10))):
|
|
98
|
+
indent = len(line) - len(line.lstrip())
|
|
99
|
+
# Skip the digit(s), period, and space
|
|
100
|
+
content = line.lstrip()
|
|
101
|
+
content = content[content.index('. ') + 2:]
|
|
102
|
+
|
|
103
|
+
return self._handle_list_item('ol', indent, content)
|
|
104
|
+
|
|
105
|
+
# Regular paragraph (not implemented in this version)
|
|
106
|
+
# Close any open list
|
|
107
|
+
html = ""
|
|
108
|
+
if self.in_list:
|
|
109
|
+
html = self._close_list()
|
|
110
|
+
self.in_list = False
|
|
111
|
+
self.list_type = None
|
|
112
|
+
self.list_level = 0
|
|
113
|
+
|
|
114
|
+
return html + line
|
|
115
|
+
|
|
116
|
+
def _handle_list_item(self, list_type, indent, content):
|
|
117
|
+
"""Handle a list item, managing nested lists."""
|
|
118
|
+
html = []
|
|
119
|
+
|
|
120
|
+
# Calculate list level based on indentation (4 spaces = 1 level)
|
|
121
|
+
new_level = indent // 4 + 1
|
|
122
|
+
|
|
123
|
+
# If we're not in a list or changing list type
|
|
124
|
+
if not self.in_list:
|
|
125
|
+
html.append(f"<{list_type}>")
|
|
126
|
+
self.in_list = True
|
|
127
|
+
self.list_type = list_type
|
|
128
|
+
self.list_level = 1
|
|
129
|
+
# If we're going deeper in nesting
|
|
130
|
+
elif new_level > self.list_level:
|
|
131
|
+
# Open a new nested list
|
|
132
|
+
html.append(f"<{list_type}>")
|
|
133
|
+
self.list_level = new_level
|
|
134
|
+
self.list_type = list_type
|
|
135
|
+
# If we're going up in the nesting
|
|
136
|
+
elif new_level < self.list_level:
|
|
137
|
+
# Close the appropriate number of lists
|
|
138
|
+
for _ in range(self.list_level - new_level):
|
|
139
|
+
html.append(f"</li></{self.list_type}>")
|
|
140
|
+
self.list_level = new_level
|
|
141
|
+
# Same level but different list type
|
|
142
|
+
elif list_type != self.list_type:
|
|
143
|
+
html.append(f"</li></{self.list_type}>")
|
|
144
|
+
html.append(f"<{list_type}>")
|
|
145
|
+
self.list_type = list_type
|
|
146
|
+
else:
|
|
147
|
+
# Same level, same type - close previous item
|
|
148
|
+
html.append("</li>")
|
|
149
|
+
|
|
150
|
+
# Add the new list item
|
|
151
|
+
html.append(f"<li>{content}")
|
|
152
|
+
|
|
153
|
+
return "".join(html)
|
|
154
|
+
|
|
155
|
+
def _close_list(self):
|
|
156
|
+
"""Close any open list tags."""
|
|
157
|
+
if not self.in_list:
|
|
158
|
+
return ""
|
|
159
|
+
|
|
160
|
+
html = "</li>"
|
|
161
|
+
for _ in range(self.list_level):
|
|
162
|
+
html += f"</{self.list_type}>"
|
|
163
|
+
|
|
164
|
+
return html
|
|
165
|
+
|
|
166
|
+
def parse(self, markdown_text):
|
|
167
|
+
"""Parse a complete markdown text and return HTML."""
|
|
168
|
+
lines = markdown_text.split('\n')
|
|
169
|
+
html_parts = []
|
|
170
|
+
|
|
171
|
+
for line in lines:
|
|
172
|
+
html_parts.append(self.parse_line(line))
|
|
173
|
+
|
|
174
|
+
# Close any open lists at the end
|
|
175
|
+
if self.in_list:
|
|
176
|
+
html_parts.append(self._close_list())
|
|
177
|
+
self.in_list = False
|
|
178
|
+
self.list_type = None
|
|
179
|
+
self.list_level = 0
|
|
180
|
+
|
|
181
|
+
return '\n'.join(filter(None, html_parts))
|
|
182
|
+
|
|
183
|
+
def parse_stream(self, stream):
|
|
184
|
+
"""Parse a stream of markdown lines and yield HTML fragments."""
|
|
185
|
+
for line in stream:
|
|
186
|
+
yield self.parse_line(line)
|
|
187
|
+
|
|
188
|
+
# Close any open lists at the end
|
|
189
|
+
if self.in_list:
|
|
190
|
+
yield self._close_list()
|
|
191
|
+
self.in_list = False
|
|
192
|
+
self.list_type = None
|
|
193
|
+
self.list_level = 0
|
|
194
|
+
>>>>>>> REPLACE
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
Now, let's create a simple example script to demonstrate how to use the parser:
|
|
198
|
+
|
|
199
|
+
example.py
|
|
200
|
+
```python
|
|
201
|
+
<<<<<<< SEARCH
|
|
202
|
+
=======
|
|
203
|
+
#!/usr/bin/env python3
|
|
204
|
+
"""
|
|
205
|
+
Example usage of the Markdown Parser
|
|
206
|
+
"""
|
|
207
|
+
|
|
208
|
+
from markdown_parser import MarkdownParser
|
|
209
|
+
|
|
210
|
+
def main():
|
|
211
|
+
# Example markdown text
|
|
212
|
+
markdown_text = """# Heading 1
|
|
213
|
+
## Heading 2
|
|
214
|
+
### Heading 3
|
|
215
|
+
#### Heading 4
|
|
216
|
+
##### Heading 5
|
|
217
|
+
###### Heading 6
|
|
218
|
+
|
|
219
|
+
- Unordered list item 1
|
|
220
|
+
- Unordered list item 2
|
|
221
|
+
- Nested unordered item 2.1
|
|
222
|
+
- Nested unordered item 2.2
|
|
223
|
+
- Deeply nested item 2.2.1
|
|
224
|
+
- Unordered list item 3
|
|
225
|
+
|
|
226
|
+
1. Ordered list item 1
|
|
227
|
+
2. Ordered list item 2
|
|
228
|
+
1. Nested ordered item 2.1
|
|
229
|
+
2. Nested ordered item 2.2
|
|
230
|
+
3. Ordered list item 3
|
|
231
|
+
|
|
232
|
+
- Mixed list example
|
|
233
|
+
1. Nested ordered in unordered
|
|
234
|
+
2. Another nested item
|
|
235
|
+
- Back to unordered
|
|
236
|
+
"""
|
|
237
|
+
|
|
238
|
+
# Parse the complete text
|
|
239
|
+
parser = MarkdownParser()
|
|
240
|
+
html = parser.parse(markdown_text)
|
|
241
|
+
print("Complete parsing result:")
|
|
242
|
+
print(html)
|
|
243
|
+
print("\n" + "-" * 50 + "\n")
|
|
244
|
+
|
|
245
|
+
# Demonstrate streaming parsing
|
|
246
|
+
print("Streaming parsing result:")
|
|
247
|
+
stream_parser = MarkdownParser()
|
|
248
|
+
for line in markdown_text.split('\n'):
|
|
249
|
+
html_fragment = stream_parser.parse_line(line)
|
|
250
|
+
if html_fragment:
|
|
251
|
+
print(f"Line: {line}")
|
|
252
|
+
print(f"HTML: {html_fragment}")
|
|
253
|
+
print()
|
|
254
|
+
|
|
255
|
+
# Don't forget to close any open tags
|
|
256
|
+
closing_html = stream_parser._close_list() if stream_parser.in_list else ""
|
|
257
|
+
if closing_html:
|
|
258
|
+
print(f"Closing HTML: {closing_html}")
|
|
259
|
+
|
|
260
|
+
if __name__ == "__main__":
|
|
261
|
+
main()
|
|
262
|
+
>>>>>>> REPLACE
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
Let's also create a simple test file to verify our parser works correctly:
|
|
266
|
+
|
|
267
|
+
test_markdown_parser.py
|
|
268
|
+
```python
|
|
269
|
+
<<<<<<< SEARCH
|
|
270
|
+
=======
|
|
271
|
+
#!/usr/bin/env python3
|
|
272
|
+
"""
|
|
273
|
+
Tests for the Markdown Parser
|
|
274
|
+
"""
|
|
275
|
+
|
|
276
|
+
import unittest
|
|
277
|
+
from markdown_parser import MarkdownParser
|
|
278
|
+
|
|
279
|
+
class TestMarkdownParser(unittest.TestCase):
|
|
280
|
+
def setUp(self):
|
|
281
|
+
self.parser = MarkdownParser()
|
|
282
|
+
|
|
283
|
+
def test_headers(self):
|
|
284
|
+
self.assertEqual(self.parser.parse_line("# Header 1"), "<h1>Header 1</h1>")
|
|
285
|
+
self.assertEqual(self.parser.parse_line("## Header 2"), "<h2>Header 2</h2>")
|
|
286
|
+
self.assertEqual(self.parser.parse_line("### Header 3"), "<h3>Header 3</h3>")
|
|
287
|
+
self.assertEqual(self.parser.parse_line("#### Header 4"), "<h4>Header 4</h4>")
|
|
288
|
+
self.assertEqual(self.parser.parse_line("##### Header 5"), "<h5>Header 5</h5>")
|
|
289
|
+
self.assertEqual(self.parser.parse_line("###### Header 6"), "<h6>Header 6</h6>")
|
|
290
|
+
|
|
291
|
+
# Invalid headers
|
|
292
|
+
self.assertEqual(self.parser.parse_line("####### Too many"), "####### Too many")
|
|
293
|
+
self.assertEqual(self.parser.parse_line("#No space"), "#No space")
|
|
294
|
+
|
|
295
|
+
def test_unordered_lists(self):
|
|
296
|
+
# Simple list
|
|
297
|
+
self.assertEqual(self.parser.parse_line("- Item 1"), "<ul><li>Item 1")
|
|
298
|
+
self.assertEqual(self.parser.parse_line("- Item 2"), "</li><li>Item 2")
|
|
299
|
+
self.assertEqual(self.parser.parse_line(""), "</li></ul>")
|
|
300
|
+
|
|
301
|
+
# Reset parser state
|
|
302
|
+
self.parser = MarkdownParser()
|
|
303
|
+
|
|
304
|
+
# Different markers
|
|
305
|
+
self.assertEqual(self.parser.parse_line("* Item 1"), "<ul><li>Item 1")
|
|
306
|
+
self.assertEqual(self.parser.parse_line("+ Item 2"), "</li><li>Item 2")
|
|
307
|
+
|
|
308
|
+
# Reset parser state
|
|
309
|
+
self.parser = MarkdownParser()
|
|
310
|
+
|
|
311
|
+
# Nested list
|
|
312
|
+
self.assertEqual(self.parser.parse_line("- Item 1"), "<ul><li>Item 1")
|
|
313
|
+
self.assertEqual(self.parser.parse_line(" - Nested 1"), "</li><ul><li>Nested 1")
|
|
314
|
+
self.assertEqual(self.parser.parse_line("- Item 2"), "</li></ul></li><li>Item 2")
|
|
315
|
+
|
|
316
|
+
def test_ordered_lists(self):
|
|
317
|
+
# Simple ordered list
|
|
318
|
+
self.assertEqual(self.parser.parse_line("1. Item 1"), "<ol><li>Item 1")
|
|
319
|
+
self.assertEqual(self.parser.parse_line("2. Item 2"), "</li><li>Item 2")
|
|
320
|
+
self.assertEqual(self.parser.parse_line(""), "</li></ol>")
|
|
321
|
+
|
|
322
|
+
# Reset parser state
|
|
323
|
+
self.parser = MarkdownParser()
|
|
324
|
+
|
|
325
|
+
# Nested ordered list
|
|
326
|
+
self.assertEqual(self.parser.parse_line("1. Item 1"), "<ol><li>Item 1")
|
|
327
|
+
self.assertEqual(self.parser.parse_line(" 1. Nested 1"), "</li><ol><li>Nested 1")
|
|
328
|
+
self.assertEqual(self.parser.parse_line("2. Item 2"), "</li></ol></li><li>Item 2")
|
|
329
|
+
|
|
330
|
+
def test_mixed_lists(self):
|
|
331
|
+
# Unordered list with nested ordered list
|
|
332
|
+
self.assertEqual(self.parser.parse_line("- Item 1"), "<ul><li>Item 1")
|
|
333
|
+
self.assertEqual(self.parser.parse_line(" 1. Nested 1"), "</li><ol><li>Nested 1")
|
|
334
|
+
self.assertEqual(self.parser.parse_line("- Item 2"), "</li></ol></li><li>Item 2")
|
|
335
|
+
|
|
336
|
+
def test_complete_parse(self):
|
|
337
|
+
markdown = "# Header\n\n- Item 1\n- Item 2\n\n1. Number 1\n2. Number 2"
|
|
338
|
+
expected = "<h1>Header</h1>\n<ul><li>Item 1</li><li>Item 2</li></ul>\n<ol><li>Number 1</li><li>Number 2</li></ol>"
|
|
339
|
+
self.assertEqual(self.parser.parse(markdown), expected)
|
|
340
|
+
|
|
341
|
+
if __name__ == "__main__":
|
|
342
|
+
unittest.main()
|
|
343
|
+
>>>>>>> REPLACE
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
You can run the example and tests with the following commands:
|
|
347
|
+
|
|
348
|
+
```bash
|
|
349
|
+
python3 example.py
|
|
350
|
+
python3 test_markdown_parser.py
|
|
351
|
+
```
|
|
352
|
+
|
|
353
|
+
> Tokens: 2.3k sent, 2.5k received. Cost: $0.04 message, $0.04 session.
|
|
354
|
+
> markdown_parser.py
|
|
355
|
+
> Create new file? (Y)es/(N)o [Yes]: y
|
|
356
|
+
> example.py
|
|
357
|
+
> Create new file? (Y)es/(N)o [Yes]: y
|
|
358
|
+
> test_markdown_parser.py
|
|
359
|
+
> Create new file? (Y)es/(N)o [Yes]: y
|
|
360
|
+
> Applied edit to example.py
|
|
361
|
+
> Applied edit to test_markdown_parser.py
|
|
362
|
+
> Applied edit to markdown_parser.py
|
|
363
|
+
>
|
|
364
|
+
>
|
|
365
|
+
> ^C again to exit
|
|
366
|
+
|
|
367
|
+
#### so i want it as a tui output, not HTML
|
|
368
|
+
|
|
369
|
+
I'll modify the code to create a TUI (Text User Interface) output instead of HTML. Let's update the files to render markdown as formatted text with proper indentation and styling for the terminal.
|
|
370
|
+
|
|
371
|
+
markdown_parser.py
|
|
372
|
+
```python
|
|
373
|
+
<<<<<<< SEARCH
|
|
374
|
+
"""
|
|
375
|
+
Streaming Markdown Parser
|
|
376
|
+
|
|
377
|
+
A simple streaming parser for Markdown that supports:
|
|
378
|
+
- Headers (h1-h6)
|
|
379
|
+
- Unordered lists (ul)
|
|
380
|
+
- Ordered lists (ol)
|
|
381
|
+
- List items (li)
|
|
382
|
+
"""
|
|
383
|
+
|
|
384
|
+
class MarkdownParser:
|
|
385
|
+
def __init__(self):
|
|
386
|
+
self.in_list = False
|
|
387
|
+
self.list_type = None
|
|
388
|
+
self.list_level = 0
|
|
389
|
+
self.buffer = []
|
|
390
|
+
|
|
391
|
+
def parse_line(self, line):
|
|
392
|
+
"""Parse a single line of markdown and return HTML."""
|
|
393
|
+
line = line.rstrip()
|
|
394
|
+
|
|
395
|
+
# Empty line
|
|
396
|
+
if not line:
|
|
397
|
+
if self.in_list:
|
|
398
|
+
# Close any open list when encountering empty line
|
|
399
|
+
html = self._close_list()
|
|
400
|
+
self.in_list = False
|
|
401
|
+
self.list_type = None
|
|
402
|
+
self.list_level = 0
|
|
403
|
+
return html
|
|
404
|
+
return ""
|
|
405
|
+
|
|
406
|
+
# Headers (h1-h6)
|
|
407
|
+
if line.startswith('#'):
|
|
408
|
+
# Count the number of # at the beginning
|
|
409
|
+
level = 0
|
|
410
|
+
for char in line:
|
|
411
|
+
if char == '#':
|
|
412
|
+
level += 1
|
|
413
|
+
else:
|
|
414
|
+
break
|
|
415
|
+
|
|
416
|
+
if level >= 1 and level <= 6 and (len(line) == level or line[level] == ' '):
|
|
417
|
+
# Close any open list
|
|
418
|
+
html = ""
|
|
419
|
+
if self.in_list:
|
|
420
|
+
html = self._close_list()
|
|
421
|
+
self.in_list = False
|
|
422
|
+
self.list_type = None
|
|
423
|
+
self.list_level = 0
|
|
424
|
+
|
|
425
|
+
# Extract header content (skip the # and any leading space)
|
|
426
|
+
content = line[level:].lstrip()
|
|
427
|
+
return f"{html}<h{level}>{content}</h{level}>"
|
|
428
|
+
|
|
429
|
+
# Unordered list
|
|
430
|
+
if line.lstrip().startswith(('- ', '* ', '+ ')):
|
|
431
|
+
indent = len(line) - len(line.lstrip())
|
|
432
|
+
content = line.lstrip()[2:] # Skip the list marker and space
|
|
433
|
+
|
|
434
|
+
return self._handle_list_item('ul', indent, content)
|
|
435
|
+
|
|
436
|
+
# Ordered list
|
|
437
|
+
if line.lstrip().startswith(tuple(f"{i}. " for i in range(1, 10))):
|
|
438
|
+
indent = len(line) - len(line.lstrip())
|
|
439
|
+
# Skip the digit(s), period, and space
|
|
440
|
+
content = line.lstrip()
|
|
441
|
+
content = content[content.index('. ') + 2:]
|
|
442
|
+
|
|
443
|
+
return self._handle_list_item('ol', indent, content)
|
|
444
|
+
|
|
445
|
+
# Regular paragraph (not implemented in this version)
|
|
446
|
+
# Close any open list
|
|
447
|
+
html = ""
|
|
448
|
+
if self.in_list:
|
|
449
|
+
html = self._close_list()
|
|
450
|
+
self.in_list = False
|
|
451
|
+
self.list_type = None
|
|
452
|
+
self.list_level = 0
|
|
453
|
+
|
|
454
|
+
return html + line
|
|
455
|
+
|
|
456
|
+
def _handle_list_item(self, list_type, indent, content):
|
|
457
|
+
"""Handle a list item, managing nested lists."""
|
|
458
|
+
html = []
|
|
459
|
+
|
|
460
|
+
# Calculate list level based on indentation (4 spaces = 1 level)
|
|
461
|
+
new_level = indent // 4 + 1
|
|
462
|
+
|
|
463
|
+
# If we're not in a list or changing list type
|
|
464
|
+
if not self.in_list:
|
|
465
|
+
html.append(f"<{list_type}>")
|
|
466
|
+
self.in_list = True
|
|
467
|
+
self.list_type = list_type
|
|
468
|
+
self.list_level = 1
|
|
469
|
+
# If we're going deeper in nesting
|
|
470
|
+
elif new_level > self.list_level:
|
|
471
|
+
# Open a new nested list
|
|
472
|
+
html.append(f"<{list_type}>")
|
|
473
|
+
self.list_level = new_level
|
|
474
|
+
self.list_type = list_type
|
|
475
|
+
# If we're going up in the nesting
|
|
476
|
+
elif new_level < self.list_level:
|
|
477
|
+
# Close the appropriate number of lists
|
|
478
|
+
for _ in range(self.list_level - new_level):
|
|
479
|
+
html.append(f"</li></{self.list_type}>")
|
|
480
|
+
self.list_level = new_level
|
|
481
|
+
# Same level but different list type
|
|
482
|
+
elif list_type != self.list_type:
|
|
483
|
+
html.append(f"</li></{self.list_type}>")
|
|
484
|
+
html.append(f"<{list_type}>")
|
|
485
|
+
self.list_type = list_type
|
|
486
|
+
else:
|
|
487
|
+
# Same level, same type - close previous item
|
|
488
|
+
html.append("</li>")
|
|
489
|
+
|
|
490
|
+
# Add the new list item
|
|
491
|
+
html.append(f"<li>{content}")
|
|
492
|
+
|
|
493
|
+
return "".join(html)
|
|
494
|
+
|
|
495
|
+
def _close_list(self):
|
|
496
|
+
"""Close any open list tags."""
|
|
497
|
+
if not self.in_list:
|
|
498
|
+
return ""
|
|
499
|
+
|
|
500
|
+
html = "</li>"
|
|
501
|
+
for _ in range(self.list_level):
|
|
502
|
+
html += f"</{self.list_type}>"
|
|
503
|
+
|
|
504
|
+
return html
|
|
505
|
+
|
|
506
|
+
def parse(self, markdown_text):
|
|
507
|
+
"""Parse a complete markdown text and return HTML."""
|
|
508
|
+
lines = markdown_text.split('\n')
|
|
509
|
+
html_parts = []
|
|
510
|
+
|
|
511
|
+
for line in lines:
|
|
512
|
+
html_parts.append(self.parse_line(line))
|
|
513
|
+
|
|
514
|
+
# Close any open lists at the end
|
|
515
|
+
if self.in_list:
|
|
516
|
+
html_parts.append(self._close_list())
|
|
517
|
+
self.in_list = False
|
|
518
|
+
self.list_type = None
|
|
519
|
+
self.list_level = 0
|
|
520
|
+
|
|
521
|
+
return '\n'.join(filter(None, html_parts))
|
|
522
|
+
|
|
523
|
+
def parse_stream(self, stream):
|
|
524
|
+
"""Parse a stream of markdown lines and yield HTML fragments."""
|
|
525
|
+
for line in stream:
|
|
526
|
+
yield self.parse_line(line)
|
|
527
|
+
|
|
528
|
+
# Close any open lists at the end
|
|
529
|
+
if self.in_list:
|
|
530
|
+
yield self._close_list()
|
|
531
|
+
self.in_list = False
|
|
532
|
+
self.list_type = None
|
|
533
|
+
self.list_level = 0
|
|
534
|
+
=======
|
|
535
|
+
"""
|
|
536
|
+
Streaming Markdown Parser for TUI
|
|
537
|
+
|
|
538
|
+
A simple streaming parser for Markdown that supports:
|
|
539
|
+
- Headers (h1-h6)
|
|
540
|
+
- Unordered lists (ul)
|
|
541
|
+
- Ordered lists (ol)
|
|
542
|
+
- List items (li)
|
|
543
|
+
|
|
544
|
+
Output is formatted for terminal display.
|
|
545
|
+
"""
|
|
546
|
+
|
|
547
|
+
class MarkdownParser:
|
|
548
|
+
def __init__(self):
|
|
549
|
+
self.in_list = False
|
|
550
|
+
self.list_type = None
|
|
551
|
+
self.list_level = 0
|
|
552
|
+
self.buffer = []
|
|
553
|
+
|
|
554
|
+
def parse_line(self, line):
|
|
555
|
+
"""Parse a single line of markdown and return formatted text for terminal."""
|
|
556
|
+
line = line.rstrip()
|
|
557
|
+
|
|
558
|
+
# Empty line
|
|
559
|
+
if not line:
|
|
560
|
+
if self.in_list:
|
|
561
|
+
# Close any open list when encountering empty line
|
|
562
|
+
self.in_list = False
|
|
563
|
+
self.list_type = None
|
|
564
|
+
self.list_level = 0
|
|
565
|
+
return ""
|
|
566
|
+
|
|
567
|
+
# Headers (h1-h6)
|
|
568
|
+
if line.startswith('#'):
|
|
569
|
+
# Count the number of # at the beginning
|
|
570
|
+
level = 0
|
|
571
|
+
for char in line:
|
|
572
|
+
if char == '#':
|
|
573
|
+
level += 1
|
|
574
|
+
else:
|
|
575
|
+
break
|
|
576
|
+
|
|
577
|
+
if level >= 1 and level <= 6 and (len(line) == level or line[level] == ' '):
|
|
578
|
+
# Close any open list
|
|
579
|
+
if self.in_list:
|
|
580
|
+
self.in_list = False
|
|
581
|
+
self.list_type = None
|
|
582
|
+
self.list_level = 0
|
|
583
|
+
|
|
584
|
+
# Extract header content (skip the # and any leading space)
|
|
585
|
+
content = line[level:].lstrip()
|
|
586
|
+
|
|
587
|
+
# Format headers based on level
|
|
588
|
+
if level == 1:
|
|
589
|
+
return f"\n{'=' * (len(content) + 4)}\n {content.upper()} \n{'=' * (len(content) + 4)}"
|
|
590
|
+
elif level == 2:
|
|
591
|
+
return f"\n{content}\n{'-' * len(content)}"
|
|
592
|
+
else:
|
|
593
|
+
# For h3-h6, use indentation and different styles
|
|
594
|
+
indent = " " * (level - 3)
|
|
595
|
+
prefix = "•" if level % 2 == 1 else ">"
|
|
596
|
+
return f"\n{indent}{prefix} {content}"
|
|
597
|
+
|
|
598
|
+
# Unordered list
|
|
599
|
+
if line.lstrip().startswith(('- ', '* ', '+ ')):
|
|
600
|
+
indent = len(line) - len(line.lstrip())
|
|
601
|
+
content = line.lstrip()[2:] # Skip the list marker and space
|
|
602
|
+
|
|
603
|
+
return self._handle_list_item('ul', indent, content)
|
|
604
|
+
|
|
605
|
+
# Ordered list
|
|
606
|
+
if line.lstrip().startswith(tuple(f"{i}. " for i in range(1, 10))):
|
|
607
|
+
indent = len(line) - len(line.lstrip())
|
|
608
|
+
# Skip the digit(s), period, and space
|
|
609
|
+
content = line.lstrip()
|
|
610
|
+
list_num = content.split('. ')[0]
|
|
611
|
+
content = content[content.index('. ') + 2:]
|
|
612
|
+
|
|
613
|
+
return self._handle_list_item('ol', indent, content, list_num)
|
|
614
|
+
|
|
615
|
+
# Regular paragraph (just return the text)
|
|
616
|
+
# Close any open list
|
|
617
|
+
if self.in_list:
|
|
618
|
+
self.in_list = False
|
|
619
|
+
self.list_type = None
|
|
620
|
+
self.list_level = 0
|
|
621
|
+
|
|
622
|
+
return line
|
|
623
|
+
|
|
624
|
+
def _handle_list_item(self, list_type, indent, content, list_num=None):
|
|
625
|
+
"""Handle a list item, managing nested lists."""
|
|
626
|
+
# Calculate list level based on indentation (4 spaces = 1 level)
|
|
627
|
+
new_level = indent // 4 + 1
|
|
628
|
+
|
|
629
|
+
# Set list state
|
|
630
|
+
self.in_list = True
|
|
631
|
+
self.list_type = list_type
|
|
632
|
+
self.list_level = new_level
|
|
633
|
+
|
|
634
|
+
# Format the list item based on type and nesting level
|
|
635
|
+
indent_str = " " * (new_level - 1)
|
|
636
|
+
|
|
637
|
+
if list_type == 'ul':
|
|
638
|
+
bullets = ['•', '◦', '▪', '▫', '◘', '◙'] # Different bullet styles for nesting
|
|
639
|
+
bullet = bullets[(new_level - 1) % len(bullets)]
|
|
640
|
+
return f"{indent_str}{bullet} {content}"
|
|
641
|
+
else: # ordered list
|
|
642
|
+
return f"{indent_str}{list_num}. {content}"
|
|
643
|
+
|
|
644
|
+
def parse(self, markdown_text):
|
|
645
|
+
"""Parse a complete markdown text and return formatted text."""
|
|
646
|
+
lines = markdown_text.split('\n')
|
|
647
|
+
text_parts = []
|
|
648
|
+
|
|
649
|
+
for line in lines:
|
|
650
|
+
parsed_line = self.parse_line(line)
|
|
651
|
+
if parsed_line:
|
|
652
|
+
text_parts.append(parsed_line)
|
|
653
|
+
|
|
654
|
+
return '\n'.join(text_parts)
|
|
655
|
+
|
|
656
|
+
def parse_stream(self, stream):
|
|
657
|
+
"""Parse a stream of markdown lines and yield formatted text fragments."""
|
|
658
|
+
for line in stream:
|
|
659
|
+
yield self.parse_line(line)
|
|
660
|
+
>>>>>>> REPLACE
|
|
661
|
+
```
|
|
662
|
+
|
|
663
|
+
example.py
|
|
664
|
+
```python
|
|
665
|
+
<<<<<<< SEARCH
|
|
666
|
+
#!/usr/bin/env python3
|
|
667
|
+
"""
|
|
668
|
+
Example usage of the Markdown Parser
|
|
669
|
+
"""
|
|
670
|
+
|
|
671
|
+
from markdown_parser import MarkdownParser
|
|
672
|
+
|
|
673
|
+
def main():
|
|
674
|
+
# Example markdown text
|
|
675
|
+
markdown_text = """# Heading 1
|
|
676
|
+
## Heading 2
|
|
677
|
+
### Heading 3
|
|
678
|
+
#### Heading 4
|
|
679
|
+
##### Heading 5
|
|
680
|
+
###### Heading 6
|
|
681
|
+
|
|
682
|
+
- Unordered list item 1
|
|
683
|
+
- Unordered list item 2
|
|
684
|
+
- Nested unordered item 2.1
|
|
685
|
+
- Nested unordered item 2.2
|
|
686
|
+
- Deeply nested item 2.2.1
|
|
687
|
+
- Unordered list item 3
|
|
688
|
+
|
|
689
|
+
1. Ordered list item 1
|
|
690
|
+
2. Ordered list item 2
|
|
691
|
+
1. Nested ordered item 2.1
|
|
692
|
+
2. Nested ordered item 2.2
|
|
693
|
+
3. Ordered list item 3
|
|
694
|
+
|
|
695
|
+
- Mixed list example
|
|
696
|
+
1. Nested ordered in unordered
|
|
697
|
+
2. Another nested item
|
|
698
|
+
- Back to unordered
|
|
699
|
+
"""
|
|
700
|
+
|
|
701
|
+
# Parse the complete text
|
|
702
|
+
parser = MarkdownParser()
|
|
703
|
+
html = parser.parse(markdown_text)
|
|
704
|
+
print("Complete parsing result:")
|
|
705
|
+
print(html)
|
|
706
|
+
print("\n" + "-" * 50 + "\n")
|
|
707
|
+
|
|
708
|
+
# Demonstrate streaming parsing
|
|
709
|
+
print("Streaming parsing result:")
|
|
710
|
+
stream_parser = MarkdownParser()
|
|
711
|
+
for line in markdown_text.split('\n'):
|
|
712
|
+
html_fragment = stream_parser.parse_line(line)
|
|
713
|
+
if html_fragment:
|
|
714
|
+
print(f"Line: {line}")
|
|
715
|
+
print(f"HTML: {html_fragment}")
|
|
716
|
+
print()
|
|
717
|
+
|
|
718
|
+
# Don't forget to close any open tags
|
|
719
|
+
closing_html = stream_parser._close_list() if stream_parser.in_list else ""
|
|
720
|
+
if closing_html:
|
|
721
|
+
print(f"Closing HTML: {closing_html}")
|
|
722
|
+
|
|
723
|
+
if __name__ == "__main__":
|
|
724
|
+
main()
|
|
725
|
+
=======
|
|
726
|
+
#!/usr/bin/env python3
|
|
727
|
+
"""
|
|
728
|
+
Example usage of the Markdown Parser for TUI output
|
|
729
|
+
"""
|
|
730
|
+
|
|
731
|
+
from markdown_parser import MarkdownParser
|
|
732
|
+
import sys
|
|
733
|
+
|
|
734
|
+
def main():
|
|
735
|
+
# Example markdown text
|
|
736
|
+
markdown_text = """# Heading 1
|
|
737
|
+
## Heading 2
|
|
738
|
+
### Heading 3
|
|
739
|
+
#### Heading 4
|
|
740
|
+
##### Heading 5
|
|
741
|
+
###### Heading 6
|
|
742
|
+
|
|
743
|
+
- Unordered list item 1
|
|
744
|
+
- Unordered list item 2
|
|
745
|
+
- Nested unordered item 2.1
|
|
746
|
+
- Nested unordered item 2.2
|
|
747
|
+
- Deeply nested item 2.2.1
|
|
748
|
+
- Unordered list item 3
|
|
749
|
+
|
|
750
|
+
1. Ordered list item 1
|
|
751
|
+
2. Ordered list item 2
|
|
752
|
+
1. Nested ordered item 2.1
|
|
753
|
+
2. Nested ordered item 2.2
|
|
754
|
+
3. Ordered list item 3
|
|
755
|
+
|
|
756
|
+
- Mixed list example
|
|
757
|
+
1. Nested ordered in unordered
|
|
758
|
+
2. Another nested item
|
|
759
|
+
- Back to unordered
|
|
760
|
+
"""
|
|
761
|
+
|
|
762
|
+
# Parse the complete text
|
|
763
|
+
parser = MarkdownParser()
|
|
764
|
+
formatted_text = parser.parse(markdown_text)
|
|
765
|
+
print("Complete parsing result:")
|
|
766
|
+
print(formatted_text)
|
|
767
|
+
print("\n" + "-" * 50 + "\n")
|
|
768
|
+
|
|
769
|
+
# Demonstrate streaming parsing
|
|
770
|
+
print("Streaming parsing result:")
|
|
771
|
+
stream_parser = MarkdownParser()
|
|
772
|
+
for line in markdown_text.split('\n'):
|
|
773
|
+
formatted_line = stream_parser.parse_line(line)
|
|
774
|
+
if formatted_line:
|
|
775
|
+
print(formatted_line)
|
|
776
|
+
|
|
777
|
+
# Check if a file was provided as an argument
|
|
778
|
+
if len(sys.argv) > 1:
|
|
779
|
+
try:
|
|
780
|
+
with open(sys.argv[1], 'r') as file:
|
|
781
|
+
file_parser = MarkdownParser()
|
|
782
|
+
print("\n" + "=" * 50)
|
|
783
|
+
print(f"Parsing file: {sys.argv[1]}")
|
|
784
|
+
print("=" * 50 + "\n")
|
|
785
|
+
print(file_parser.parse(file.read()))
|
|
786
|
+
except FileNotFoundError:
|
|
787
|
+
print(f"Error: File '{sys.argv[1]}' not found.")
|
|
788
|
+
except Exception as e:
|
|
789
|
+
print(f"Error parsing file: {e}")
|
|
790
|
+
|
|
791
|
+
if __name__ == "__main__":
|
|
792
|
+
main()
|
|
793
|
+
>>>>>>> REPLACE
|
|
794
|
+
```
|
|
795
|
+
|
|
796
|
+
test_markdown_parser.py
|
|
797
|
+
```python
|
|
798
|
+
<<<<<<< SEARCH
|
|
799
|
+
#!/usr/bin/env python3
|
|
800
|
+
"""
|
|
801
|
+
Tests for the Markdown Parser
|
|
802
|
+
"""
|
|
803
|
+
|
|
804
|
+
import unittest
|
|
805
|
+
from markdown_parser import MarkdownParser
|
|
806
|
+
|
|
807
|
+
class TestMarkdownParser(unittest.TestCase):
|
|
808
|
+
def setUp(self):
|
|
809
|
+
self.parser = MarkdownParser()
|
|
810
|
+
|
|
811
|
+
def test_headers(self):
|
|
812
|
+
self.assertEqual(self.parser.parse_line("# Header 1"), "<h1>Header 1</h1>")
|
|
813
|
+
self.assertEqual(self.parser.parse_line("## Header 2"), "<h2>Header 2</h2>")
|
|
814
|
+
self.assertEqual(self.parser.parse_line("### Header 3"), "<h3>Header 3</h3>")
|
|
815
|
+
self.assertEqual(self.parser.parse_line("#### Header 4"), "<h4>Header 4</h4>")
|
|
816
|
+
self.assertEqual(self.parser.parse_line("##### Header 5"), "<h5>Header 5</h5>")
|
|
817
|
+
self.assertEqual(self.parser.parse_line("###### Header 6"), "<h6>Header 6</h6>")
|
|
818
|
+
|
|
819
|
+
# Invalid headers
|
|
820
|
+
self.assertEqual(self.parser.parse_line("####### Too many"), "####### Too many")
|
|
821
|
+
self.assertEqual(self.parser.parse_line("#No space"), "#No space")
|
|
822
|
+
|
|
823
|
+
def test_unordered_lists(self):
|
|
824
|
+
# Simple list
|
|
825
|
+
self.assertEqual(self.parser.parse_line("- Item 1"), "<ul><li>Item 1")
|
|
826
|
+
self.assertEqual(self.parser.parse_line("- Item 2"), "</li><li>Item 2")
|
|
827
|
+
self.assertEqual(self.parser.parse_line(""), "</li></ul>")
|
|
828
|
+
|
|
829
|
+
# Reset parser state
|
|
830
|
+
self.parser = MarkdownParser()
|
|
831
|
+
|
|
832
|
+
# Different markers
|
|
833
|
+
self.assertEqual(self.parser.parse_line("* Item 1"), "<ul><li>Item 1")
|
|
834
|
+
self.assertEqual(self.parser.parse_line("+ Item 2"), "</li><li>Item 2")
|
|
835
|
+
|
|
836
|
+
# Reset parser state
|
|
837
|
+
self.parser = MarkdownParser()
|
|
838
|
+
|
|
839
|
+
# Nested list
|
|
840
|
+
self.assertEqual(self.parser.parse_line("- Item 1"), "<ul><li>Item 1")
|
|
841
|
+
self.assertEqual(self.parser.parse_line(" - Nested 1"), "</li><ul><li>Nested 1")
|
|
842
|
+
self.assertEqual(self.parser.parse_line("- Item 2"), "</li></ul></li><li>Item 2")
|
|
843
|
+
|
|
844
|
+
def test_ordered_lists(self):
|
|
845
|
+
# Simple ordered list
|
|
846
|
+
self.assertEqual(self.parser.parse_line("1. Item 1"), "<ol><li>Item 1")
|
|
847
|
+
self.assertEqual(self.parser.parse_line("2. Item 2"), "</li><li>Item 2")
|
|
848
|
+
self.assertEqual(self.parser.parse_line(""), "</li></ol>")
|
|
849
|
+
|
|
850
|
+
# Reset parser state
|
|
851
|
+
self.parser = MarkdownParser()
|
|
852
|
+
|
|
853
|
+
# Nested ordered list
|
|
854
|
+
self.assertEqual(self.parser.parse_line("1. Item 1"), "<ol><li>Item 1")
|
|
855
|
+
self.assertEqual(self.parser.parse_line(" 1. Nested 1"), "</li><ol><li>Nested 1")
|
|
856
|
+
self.assertEqual(self.parser.parse_line("2. Item 2"), "</li></ol></li><li>Item 2")
|
|
857
|
+
|
|
858
|
+
def test_mixed_lists(self):
|
|
859
|
+
# Unordered list with nested ordered list
|
|
860
|
+
self.assertEqual(self.parser.parse_line("- Item 1"), "<ul><li>Item 1")
|
|
861
|
+
self.assertEqual(self.parser.parse_line(" 1. Nested 1"), "</li><ol><li>Nested 1")
|
|
862
|
+
self.assertEqual(self.parser.parse_line("- Item 2"), "</li></ol></li><li>Item 2")
|
|
863
|
+
|
|
864
|
+
def test_complete_parse(self):
|
|
865
|
+
markdown = "# Header\n\n- Item 1\n- Item 2\n\n1. Number 1\n2. Number 2"
|
|
866
|
+
expected = "<h1>Header</h1>\n<ul><li>Item 1</li><li>Item 2</li></ul>\n<ol><li>Number 1</li><li>Number 2</li></ol>"
|
|
867
|
+
self.assertEqual(self.parser.parse(markdown), expected)
|
|
868
|
+
|
|
869
|
+
if __name__ == "__main__":
|
|
870
|
+
unittest.main()
|
|
871
|
+
=======
|
|
872
|
+
#!/usr/bin/env python3
|
|
873
|
+
"""
|
|
874
|
+
Tests for the Markdown Parser TUI output
|
|
875
|
+
"""
|
|
876
|
+
|
|
877
|
+
import unittest
|
|
878
|
+
from markdown_parser import MarkdownParser
|
|
879
|
+
|
|
880
|
+
class TestMarkdownParser(unittest.TestCase):
|
|
881
|
+
def setUp(self):
|
|
882
|
+
self.parser = MarkdownParser()
|
|
883
|
+
|
|
884
|
+
def test_headers(self):
|
|
885
|
+
h1_expected = "\n=============\n HEADER 1 \n============="
|
|
886
|
+
self.assertEqual(self.parser.parse_line("# Header 1"), h1_expected)
|
|
887
|
+
|
|
888
|
+
h2_expected = "\nHeader 2\n--------"
|
|
889
|
+
self.assertEqual(self.parser.parse_line("## Header 2"), h2_expected)
|
|
890
|
+
|
|
891
|
+
h3_expected = "\n• Header 3"
|
|
892
|
+
self.assertEqual(self.parser.parse_line("### Header 3"), h3_expected)
|
|
893
|
+
|
|
894
|
+
h4_expected = "\n > Header 4"
|
|
895
|
+
self.assertEqual(self.parser.parse_line("#### Header 4"), h4_expected)
|
|
896
|
+
|
|
897
|
+
h5_expected = "\n • Header 5"
|
|
898
|
+
self.assertEqual(self.parser.parse_line("##### Header 5"), h5_expected)
|
|
899
|
+
|
|
900
|
+
h6_expected = "\n > Header 6"
|
|
901
|
+
self.assertEqual(self.parser.parse_line("###### Header 6"), h6_expected)
|
|
902
|
+
|
|
903
|
+
# Invalid headers
|
|
904
|
+
self.assertEqual(self.parser.parse_line("####### Too many"), "####### Too many")
|
|
905
|
+
self.assertEqual(self.parser.parse_line("#No space"), "#No space")
|
|
906
|
+
|
|
907
|
+
def test_unordered_lists(self):
|
|
908
|
+
# Simple list
|
|
909
|
+
self.assertEqual(self.parser.parse_line("- Item 1"), "• Item 1")
|
|
910
|
+
self.assertEqual(self.parser.parse_line("- Item 2"), "• Item 2")
|
|
911
|
+
self.assertEqual(self.parser.parse_line(""), "")
|
|
912
|
+
|
|
913
|
+
# Reset parser state
|
|
914
|
+
self.parser = MarkdownParser()
|
|
915
|
+
|
|
916
|
+
# Different markers
|
|
917
|
+
self.assertEqual(self.parser.parse_line("* Item 1"), "• Item 1")
|
|
918
|
+
self.assertEqual(self.parser.parse_line("+ Item 2"), "• Item 2")
|
|
919
|
+
|
|
920
|
+
# Reset parser state
|
|
921
|
+
self.parser = MarkdownParser()
|
|
922
|
+
|
|
923
|
+
# Nested list
|
|
924
|
+
self.assertEqual(self.parser.parse_line("- Item 1"), "• Item 1")
|
|
925
|
+
self.assertEqual(self.parser.parse_line(" - Nested 1"), " ◦ Nested 1")
|
|
926
|
+
self.assertEqual(self.parser.parse_line("- Item 2"), "• Item 2")
|
|
927
|
+
|
|
928
|
+
def test_ordered_lists(self):
|
|
929
|
+
# Simple ordered list
|
|
930
|
+
self.assertEqual(self.parser.parse_line("1. Item 1"), "1. Item 1")
|
|
931
|
+
self.assertEqual(self.parser.parse_line("2. Item 2"), "2. Item 2")
|
|
932
|
+
self.assertEqual(self.parser.parse_line(""), "")
|
|
933
|
+
|
|
934
|
+
# Reset parser state
|
|
935
|
+
self.parser = MarkdownParser()
|
|
936
|
+
|
|
937
|
+
# Nested ordered list
|
|
938
|
+
self.assertEqual(self.parser.parse_line("1. Item 1"), "1. Item 1")
|
|
939
|
+
self.assertEqual(self.parser.parse_line(" 1. Nested 1"), " 1. Nested 1")
|
|
940
|
+
self.assertEqual(self.parser.parse_line("2. Item 2"), "2. Item 2")
|
|
941
|
+
|
|
942
|
+
def test_mixed_lists(self):
|
|
943
|
+
# Unordered list with nested ordered list
|
|
944
|
+
self.assertEqual(self.parser.parse_line("- Item 1"), "• Item 1")
|
|
945
|
+
self.assertEqual(self.parser.parse_line(" 1. Nested 1"), " 1. Nested 1")
|
|
946
|
+
self.assertEqual(self.parser.parse_line("- Item 2"), "• Item 2")
|
|
947
|
+
|
|
948
|
+
def test_complete_parse(self):
|
|
949
|
+
markdown = "# Header\n\n- Item 1\n- Item 2\n\n1. Number 1\n2. Number 2"
|
|
950
|
+
expected = "\n===========\n HEADER \n===========\n• Item 1\n• Item 2\n1. Number 1\n2. Number 2"
|
|
951
|
+
self.assertEqual(self.parser.parse(markdown), expected)
|
|
952
|
+
|
|
953
|
+
if __name__ == "__main__":
|
|
954
|
+
unittest.main()
|
|
955
|
+
>>>>>>> REPLACE
|
|
956
|
+
```
|
|
957
|
+
|
|
958
|
+
Now you can run the example and tests with the following commands:
|
|
959
|
+
|
|
960
|
+
```bash
|
|
961
|
+
python3 example.py
|
|
962
|
+
python3 test_markdown_parser.py
|
|
963
|
+
```
|
|
964
|
+
|
|
965
|
+
You can also parse a markdown file directly by providing its path as an argument:
|
|
966
|
+
|
|
967
|
+
```bash
|
|
968
|
+
python3 example.py your_markdown_file.md
|
|
969
|
+
```
|
|
970
|
+
|
|
971
|
+
> Tokens: 7.6k sent, 4.7k received. Cost: $0.09 message, $0.14 session.
|
|
972
|
+
> Committing markdown_parser.py before applying edits.
|
|
973
|
+
> Committing example.py before applying edits.
|
|
974
|
+
> Committing test_markdown_parser.py before applying edits.
|
|
975
|
+
>
|
|
976
|
+
>
|
|
977
|
+
> ^C again to exit
|
|
978
|
+
|
|
979
|
+
# aider chat started at 2025-03-11 17:35:49
|
|
980
|
+
|
|
981
|
+
> /home/chris/.local/bin/aider
|
|
982
|
+
> Found OPENROUTER_API_KEY so using openrouter/anthropic/claude-3.7-sonnet since no --model was specified.
|
|
983
|
+
> Aider v0.76.2
|
|
984
|
+
> Main model: openrouter/anthropic/claude-3.7-sonnet with diff edit format, infinite output
|
|
985
|
+
> Weak model: openrouter/anthropic/claude-3-5-haiku
|
|
986
|
+
> Git repo: .git with 3 files
|
|
987
|
+
> Repo-map: using 4096 tokens, auto refresh
|
|
988
|
+
>
|
|
989
|
+
>
|
|
990
|
+
> ^C again to exit
|
|
991
|
+
>
|
|
992
|
+
>
|
|
993
|
+
> ^C KeyboardInterrupt
|
|
994
|
+
|
|
995
|
+
# aider chat started at 2025-03-11 17:36:10
|
|
996
|
+
|
|
997
|
+
> /home/chris/.local/bin/aider --model openrouter/google/gemini-2.0-flash-exp:free
|
|
998
|
+
> Warning for openrouter/google/gemini-2.0-flash-exp:free: Unknown context window size and costs, using sane defaults.
|
|
999
|
+
> Did you mean one of these?
|
|
1000
|
+
> - openrouter/google/gemini-2.0-flash-001
|
|
1001
|
+
> You can skip this check with --no-show-model-warnings
|
|
1002
|
+
> https://aider.chat/docs/llms/warnings.html
|
|
1003
|
+
> Open documentation url for more info? (Y)es/(N)o/(D)on't ask again [Yes]: d
|
|
1004
|
+
> Aider v0.76.2
|
|
1005
|
+
> Model: openrouter/google/gemini-2.0-flash-exp:free with whole edit format
|
|
1006
|
+
> Git repo: .git with 3 files
|
|
1007
|
+
> Repo-map: using 1024 tokens, auto refresh
|
|
1008
|
+
|
|
1009
|
+
#### /models
|
|
1010
|
+
> Please provide a partial model name to search for.
|
|
1011
|
+
|
|
1012
|
+
#### /models free
|
|
1013
|
+
> Models which match "free":
|
|
1014
|
+
> - openrouter/meta-llama/llama-3-8b-instruct:free
|
|
1015
|
+
> - openrouter/mistralai/mistral-7b-instruct:free
|
|
1016
|
+
|
|
1017
|
+
#### models gemini
|
|
1018
|
+
|
|
1019
|
+
Okay, I understand the format you want me to use.
|
|
1020
|
+
|
|
1021
|
+
Please provide the request you would like me to fulfill. I will then tell you which files I think need to be edited.
|
|
1022
|
+
|
|
1023
|
+
> Tokens: 990 sent, 37 received.
|
|
1024
|
+
>
|
|
1025
|
+
>
|
|
1026
|
+
> ^C again to exit
|
|
1027
|
+
|
|
1028
|
+
#### /models gemini
|
|
1029
|
+
> Models which match "gemini":
|
|
1030
|
+
> - gemini-1.0-pro
|
|
1031
|
+
> - gemini-1.0-pro-001
|
|
1032
|
+
> - gemini-1.0-pro-002
|
|
1033
|
+
> - gemini-1.0-pro-vision
|
|
1034
|
+
> - gemini-1.0-pro-vision-001
|
|
1035
|
+
> - gemini-1.0-ultra
|
|
1036
|
+
> - gemini-1.0-ultra-001
|
|
1037
|
+
> - gemini-1.5-flash
|
|
1038
|
+
> - gemini-1.5-flash-001
|
|
1039
|
+
> - gemini-1.5-flash-002
|
|
1040
|
+
> - gemini-1.5-flash-exp-0827
|
|
1041
|
+
> - gemini-1.5-flash-preview-0514
|
|
1042
|
+
> - gemini-1.5-pro
|
|
1043
|
+
> - gemini-1.5-pro-001
|
|
1044
|
+
> - gemini-1.5-pro-002
|
|
1045
|
+
> - gemini-1.5-pro-preview-0215
|
|
1046
|
+
> - gemini-1.5-pro-preview-0409
|
|
1047
|
+
> - gemini-1.5-pro-preview-0514
|
|
1048
|
+
> - gemini-2.0-flash-001
|
|
1049
|
+
> - gemini-2.0-flash-exp
|
|
1050
|
+
> - gemini-2.0-flash-thinking-exp
|
|
1051
|
+
> - gemini-2.0-flash-thinking-exp-01-21
|
|
1052
|
+
> - gemini-2.0-pro-exp-02-05
|
|
1053
|
+
> - gemini-flash-experimental
|
|
1054
|
+
> - gemini-pro
|
|
1055
|
+
> - gemini-pro-experimental
|
|
1056
|
+
> - gemini-pro-vision
|
|
1057
|
+
> - gemini/gemini-1.5-flash
|
|
1058
|
+
> - gemini/gemini-1.5-flash-001
|
|
1059
|
+
> - gemini/gemini-1.5-flash-002
|
|
1060
|
+
> - gemini/gemini-1.5-flash-8b
|
|
1061
|
+
> - gemini/gemini-1.5-flash-8b-exp-0827
|
|
1062
|
+
> - gemini/gemini-1.5-flash-8b-exp-0924
|
|
1063
|
+
> - gemini/gemini-1.5-flash-exp-0827
|
|
1064
|
+
> - gemini/gemini-1.5-flash-latest
|
|
1065
|
+
> - gemini/gemini-1.5-pro
|
|
1066
|
+
> - gemini/gemini-1.5-pro-001
|
|
1067
|
+
> - gemini/gemini-1.5-pro-002
|
|
1068
|
+
> - gemini/gemini-1.5-pro-exp-0801
|
|
1069
|
+
> - gemini/gemini-1.5-pro-exp-0827
|
|
1070
|
+
> - gemini/gemini-1.5-pro-latest
|
|
1071
|
+
> - gemini/gemini-2.0-flash
|
|
1072
|
+
> - gemini/gemini-2.0-flash-001
|
|
1073
|
+
> - gemini/gemini-2.0-flash-exp
|
|
1074
|
+
> - gemini/gemini-2.0-flash-lite-preview-02-05
|
|
1075
|
+
> - gemini/gemini-2.0-flash-thinking-exp
|
|
1076
|
+
> - gemini/gemini-2.0-pro-exp-02-05
|
|
1077
|
+
> - gemini/gemini-exp-1114
|
|
1078
|
+
> - gemini/gemini-exp-1206
|
|
1079
|
+
> - gemini/gemini-gemma-2-27b-it
|
|
1080
|
+
> - gemini/gemini-gemma-2-9b-it
|
|
1081
|
+
> - gemini/gemini-pro
|
|
1082
|
+
> - gemini/gemini-pro-vision
|
|
1083
|
+
> - openrouter/google/gemini-2.0-flash-001
|
|
1084
|
+
> - openrouter/google/gemini-pro-1.5
|
|
1085
|
+
> - openrouter/google/gemini-pro-vision
|
|
1086
|
+
> - vertex_ai-language-models/gemini-1.0-pro
|
|
1087
|
+
> - vertex_ai-language-models/gemini-1.0-pro-001
|
|
1088
|
+
> - vertex_ai-language-models/gemini-1.0-pro-002
|
|
1089
|
+
> - vertex_ai-language-models/gemini-1.0-ultra
|
|
1090
|
+
> - vertex_ai-language-models/gemini-1.0-ultra-001
|
|
1091
|
+
> - vertex_ai-language-models/gemini-1.5-flash
|
|
1092
|
+
> - vertex_ai-language-models/gemini-1.5-flash-001
|
|
1093
|
+
> - vertex_ai-language-models/gemini-1.5-flash-002
|
|
1094
|
+
> - vertex_ai-language-models/gemini-1.5-flash-exp-0827
|
|
1095
|
+
> - vertex_ai-language-models/gemini-1.5-flash-preview-0514
|
|
1096
|
+
> - vertex_ai-language-models/gemini-1.5-pro
|
|
1097
|
+
> - vertex_ai-language-models/gemini-1.5-pro-001
|
|
1098
|
+
> - vertex_ai-language-models/gemini-1.5-pro-002
|
|
1099
|
+
> - vertex_ai-language-models/gemini-1.5-pro-preview-0215
|
|
1100
|
+
> - vertex_ai-language-models/gemini-1.5-pro-preview-0409
|
|
1101
|
+
> - vertex_ai-language-models/gemini-1.5-pro-preview-0514
|
|
1102
|
+
> - vertex_ai-language-models/gemini-2.0-flash-001
|
|
1103
|
+
> - vertex_ai-language-models/gemini-2.0-flash-exp
|
|
1104
|
+
> - vertex_ai-language-models/gemini-2.0-flash-thinking-exp
|
|
1105
|
+
> - vertex_ai-language-models/gemini-2.0-flash-thinking-exp-01-21
|
|
1106
|
+
> - vertex_ai-language-models/gemini-2.0-pro-exp-02-05
|
|
1107
|
+
> - vertex_ai-language-models/gemini-flash-experimental
|
|
1108
|
+
> - vertex_ai-language-models/gemini-pro
|
|
1109
|
+
> - vertex_ai-language-models/gemini-pro-experimental
|
|
1110
|
+
> - vertex_ai-vision-models/gemini-1.0-pro-vision
|
|
1111
|
+
> - vertex_ai-vision-models/gemini-1.0-pro-vision-001
|
|
1112
|
+
> - vertex_ai-vision-models/gemini-pro-vision
|
|
1113
|
+
|
|
1114
|
+
#### /modesl free
|
|
1115
|
+
> Invalid command: /modesl
|
|
1116
|
+
|
|
1117
|
+
#### /models :free
|
|
1118
|
+
> Models which match ":free":
|
|
1119
|
+
> - openrouter/meta-llama/llama-3-8b-instruct:free
|
|
1120
|
+
> - openrouter/mistralai/mistral-7b-instruct:free
|
|
1121
|
+
|
|
1122
|
+
#### /model openrouter/google/gemini-2.0-pro-exp-02-05:free
|
|
1123
|
+
> Warning for openrouter/google/gemini-2.0-pro-exp-02-05:free: Unknown context window size and costs, using sane defaults.
|
|
1124
|
+
> Warning for openrouter/google/gemini-2.0-flash-exp:free: Unknown context window size and costs, using sane defaults.
|
|
1125
|
+
> Did you mean one of these?
|
|
1126
|
+
> - openrouter/google/gemini-2.0-flash-001
|
|
1127
|
+
> Aider v0.76.2
|
|
1128
|
+
> Main model: openrouter/google/gemini-2.0-pro-exp-02-05:free with whole edit format
|
|
1129
|
+
> Weak model: openrouter/google/gemini-2.0-flash-exp:free
|
|
1130
|
+
> Git repo: .git with 3 files
|
|
1131
|
+
> Repo-map: using 1024 tokens, auto refresh
|
|
1132
|
+
>
|
|
1133
|
+
>
|
|
1134
|
+
> ^C again to exit
|
|
1135
|
+
>
|
|
1136
|
+
>
|
|
1137
|
+
> ^C KeyboardInterrupt
|
|
1138
|
+
|
|
1139
|
+
# aider chat started at 2025-03-11 17:41:04
|
|
1140
|
+
|
|
1141
|
+
> /home/chris/.local/bin/aider --model openrouter/google/gemini-2.0-flash-exp:free --api-key openrouter=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
|
|
1142
|
+
> Warning for openrouter/google/gemini-2.0-flash-exp:free: Unknown context window size and costs, using sane defaults.
|
|
1143
|
+
> Did you mean one of these?
|
|
1144
|
+
> - openrouter/google/gemini-2.0-flash-001
|
|
1145
|
+
> You can skip this check with --no-show-model-warnings
|
|
1146
|
+
> https://aider.chat/docs/llms/warnings.html
|
|
1147
|
+
> Open documentation url for more info? (Y)es/(N)o/(D)on't ask again [Yes]: d
|
|
1148
|
+
> Aider v0.76.2
|
|
1149
|
+
> Model: openrouter/google/gemini-2.0-flash-exp:free with whole edit format
|
|
1150
|
+
> Git repo: .git with 3 files
|
|
1151
|
+
> Repo-map: using 1024 tokens, auto refresh
|
|
1152
|
+
|
|
1153
|
+
#### /models free
|
|
1154
|
+
> Models which match "free":
|
|
1155
|
+
> - openrouter/meta-llama/llama-3-8b-instruct:free
|
|
1156
|
+
> - openrouter/mistralai/mistral-7b-instruct:free
|
|
1157
|
+
|
|
1158
|
+
#### /model openrouter/google/gemini-2.0-pro-exp-02-05:free
|
|
1159
|
+
> Warning for openrouter/google/gemini-2.0-pro-exp-02-05:free: Unknown context window size and costs, using sane defaults.
|
|
1160
|
+
> Warning for openrouter/google/gemini-2.0-flash-exp:free: Unknown context window size and costs, using sane defaults.
|
|
1161
|
+
> Did you mean one of these?
|
|
1162
|
+
> - openrouter/google/gemini-2.0-flash-001
|
|
1163
|
+
> Aider v0.76.2
|
|
1164
|
+
> Main model: openrouter/google/gemini-2.0-pro-exp-02-05:free with whole edit format
|
|
1165
|
+
> Weak model: openrouter/google/gemini-2.0-flash-exp:free
|
|
1166
|
+
> Git repo: .git with 3 files
|
|
1167
|
+
> Repo-map: using 1024 tokens, auto refresh
|
|
1168
|
+
|
|
1169
|
+
#### so i want it as a tui output, not HTML
|
|
1170
|
+
|
|
1171
|
+
Okay, I understand. You want to change the output from HTML to a TUI (Text User Interface) output.
|
|
1172
|
+
|
|
1173
|
+
Based on the file summaries you provided earlier, and this new requirement, the files most likely to need changes are:
|
|
1174
|
+
|
|
1175
|
+
* **`markdown_parser.py`**: The core logic for parsing the Markdown will likely remain the same, but the output generation will need to change significantly. Instead of generating HTML tags, it will need to produce TUI-compatible output, possibly using a library like `rich` or `textual`.
|
|
1176
|
+
|
|
1177
|
+
I'll stop here and wait for you to add these files to the chat so I can make specific suggestions.
|
|
1178
|
+
|
|
1179
|
+
> Tokens: 999 sent, 131 received.
|
|
1180
|
+
|
|
1181
|
+
#### so rich doesn't cut it because it can't do streaming in a reasonable way. It shows an elipses and it's terrible. textual looks wrong too. I just want streaming output.
|
|
1182
|
+
|
|
1183
|
+
Okay, I understand. You want plain, streaming text output, and you've ruled out `rich` and `textual`. I also understand the formatting requirements for file changes.
|
|
1184
|
+
|
|
1185
|
+
Given this, the file that needs to change is still:
|
|
1186
|
+
|
|
1187
|
+
* **`markdown_parser.py`**: This file will need to be modified to output plain text instead of HTML. The parsing logic will stay mostly the same, but the output formatting will change.
|
|
1188
|
+
|
|
1189
|
+
Here's the updated `markdown_parser.py` file:
|
|
1190
|
+
|
|
1191
|
+
markdown_parser.py
|
|
1192
|
+
```
|
|
1193
|
+
class MarkdownParser:
|
|
1194
|
+
def __init__(self):
|
|
1195
|
+
self.in_list = False
|
|
1196
|
+
self.list_type = None
|
|
1197
|
+
self.list_level = 0
|
|
1198
|
+
|
|
1199
|
+
def parse_line(self, line):
|
|
1200
|
+
output = ""
|
|
1201
|
+
stripped_line = line.lstrip()
|
|
1202
|
+
indent = len(line) - len(stripped_line)
|
|
1203
|
+
|
|
1204
|
+
if stripped_line.startswith("#"):
|
|
1205
|
+
level = 0
|
|
1206
|
+
for char in stripped_line:
|
|
1207
|
+
if char == "#":
|
|
1208
|
+
level += 1
|
|
1209
|
+
else:
|
|
1210
|
+
break
|
|
1211
|
+
output += f"Header {level}: {stripped_line[level:].strip()}\n"
|
|
1212
|
+
self._close_list()
|
|
1213
|
+
elif stripped_line.startswith("* ") or stripped_line.startswith("- "):
|
|
1214
|
+
output += self._handle_list_item("unordered", indent, stripped_line[2:])
|
|
1215
|
+
elif stripped_line.startswith("1. "): # Simplified ordered list check
|
|
1216
|
+
output += self._handle_list_item("ordered", indent, stripped_line[3:])
|
|
1217
|
+
else:
|
|
1218
|
+
if self.in_list:
|
|
1219
|
+
self._close_list()
|
|
1220
|
+
output += f"{stripped_line}\n"
|
|
1221
|
+
|
|
1222
|
+
return output
|
|
1223
|
+
|
|
1224
|
+
def _handle_list_item(self, list_type, indent, content):
|
|
1225
|
+
output = ""
|
|
1226
|
+
level = indent // 4 # Assume 4 spaces per indent level
|
|
1227
|
+
|
|
1228
|
+
if not self.in_list:
|
|
1229
|
+
self.in_list = True
|
|
1230
|
+
self.list_type = list_type
|
|
1231
|
+
self.list_level = level
|
|
1232
|
+
elif self.list_type != list_type or self.list_level != level:
|
|
1233
|
+
# type or level changed.
|
|
1234
|
+
output += self._close_list()
|
|
1235
|
+
self.in_list = True
|
|
1236
|
+
self.list_type = list_type
|
|
1237
|
+
self.list_level = level
|
|
1238
|
+
|
|
1239
|
+
if list_type == "unordered":
|
|
1240
|
+
output += f"{' ' * level}* {content.strip()}\n"
|
|
1241
|
+
elif list_type == "ordered":
|
|
1242
|
+
output += f"{' ' * level}1. {content.strip()}\n" # Simplified numbering
|
|
1243
|
+
return output;
|
|
1244
|
+
|
|
1245
|
+
def _close_list(self):
|
|
1246
|
+
output = ""
|
|
1247
|
+
if (self.in_list):
|
|
1248
|
+
output = "" # No newlines needed.
|
|
1249
|
+
self.in_list = False
|
|
1250
|
+
self.list_type = None
|
|
1251
|
+
self.list_level = 0
|
|
1252
|
+
return output
|
|
1253
|
+
|
|
1254
|
+
def parse(self, markdown_text):
|
|
1255
|
+
output = ""
|
|
1256
|
+
for line in markdown_text.splitlines():
|
|
1257
|
+
output += self.parse_line(line)
|
|
1258
|
+
output += self._close_list() # close any open lists
|
|
1259
|
+
return output
|
|
1260
|
+
|
|
1261
|
+
def parse_stream(self, stream):
|
|
1262
|
+
output = ""
|
|
1263
|
+
for line in stream:
|
|
1264
|
+
output += self.parse_line(line)
|
|
1265
|
+
output += self._close_list() # close any open lists at the end of stream
|
|
1266
|
+
return output
|
|
1267
|
+
```
|
|
1268
|
+
|
|
1269
|
+
> Tokens: 1.2k sent, 667 received.
|
|
1270
|
+
> markdown_parser.py
|
|
1271
|
+
> Add file to the chat? (Y)es/(N)o/(D)on't ask again [Yes]: y
|
|
1272
|
+
|
|
1273
|
+
You've already added the `markdown_parser.py` file, and I've provided the updated version for TUI output in my previous response. There are no other files that need changes based on your current requirements. If you have further modifications or new features, then we might need to add or modify other files.
|
|
1274
|
+
|
|
1275
|
+
> Tokens: 2.8k sent, 62 received.
|
|
1276
|
+
|
|
1277
|
+
#### example.py still outputs html
|
|
1278
|
+
> example.py
|
|
1279
|
+
> Add file to the chat? (Y)es/(N)o/(D)on't ask again [Yes]: y
|
|
1280
|
+
|
|
1281
|
+
You are absolutely correct! I apologize for the oversight. The `example.py` file needs to be updated to reflect the changes in `markdown_parser.py`, specifically that it now outputs plain text instead of HTML.
|
|
1282
|
+
|
|
1283
|
+
Here's the corrected `example.py`:
|
|
1284
|
+
|
|
1285
|
+
example.py
|
|
1286
|
+
```
|
|
1287
|
+
#!/usr/bin/env python3
|
|
1288
|
+
"""
|
|
1289
|
+
Example usage of the Markdown Parser
|
|
1290
|
+
"""
|
|
1291
|
+
|
|
1292
|
+
from markdown_parser import MarkdownParser
|
|
1293
|
+
|
|
1294
|
+
def main():
|
|
1295
|
+
# Example markdown text
|
|
1296
|
+
markdown_text = """# Heading 1
|
|
1297
|
+
## Heading 2
|
|
1298
|
+
### Heading 3
|
|
1299
|
+
#### Heading 4
|
|
1300
|
+
##### Heading 5
|
|
1301
|
+
###### Heading 6
|
|
1302
|
+
|
|
1303
|
+
- Unordered list item 1
|
|
1304
|
+
- Unordered list item 2
|
|
1305
|
+
- Nested unordered item 2.1
|
|
1306
|
+
- Nested unordered item 2.2
|
|
1307
|
+
- Deeply nested item 2.2.1
|
|
1308
|
+
- Unordered list item 3
|
|
1309
|
+
|
|
1310
|
+
1. Ordered list item 1
|
|
1311
|
+
2. Ordered list item 2
|
|
1312
|
+
1. Nested ordered item 2.1
|
|
1313
|
+
2. Nested ordered item 2.2
|
|
1314
|
+
3. Ordered list item 3
|
|
1315
|
+
|
|
1316
|
+
- Mixed list example
|
|
1317
|
+
1. Nested ordered in unordered
|
|
1318
|
+
2. Another nested item
|
|
1319
|
+
- Back to unordered
|
|
1320
|
+
"""
|
|
1321
|
+
|
|
1322
|
+
# Parse the complete text
|
|
1323
|
+
parser = MarkdownParser()
|
|
1324
|
+
text = parser.parse(markdown_text)
|
|
1325
|
+
print("Complete parsing result:")
|
|
1326
|
+
print(text)
|
|
1327
|
+
print("\n" + "-" * 50 + "\n")
|
|
1328
|
+
|
|
1329
|
+
# Demonstrate streaming parsing
|
|
1330
|
+
print("Streaming parsing result:")
|
|
1331
|
+
stream_parser = MarkdownParser()
|
|
1332
|
+
for line in markdown_text.split('\n'):
|
|
1333
|
+
text_fragment = stream_parser.parse_line(line)
|
|
1334
|
+
if text_fragment:
|
|
1335
|
+
print(text_fragment, end="") # Print without extra newline
|
|
1336
|
+
|
|
1337
|
+
# Don't forget to close any open tags
|
|
1338
|
+
closing_text = stream_parser._close_list()
|
|
1339
|
+
if closing_text:
|
|
1340
|
+
print(closing_text, end="")
|
|
1341
|
+
|
|
1342
|
+
if __name__ == "__main__":
|
|
1343
|
+
main()
|
|
1344
|
+
```
|
|
1345
|
+
|
|
1346
|
+
Key changes in `example.py`:
|
|
1347
|
+
|
|
1348
|
+
* Changed `html` variable names to `text`.
|
|
1349
|
+
* Removed the `print(f"Line: {line}")` line in the streaming section, as it's redundant with the direct output.
|
|
1350
|
+
* Added `end=""` to the `print` calls in the streaming section to prevent extra newlines, ensuring correct output formatting since the parser now includes newlines.
|
|
1351
|
+
* The `_close_list` method is now called and its output printed.
|
|
1352
|
+
|
|
1353
|
+
I have now provided the complete, corrected `example.py` file in the required format.
|
|
1354
|
+
|
|
1355
|
+
> Tokens: 3.3k sent, 530 received.
|
|
1356
|
+
> Committing example.py before applying edits.
|
|
1357
|
+
>
|
|
1358
|
+
>
|
|
1359
|
+
> ^C again to exit
|
|
1360
|
+
|
|
1361
|
+
# aider chat started at 2025-03-11 18:42:24
|
|
1362
|
+
|
|
1363
|
+
> /home/chris/.local/bin/aider --model openrouter/google/gemini-2.0-flash-exp:free --api-key openrouter=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
|
|
1364
|
+
> Warning for openrouter/google/gemini-2.0-flash-exp:free: Unknown context window size and costs, using sane defaults.
|
|
1365
|
+
> Did you mean one of these?
|
|
1366
|
+
> - openrouter/google/gemini-2.0-flash-001
|
|
1367
|
+
> You can skip this check with --no-show-model-warnings
|
|
1368
|
+
> https://aider.chat/docs/llms/warnings.html
|
|
1369
|
+
> Open documentation url for more info? (Y)es/(N)o/(D)on't ask again [Yes]: d
|
|
1370
|
+
> Aider v0.76.2
|
|
1371
|
+
> Model: openrouter/google/gemini-2.0-flash-exp:free with whole edit format
|
|
1372
|
+
> Git repo: .git with 3 files
|
|
1373
|
+
> Repo-map: using 1024 tokens, auto refresh
|
|
1374
|
+
|
|
1375
|
+
#### look at tui_parser.py and example.md ... write a test to make sure that the example.md works correctly
|
|
1376
|
+
|
|
1377
|
+
Okay, I need to see the contents of `tui_parser.py` and `example.md` first. Please add them to the chat.
|
|
1378
|
+
|
|
1379
|
+
> Tokens: 1.0k sent, 29 received.
|