streamdown 0.19.0__tar.gz → 0.20.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (74) hide show
  1. streamdown-0.20.0/.aider.chat.history.md +1379 -0
  2. streamdown-0.20.0/.aider.input.history +69 -0
  3. streamdown-0.20.0/.aider.tags.cache.v4/cache.db +0 -0
  4. {streamdown-0.19.0 → streamdown-0.20.0}/.gitignore +0 -1
  5. {streamdown-0.19.0 → streamdown-0.20.0}/PKG-INFO +24 -15
  6. {streamdown-0.19.0 → streamdown-0.20.0}/README.md +23 -14
  7. streamdown-0.20.0/fucking-garbage.md +19 -0
  8. {streamdown-0.19.0 → streamdown-0.20.0}/pyproject.toml +3 -1
  9. {streamdown-0.19.0 → streamdown-0.20.0}/ss.py +5 -1
  10. {streamdown-0.19.0 → streamdown-0.20.0}/streamdown/sd.py +88 -23
  11. streamdown-0.20.0/test.py +13 -0
  12. streamdown-0.20.0/tester.py +29 -0
  13. streamdown-0.20.0/tests/bg-messed-up.md +8 -0
  14. streamdown-0.20.0/tests/broken-code.md +52 -0
  15. {streamdown-0.19.0 → streamdown-0.20.0}/tests/chunk-buffer.sh +1 -1
  16. streamdown-0.20.0/tests/qwen3.md +2276 -0
  17. streamdown-0.20.0/tests/rerun.zsh +3 -0
  18. streamdown-0.19.0/test.py +0 -59
  19. streamdown-0.19.0/tests/escape.md +0 -7
  20. {streamdown-0.19.0 → streamdown-0.20.0}/.vimrc +0 -0
  21. {streamdown-0.19.0 → streamdown-0.20.0}/24-bit-color.sh +0 -0
  22. {streamdown-0.19.0 → streamdown-0.20.0}/LICENSE.MIT +0 -0
  23. {streamdown-0.19.0 → streamdown-0.20.0}/assets/logo.png +0 -0
  24. {streamdown-0.19.0 → streamdown-0.20.0}/assets/logo.svg +0 -0
  25. {streamdown-0.19.0 → streamdown-0.20.0}/configurable.png +0 -0
  26. {streamdown-0.19.0 → streamdown-0.20.0}/copyable.png +0 -0
  27. {streamdown-0.19.0 → streamdown-0.20.0}/dunder.png +0 -0
  28. {streamdown-0.19.0 → streamdown-0.20.0}/error.txt +0 -0
  29. {streamdown-0.19.0 → streamdown-0.20.0}/newdir/file_0.py +0 -0
  30. {streamdown-0.19.0 → streamdown-0.20.0}/newdir/file_1.rb +0 -0
  31. {streamdown-0.19.0 → streamdown-0.20.0}/newdir/file_2.jl +0 -0
  32. {streamdown-0.19.0 → streamdown-0.20.0}/passthrough.py +0 -0
  33. {streamdown-0.19.0 → streamdown-0.20.0}/python-go.png +0 -0
  34. {streamdown-0.19.0 → streamdown-0.20.0}/requirements.txt +0 -0
  35. {streamdown-0.19.0 → streamdown-0.20.0}/somelog.txt +0 -0
  36. {streamdown-0.19.0 → streamdown-0.20.0}/ss-new.py +0 -0
  37. {streamdown-0.19.0 → streamdown-0.20.0}/streamdown/__init__.py +0 -0
  38. {streamdown-0.19.0 → streamdown-0.20.0}/streamdown/plugins/README.md +0 -0
  39. {streamdown-0.19.0 → streamdown-0.20.0}/streamdown/plugins/latex.py +0 -0
  40. {streamdown-0.19.0 → streamdown-0.20.0}/streamdown/ss +0 -0
  41. {streamdown-0.19.0 → streamdown-0.20.0}/streamdown/ss1 +0 -0
  42. {streamdown-0.19.0 → streamdown-0.20.0}/table.png +0 -0
  43. {streamdown-0.19.0 → streamdown-0.20.0}/temp.py +0 -0
  44. {streamdown-0.19.0 → streamdown-0.20.0}/test_input.md +0 -0
  45. {streamdown-0.19.0 → streamdown-0.20.0}/tests/README.md +0 -0
  46. {streamdown-0.19.0 → streamdown-0.20.0}/tests/backtick-with-post-spaces.md +0 -0
  47. {streamdown-0.19.0 → streamdown-0.20.0}/tests/block.md +0 -0
  48. {streamdown-0.19.0 → streamdown-0.20.0}/tests/bold_reset_with_link.md +0 -0
  49. {streamdown-0.19.0 → streamdown-0.20.0}/tests/broken-example.md +0 -0
  50. {streamdown-0.19.0 → streamdown-0.20.0}/tests/chinese.md +0 -0
  51. {streamdown-0.19.0 → streamdown-0.20.0}/tests/cjj.mv +0 -0
  52. {streamdown-0.19.0 → streamdown-0.20.0}/tests/cjk-table.md +0 -0
  53. {streamdown-0.19.0 → streamdown-0.20.0}/tests/cjk-wrap.md +0 -0
  54. {streamdown-0.19.0 → streamdown-0.20.0}/tests/code.md +0 -0
  55. {streamdown-0.19.0 → streamdown-0.20.0}/tests/example.md +0 -0
  56. {streamdown-0.19.0 → streamdown-0.20.0}/tests/fizzbuzz.md +0 -0
  57. {streamdown-0.19.0 → streamdown-0.20.0}/tests/inline.md +0 -0
  58. {streamdown-0.19.0 → streamdown-0.20.0}/tests/jimmy_webb.md +0 -0
  59. {streamdown-0.19.0 → streamdown-0.20.0}/tests/line-buffer.sh +0 -0
  60. {streamdown-0.19.0 → streamdown-0.20.0}/tests/line-wrap.md +0 -0
  61. {streamdown-0.19.0 → streamdown-0.20.0}/tests/links.md +0 -0
  62. {streamdown-0.19.0 → streamdown-0.20.0}/tests/managerie.md +0 -0
  63. {streamdown-0.19.0 → streamdown-0.20.0}/tests/mandlebrot.md +0 -0
  64. {streamdown-0.19.0 → streamdown-0.20.0}/tests/markdown.md +0 -0
  65. {streamdown-0.19.0 → streamdown-0.20.0}/tests/nested-example.md +0 -0
  66. {streamdown-0.19.0 → streamdown-0.20.0}/tests/outline.md +0 -0
  67. {streamdown-0.19.0 → streamdown-0.20.0}/tests/pvgo_512.jpg +0 -0
  68. {streamdown-0.19.0 → streamdown-0.20.0}/tests/pythonvgo.md +0 -0
  69. {streamdown-0.19.0 → streamdown-0.20.0}/tests/strip-chunks.sh +0 -0
  70. {streamdown-0.19.0 → streamdown-0.20.0}/tests/table-break.md +0 -0
  71. {streamdown-0.19.0 → streamdown-0.20.0}/tests/table_test.md +0 -0
  72. {streamdown-0.19.0 → streamdown-0.20.0}/tests/test.md +0 -0
  73. {streamdown-0.19.0 → streamdown-0.20.0}/tests/test_input.md +0 -0
  74. {streamdown-0.19.0 → streamdown-0.20.0}/tests/wm.md +0 -0
@@ -0,0 +1,1379 @@
1
+
2
+ # aider chat started at 2025-03-11 17:24:49
3
+
4
+ > Newer aider version v0.76.2 is available.
5
+ > /home/chris/.local/share/uv/tools/aider-chat/bin/python -m pip install --upgrade --upgrade-strategy only-if-needed aider-chat
6
+ > Run pip install? (Y)es/(N)o [Yes]: y
7
+ > Re-run aider to use new version.
8
+
9
+ # aider chat started at 2025-03-11 17:25:22
10
+
11
+ > No git repo found, create one to track aider's changes (recommended)? (Y)es/(N)o [Yes]: y
12
+ > Added .aider* to .gitignore
13
+ > Git repository created in /home/chris/code/stream
14
+ > /home/chris/.local/bin/aider
15
+ > Found OPENROUTER_API_KEY so using openrouter/anthropic/claude-3.7-sonnet since no --model was specified.
16
+ > Aider v0.76.2
17
+ > Main model: openrouter/anthropic/claude-3.7-sonnet with diff edit format, infinite output
18
+ > Weak model: openrouter/anthropic/claude-3-5-haiku
19
+ > Git repo: .git with 0 files
20
+ > Repo-map: using 4096 tokens, auto refresh
21
+ > https://aider.chat/HISTORY.html#release-notes
22
+ > Would you like to see what's new in this version? (Y)es/(N)o [Yes]: n
23
+
24
+ #### let's make a streaming markdown parser. For now let's support h1-h6, ul, ol, and li.
25
+
26
+ I'll create a streaming markdown parser that supports h1-h6, ul, ol, and li elements. Let's create a new Python module for this.
27
+
28
+ First, I'll create a basic structure with a streaming parser that processes markdown content incrementally.
29
+
30
+ markdown_parser.py
31
+ ```python
32
+ <<<<<<< SEARCH
33
+ =======
34
+ """
35
+ Streaming Markdown Parser
36
+
37
+ A simple streaming parser for Markdown that supports:
38
+ - Headers (h1-h6)
39
+ - Unordered lists (ul)
40
+ - Ordered lists (ol)
41
+ - List items (li)
42
+ """
43
+
44
+ class MarkdownParser:
45
+ def __init__(self):
46
+ self.in_list = False
47
+ self.list_type = None
48
+ self.list_level = 0
49
+ self.buffer = []
50
+
51
+ def parse_line(self, line):
52
+ """Parse a single line of markdown and return HTML."""
53
+ line = line.rstrip()
54
+
55
+ # Empty line
56
+ if not line:
57
+ if self.in_list:
58
+ # Close any open list when encountering empty line
59
+ html = self._close_list()
60
+ self.in_list = False
61
+ self.list_type = None
62
+ self.list_level = 0
63
+ return html
64
+ return ""
65
+
66
+ # Headers (h1-h6)
67
+ if line.startswith('#'):
68
+ # Count the number of # at the beginning
69
+ level = 0
70
+ for char in line:
71
+ if char == '#':
72
+ level += 1
73
+ else:
74
+ break
75
+
76
+ if level >= 1 and level <= 6 and (len(line) == level or line[level] == ' '):
77
+ # Close any open list
78
+ html = ""
79
+ if self.in_list:
80
+ html = self._close_list()
81
+ self.in_list = False
82
+ self.list_type = None
83
+ self.list_level = 0
84
+
85
+ # Extract header content (skip the # and any leading space)
86
+ content = line[level:].lstrip()
87
+ return f"{html}<h{level}>{content}</h{level}>"
88
+
89
+ # Unordered list
90
+ if line.lstrip().startswith(('- ', '* ', '+ ')):
91
+ indent = len(line) - len(line.lstrip())
92
+ content = line.lstrip()[2:] # Skip the list marker and space
93
+
94
+ return self._handle_list_item('ul', indent, content)
95
+
96
+ # Ordered list
97
+ if line.lstrip().startswith(tuple(f"{i}. " for i in range(1, 10))):
98
+ indent = len(line) - len(line.lstrip())
99
+ # Skip the digit(s), period, and space
100
+ content = line.lstrip()
101
+ content = content[content.index('. ') + 2:]
102
+
103
+ return self._handle_list_item('ol', indent, content)
104
+
105
+ # Regular paragraph (not implemented in this version)
106
+ # Close any open list
107
+ html = ""
108
+ if self.in_list:
109
+ html = self._close_list()
110
+ self.in_list = False
111
+ self.list_type = None
112
+ self.list_level = 0
113
+
114
+ return html + line
115
+
116
+ def _handle_list_item(self, list_type, indent, content):
117
+ """Handle a list item, managing nested lists."""
118
+ html = []
119
+
120
+ # Calculate list level based on indentation (4 spaces = 1 level)
121
+ new_level = indent // 4 + 1
122
+
123
+ # If we're not in a list or changing list type
124
+ if not self.in_list:
125
+ html.append(f"<{list_type}>")
126
+ self.in_list = True
127
+ self.list_type = list_type
128
+ self.list_level = 1
129
+ # If we're going deeper in nesting
130
+ elif new_level > self.list_level:
131
+ # Open a new nested list
132
+ html.append(f"<{list_type}>")
133
+ self.list_level = new_level
134
+ self.list_type = list_type
135
+ # If we're going up in the nesting
136
+ elif new_level < self.list_level:
137
+ # Close the appropriate number of lists
138
+ for _ in range(self.list_level - new_level):
139
+ html.append(f"</li></{self.list_type}>")
140
+ self.list_level = new_level
141
+ # Same level but different list type
142
+ elif list_type != self.list_type:
143
+ html.append(f"</li></{self.list_type}>")
144
+ html.append(f"<{list_type}>")
145
+ self.list_type = list_type
146
+ else:
147
+ # Same level, same type - close previous item
148
+ html.append("</li>")
149
+
150
+ # Add the new list item
151
+ html.append(f"<li>{content}")
152
+
153
+ return "".join(html)
154
+
155
+ def _close_list(self):
156
+ """Close any open list tags."""
157
+ if not self.in_list:
158
+ return ""
159
+
160
+ html = "</li>"
161
+ for _ in range(self.list_level):
162
+ html += f"</{self.list_type}>"
163
+
164
+ return html
165
+
166
+ def parse(self, markdown_text):
167
+ """Parse a complete markdown text and return HTML."""
168
+ lines = markdown_text.split('\n')
169
+ html_parts = []
170
+
171
+ for line in lines:
172
+ html_parts.append(self.parse_line(line))
173
+
174
+ # Close any open lists at the end
175
+ if self.in_list:
176
+ html_parts.append(self._close_list())
177
+ self.in_list = False
178
+ self.list_type = None
179
+ self.list_level = 0
180
+
181
+ return '\n'.join(filter(None, html_parts))
182
+
183
+ def parse_stream(self, stream):
184
+ """Parse a stream of markdown lines and yield HTML fragments."""
185
+ for line in stream:
186
+ yield self.parse_line(line)
187
+
188
+ # Close any open lists at the end
189
+ if self.in_list:
190
+ yield self._close_list()
191
+ self.in_list = False
192
+ self.list_type = None
193
+ self.list_level = 0
194
+ >>>>>>> REPLACE
195
+ ```
196
+
197
+ Now, let's create a simple example script to demonstrate how to use the parser:
198
+
199
+ example.py
200
+ ```python
201
+ <<<<<<< SEARCH
202
+ =======
203
+ #!/usr/bin/env python3
204
+ """
205
+ Example usage of the Markdown Parser
206
+ """
207
+
208
+ from markdown_parser import MarkdownParser
209
+
210
+ def main():
211
+ # Example markdown text
212
+ markdown_text = """# Heading 1
213
+ ## Heading 2
214
+ ### Heading 3
215
+ #### Heading 4
216
+ ##### Heading 5
217
+ ###### Heading 6
218
+
219
+ - Unordered list item 1
220
+ - Unordered list item 2
221
+ - Nested unordered item 2.1
222
+ - Nested unordered item 2.2
223
+ - Deeply nested item 2.2.1
224
+ - Unordered list item 3
225
+
226
+ 1. Ordered list item 1
227
+ 2. Ordered list item 2
228
+ 1. Nested ordered item 2.1
229
+ 2. Nested ordered item 2.2
230
+ 3. Ordered list item 3
231
+
232
+ - Mixed list example
233
+ 1. Nested ordered in unordered
234
+ 2. Another nested item
235
+ - Back to unordered
236
+ """
237
+
238
+ # Parse the complete text
239
+ parser = MarkdownParser()
240
+ html = parser.parse(markdown_text)
241
+ print("Complete parsing result:")
242
+ print(html)
243
+ print("\n" + "-" * 50 + "\n")
244
+
245
+ # Demonstrate streaming parsing
246
+ print("Streaming parsing result:")
247
+ stream_parser = MarkdownParser()
248
+ for line in markdown_text.split('\n'):
249
+ html_fragment = stream_parser.parse_line(line)
250
+ if html_fragment:
251
+ print(f"Line: {line}")
252
+ print(f"HTML: {html_fragment}")
253
+ print()
254
+
255
+ # Don't forget to close any open tags
256
+ closing_html = stream_parser._close_list() if stream_parser.in_list else ""
257
+ if closing_html:
258
+ print(f"Closing HTML: {closing_html}")
259
+
260
+ if __name__ == "__main__":
261
+ main()
262
+ >>>>>>> REPLACE
263
+ ```
264
+
265
+ Let's also create a simple test file to verify our parser works correctly:
266
+
267
+ test_markdown_parser.py
268
+ ```python
269
+ <<<<<<< SEARCH
270
+ =======
271
+ #!/usr/bin/env python3
272
+ """
273
+ Tests for the Markdown Parser
274
+ """
275
+
276
+ import unittest
277
+ from markdown_parser import MarkdownParser
278
+
279
+ class TestMarkdownParser(unittest.TestCase):
280
+ def setUp(self):
281
+ self.parser = MarkdownParser()
282
+
283
+ def test_headers(self):
284
+ self.assertEqual(self.parser.parse_line("# Header 1"), "<h1>Header 1</h1>")
285
+ self.assertEqual(self.parser.parse_line("## Header 2"), "<h2>Header 2</h2>")
286
+ self.assertEqual(self.parser.parse_line("### Header 3"), "<h3>Header 3</h3>")
287
+ self.assertEqual(self.parser.parse_line("#### Header 4"), "<h4>Header 4</h4>")
288
+ self.assertEqual(self.parser.parse_line("##### Header 5"), "<h5>Header 5</h5>")
289
+ self.assertEqual(self.parser.parse_line("###### Header 6"), "<h6>Header 6</h6>")
290
+
291
+ # Invalid headers
292
+ self.assertEqual(self.parser.parse_line("####### Too many"), "####### Too many")
293
+ self.assertEqual(self.parser.parse_line("#No space"), "#No space")
294
+
295
+ def test_unordered_lists(self):
296
+ # Simple list
297
+ self.assertEqual(self.parser.parse_line("- Item 1"), "<ul><li>Item 1")
298
+ self.assertEqual(self.parser.parse_line("- Item 2"), "</li><li>Item 2")
299
+ self.assertEqual(self.parser.parse_line(""), "</li></ul>")
300
+
301
+ # Reset parser state
302
+ self.parser = MarkdownParser()
303
+
304
+ # Different markers
305
+ self.assertEqual(self.parser.parse_line("* Item 1"), "<ul><li>Item 1")
306
+ self.assertEqual(self.parser.parse_line("+ Item 2"), "</li><li>Item 2")
307
+
308
+ # Reset parser state
309
+ self.parser = MarkdownParser()
310
+
311
+ # Nested list
312
+ self.assertEqual(self.parser.parse_line("- Item 1"), "<ul><li>Item 1")
313
+ self.assertEqual(self.parser.parse_line(" - Nested 1"), "</li><ul><li>Nested 1")
314
+ self.assertEqual(self.parser.parse_line("- Item 2"), "</li></ul></li><li>Item 2")
315
+
316
+ def test_ordered_lists(self):
317
+ # Simple ordered list
318
+ self.assertEqual(self.parser.parse_line("1. Item 1"), "<ol><li>Item 1")
319
+ self.assertEqual(self.parser.parse_line("2. Item 2"), "</li><li>Item 2")
320
+ self.assertEqual(self.parser.parse_line(""), "</li></ol>")
321
+
322
+ # Reset parser state
323
+ self.parser = MarkdownParser()
324
+
325
+ # Nested ordered list
326
+ self.assertEqual(self.parser.parse_line("1. Item 1"), "<ol><li>Item 1")
327
+ self.assertEqual(self.parser.parse_line(" 1. Nested 1"), "</li><ol><li>Nested 1")
328
+ self.assertEqual(self.parser.parse_line("2. Item 2"), "</li></ol></li><li>Item 2")
329
+
330
+ def test_mixed_lists(self):
331
+ # Unordered list with nested ordered list
332
+ self.assertEqual(self.parser.parse_line("- Item 1"), "<ul><li>Item 1")
333
+ self.assertEqual(self.parser.parse_line(" 1. Nested 1"), "</li><ol><li>Nested 1")
334
+ self.assertEqual(self.parser.parse_line("- Item 2"), "</li></ol></li><li>Item 2")
335
+
336
+ def test_complete_parse(self):
337
+ markdown = "# Header\n\n- Item 1\n- Item 2\n\n1. Number 1\n2. Number 2"
338
+ expected = "<h1>Header</h1>\n<ul><li>Item 1</li><li>Item 2</li></ul>\n<ol><li>Number 1</li><li>Number 2</li></ol>"
339
+ self.assertEqual(self.parser.parse(markdown), expected)
340
+
341
+ if __name__ == "__main__":
342
+ unittest.main()
343
+ >>>>>>> REPLACE
344
+ ```
345
+
346
+ You can run the example and tests with the following commands:
347
+
348
+ ```bash
349
+ python3 example.py
350
+ python3 test_markdown_parser.py
351
+ ```
352
+
353
+ > Tokens: 2.3k sent, 2.5k received. Cost: $0.04 message, $0.04 session.
354
+ > markdown_parser.py
355
+ > Create new file? (Y)es/(N)o [Yes]: y
356
+ > example.py
357
+ > Create new file? (Y)es/(N)o [Yes]: y
358
+ > test_markdown_parser.py
359
+ > Create new file? (Y)es/(N)o [Yes]: y
360
+ > Applied edit to example.py
361
+ > Applied edit to test_markdown_parser.py
362
+ > Applied edit to markdown_parser.py
363
+ >
364
+ >
365
+ > ^C again to exit
366
+
367
+ #### so i want it as a tui output, not HTML
368
+
369
+ I'll modify the code to create a TUI (Text User Interface) output instead of HTML. Let's update the files to render markdown as formatted text with proper indentation and styling for the terminal.
370
+
371
+ markdown_parser.py
372
+ ```python
373
+ <<<<<<< SEARCH
374
+ """
375
+ Streaming Markdown Parser
376
+
377
+ A simple streaming parser for Markdown that supports:
378
+ - Headers (h1-h6)
379
+ - Unordered lists (ul)
380
+ - Ordered lists (ol)
381
+ - List items (li)
382
+ """
383
+
384
+ class MarkdownParser:
385
+ def __init__(self):
386
+ self.in_list = False
387
+ self.list_type = None
388
+ self.list_level = 0
389
+ self.buffer = []
390
+
391
+ def parse_line(self, line):
392
+ """Parse a single line of markdown and return HTML."""
393
+ line = line.rstrip()
394
+
395
+ # Empty line
396
+ if not line:
397
+ if self.in_list:
398
+ # Close any open list when encountering empty line
399
+ html = self._close_list()
400
+ self.in_list = False
401
+ self.list_type = None
402
+ self.list_level = 0
403
+ return html
404
+ return ""
405
+
406
+ # Headers (h1-h6)
407
+ if line.startswith('#'):
408
+ # Count the number of # at the beginning
409
+ level = 0
410
+ for char in line:
411
+ if char == '#':
412
+ level += 1
413
+ else:
414
+ break
415
+
416
+ if level >= 1 and level <= 6 and (len(line) == level or line[level] == ' '):
417
+ # Close any open list
418
+ html = ""
419
+ if self.in_list:
420
+ html = self._close_list()
421
+ self.in_list = False
422
+ self.list_type = None
423
+ self.list_level = 0
424
+
425
+ # Extract header content (skip the # and any leading space)
426
+ content = line[level:].lstrip()
427
+ return f"{html}<h{level}>{content}</h{level}>"
428
+
429
+ # Unordered list
430
+ if line.lstrip().startswith(('- ', '* ', '+ ')):
431
+ indent = len(line) - len(line.lstrip())
432
+ content = line.lstrip()[2:] # Skip the list marker and space
433
+
434
+ return self._handle_list_item('ul', indent, content)
435
+
436
+ # Ordered list
437
+ if line.lstrip().startswith(tuple(f"{i}. " for i in range(1, 10))):
438
+ indent = len(line) - len(line.lstrip())
439
+ # Skip the digit(s), period, and space
440
+ content = line.lstrip()
441
+ content = content[content.index('. ') + 2:]
442
+
443
+ return self._handle_list_item('ol', indent, content)
444
+
445
+ # Regular paragraph (not implemented in this version)
446
+ # Close any open list
447
+ html = ""
448
+ if self.in_list:
449
+ html = self._close_list()
450
+ self.in_list = False
451
+ self.list_type = None
452
+ self.list_level = 0
453
+
454
+ return html + line
455
+
456
+ def _handle_list_item(self, list_type, indent, content):
457
+ """Handle a list item, managing nested lists."""
458
+ html = []
459
+
460
+ # Calculate list level based on indentation (4 spaces = 1 level)
461
+ new_level = indent // 4 + 1
462
+
463
+ # If we're not in a list or changing list type
464
+ if not self.in_list:
465
+ html.append(f"<{list_type}>")
466
+ self.in_list = True
467
+ self.list_type = list_type
468
+ self.list_level = 1
469
+ # If we're going deeper in nesting
470
+ elif new_level > self.list_level:
471
+ # Open a new nested list
472
+ html.append(f"<{list_type}>")
473
+ self.list_level = new_level
474
+ self.list_type = list_type
475
+ # If we're going up in the nesting
476
+ elif new_level < self.list_level:
477
+ # Close the appropriate number of lists
478
+ for _ in range(self.list_level - new_level):
479
+ html.append(f"</li></{self.list_type}>")
480
+ self.list_level = new_level
481
+ # Same level but different list type
482
+ elif list_type != self.list_type:
483
+ html.append(f"</li></{self.list_type}>")
484
+ html.append(f"<{list_type}>")
485
+ self.list_type = list_type
486
+ else:
487
+ # Same level, same type - close previous item
488
+ html.append("</li>")
489
+
490
+ # Add the new list item
491
+ html.append(f"<li>{content}")
492
+
493
+ return "".join(html)
494
+
495
+ def _close_list(self):
496
+ """Close any open list tags."""
497
+ if not self.in_list:
498
+ return ""
499
+
500
+ html = "</li>"
501
+ for _ in range(self.list_level):
502
+ html += f"</{self.list_type}>"
503
+
504
+ return html
505
+
506
+ def parse(self, markdown_text):
507
+ """Parse a complete markdown text and return HTML."""
508
+ lines = markdown_text.split('\n')
509
+ html_parts = []
510
+
511
+ for line in lines:
512
+ html_parts.append(self.parse_line(line))
513
+
514
+ # Close any open lists at the end
515
+ if self.in_list:
516
+ html_parts.append(self._close_list())
517
+ self.in_list = False
518
+ self.list_type = None
519
+ self.list_level = 0
520
+
521
+ return '\n'.join(filter(None, html_parts))
522
+
523
+ def parse_stream(self, stream):
524
+ """Parse a stream of markdown lines and yield HTML fragments."""
525
+ for line in stream:
526
+ yield self.parse_line(line)
527
+
528
+ # Close any open lists at the end
529
+ if self.in_list:
530
+ yield self._close_list()
531
+ self.in_list = False
532
+ self.list_type = None
533
+ self.list_level = 0
534
+ =======
535
+ """
536
+ Streaming Markdown Parser for TUI
537
+
538
+ A simple streaming parser for Markdown that supports:
539
+ - Headers (h1-h6)
540
+ - Unordered lists (ul)
541
+ - Ordered lists (ol)
542
+ - List items (li)
543
+
544
+ Output is formatted for terminal display.
545
+ """
546
+
547
+ class MarkdownParser:
548
+ def __init__(self):
549
+ self.in_list = False
550
+ self.list_type = None
551
+ self.list_level = 0
552
+ self.buffer = []
553
+
554
+ def parse_line(self, line):
555
+ """Parse a single line of markdown and return formatted text for terminal."""
556
+ line = line.rstrip()
557
+
558
+ # Empty line
559
+ if not line:
560
+ if self.in_list:
561
+ # Close any open list when encountering empty line
562
+ self.in_list = False
563
+ self.list_type = None
564
+ self.list_level = 0
565
+ return ""
566
+
567
+ # Headers (h1-h6)
568
+ if line.startswith('#'):
569
+ # Count the number of # at the beginning
570
+ level = 0
571
+ for char in line:
572
+ if char == '#':
573
+ level += 1
574
+ else:
575
+ break
576
+
577
+ if level >= 1 and level <= 6 and (len(line) == level or line[level] == ' '):
578
+ # Close any open list
579
+ if self.in_list:
580
+ self.in_list = False
581
+ self.list_type = None
582
+ self.list_level = 0
583
+
584
+ # Extract header content (skip the # and any leading space)
585
+ content = line[level:].lstrip()
586
+
587
+ # Format headers based on level
588
+ if level == 1:
589
+ return f"\n{'=' * (len(content) + 4)}\n {content.upper()} \n{'=' * (len(content) + 4)}"
590
+ elif level == 2:
591
+ return f"\n{content}\n{'-' * len(content)}"
592
+ else:
593
+ # For h3-h6, use indentation and different styles
594
+ indent = " " * (level - 3)
595
+ prefix = "•" if level % 2 == 1 else ">"
596
+ return f"\n{indent}{prefix} {content}"
597
+
598
+ # Unordered list
599
+ if line.lstrip().startswith(('- ', '* ', '+ ')):
600
+ indent = len(line) - len(line.lstrip())
601
+ content = line.lstrip()[2:] # Skip the list marker and space
602
+
603
+ return self._handle_list_item('ul', indent, content)
604
+
605
+ # Ordered list
606
+ if line.lstrip().startswith(tuple(f"{i}. " for i in range(1, 10))):
607
+ indent = len(line) - len(line.lstrip())
608
+ # Skip the digit(s), period, and space
609
+ content = line.lstrip()
610
+ list_num = content.split('. ')[0]
611
+ content = content[content.index('. ') + 2:]
612
+
613
+ return self._handle_list_item('ol', indent, content, list_num)
614
+
615
+ # Regular paragraph (just return the text)
616
+ # Close any open list
617
+ if self.in_list:
618
+ self.in_list = False
619
+ self.list_type = None
620
+ self.list_level = 0
621
+
622
+ return line
623
+
624
+ def _handle_list_item(self, list_type, indent, content, list_num=None):
625
+ """Handle a list item, managing nested lists."""
626
+ # Calculate list level based on indentation (4 spaces = 1 level)
627
+ new_level = indent // 4 + 1
628
+
629
+ # Set list state
630
+ self.in_list = True
631
+ self.list_type = list_type
632
+ self.list_level = new_level
633
+
634
+ # Format the list item based on type and nesting level
635
+ indent_str = " " * (new_level - 1)
636
+
637
+ if list_type == 'ul':
638
+ bullets = ['•', '◦', '▪', '▫', '◘', '◙'] # Different bullet styles for nesting
639
+ bullet = bullets[(new_level - 1) % len(bullets)]
640
+ return f"{indent_str}{bullet} {content}"
641
+ else: # ordered list
642
+ return f"{indent_str}{list_num}. {content}"
643
+
644
+ def parse(self, markdown_text):
645
+ """Parse a complete markdown text and return formatted text."""
646
+ lines = markdown_text.split('\n')
647
+ text_parts = []
648
+
649
+ for line in lines:
650
+ parsed_line = self.parse_line(line)
651
+ if parsed_line:
652
+ text_parts.append(parsed_line)
653
+
654
+ return '\n'.join(text_parts)
655
+
656
+ def parse_stream(self, stream):
657
+ """Parse a stream of markdown lines and yield formatted text fragments."""
658
+ for line in stream:
659
+ yield self.parse_line(line)
660
+ >>>>>>> REPLACE
661
+ ```
662
+
663
+ example.py
664
+ ```python
665
+ <<<<<<< SEARCH
666
+ #!/usr/bin/env python3
667
+ """
668
+ Example usage of the Markdown Parser
669
+ """
670
+
671
+ from markdown_parser import MarkdownParser
672
+
673
+ def main():
674
+ # Example markdown text
675
+ markdown_text = """# Heading 1
676
+ ## Heading 2
677
+ ### Heading 3
678
+ #### Heading 4
679
+ ##### Heading 5
680
+ ###### Heading 6
681
+
682
+ - Unordered list item 1
683
+ - Unordered list item 2
684
+ - Nested unordered item 2.1
685
+ - Nested unordered item 2.2
686
+ - Deeply nested item 2.2.1
687
+ - Unordered list item 3
688
+
689
+ 1. Ordered list item 1
690
+ 2. Ordered list item 2
691
+ 1. Nested ordered item 2.1
692
+ 2. Nested ordered item 2.2
693
+ 3. Ordered list item 3
694
+
695
+ - Mixed list example
696
+ 1. Nested ordered in unordered
697
+ 2. Another nested item
698
+ - Back to unordered
699
+ """
700
+
701
+ # Parse the complete text
702
+ parser = MarkdownParser()
703
+ html = parser.parse(markdown_text)
704
+ print("Complete parsing result:")
705
+ print(html)
706
+ print("\n" + "-" * 50 + "\n")
707
+
708
+ # Demonstrate streaming parsing
709
+ print("Streaming parsing result:")
710
+ stream_parser = MarkdownParser()
711
+ for line in markdown_text.split('\n'):
712
+ html_fragment = stream_parser.parse_line(line)
713
+ if html_fragment:
714
+ print(f"Line: {line}")
715
+ print(f"HTML: {html_fragment}")
716
+ print()
717
+
718
+ # Don't forget to close any open tags
719
+ closing_html = stream_parser._close_list() if stream_parser.in_list else ""
720
+ if closing_html:
721
+ print(f"Closing HTML: {closing_html}")
722
+
723
+ if __name__ == "__main__":
724
+ main()
725
+ =======
726
+ #!/usr/bin/env python3
727
+ """
728
+ Example usage of the Markdown Parser for TUI output
729
+ """
730
+
731
+ from markdown_parser import MarkdownParser
732
+ import sys
733
+
734
+ def main():
735
+ # Example markdown text
736
+ markdown_text = """# Heading 1
737
+ ## Heading 2
738
+ ### Heading 3
739
+ #### Heading 4
740
+ ##### Heading 5
741
+ ###### Heading 6
742
+
743
+ - Unordered list item 1
744
+ - Unordered list item 2
745
+ - Nested unordered item 2.1
746
+ - Nested unordered item 2.2
747
+ - Deeply nested item 2.2.1
748
+ - Unordered list item 3
749
+
750
+ 1. Ordered list item 1
751
+ 2. Ordered list item 2
752
+ 1. Nested ordered item 2.1
753
+ 2. Nested ordered item 2.2
754
+ 3. Ordered list item 3
755
+
756
+ - Mixed list example
757
+ 1. Nested ordered in unordered
758
+ 2. Another nested item
759
+ - Back to unordered
760
+ """
761
+
762
+ # Parse the complete text
763
+ parser = MarkdownParser()
764
+ formatted_text = parser.parse(markdown_text)
765
+ print("Complete parsing result:")
766
+ print(formatted_text)
767
+ print("\n" + "-" * 50 + "\n")
768
+
769
+ # Demonstrate streaming parsing
770
+ print("Streaming parsing result:")
771
+ stream_parser = MarkdownParser()
772
+ for line in markdown_text.split('\n'):
773
+ formatted_line = stream_parser.parse_line(line)
774
+ if formatted_line:
775
+ print(formatted_line)
776
+
777
+ # Check if a file was provided as an argument
778
+ if len(sys.argv) > 1:
779
+ try:
780
+ with open(sys.argv[1], 'r') as file:
781
+ file_parser = MarkdownParser()
782
+ print("\n" + "=" * 50)
783
+ print(f"Parsing file: {sys.argv[1]}")
784
+ print("=" * 50 + "\n")
785
+ print(file_parser.parse(file.read()))
786
+ except FileNotFoundError:
787
+ print(f"Error: File '{sys.argv[1]}' not found.")
788
+ except Exception as e:
789
+ print(f"Error parsing file: {e}")
790
+
791
+ if __name__ == "__main__":
792
+ main()
793
+ >>>>>>> REPLACE
794
+ ```
795
+
796
+ test_markdown_parser.py
797
+ ```python
798
+ <<<<<<< SEARCH
799
+ #!/usr/bin/env python3
800
+ """
801
+ Tests for the Markdown Parser
802
+ """
803
+
804
+ import unittest
805
+ from markdown_parser import MarkdownParser
806
+
807
+ class TestMarkdownParser(unittest.TestCase):
808
+ def setUp(self):
809
+ self.parser = MarkdownParser()
810
+
811
+ def test_headers(self):
812
+ self.assertEqual(self.parser.parse_line("# Header 1"), "<h1>Header 1</h1>")
813
+ self.assertEqual(self.parser.parse_line("## Header 2"), "<h2>Header 2</h2>")
814
+ self.assertEqual(self.parser.parse_line("### Header 3"), "<h3>Header 3</h3>")
815
+ self.assertEqual(self.parser.parse_line("#### Header 4"), "<h4>Header 4</h4>")
816
+ self.assertEqual(self.parser.parse_line("##### Header 5"), "<h5>Header 5</h5>")
817
+ self.assertEqual(self.parser.parse_line("###### Header 6"), "<h6>Header 6</h6>")
818
+
819
+ # Invalid headers
820
+ self.assertEqual(self.parser.parse_line("####### Too many"), "####### Too many")
821
+ self.assertEqual(self.parser.parse_line("#No space"), "#No space")
822
+
823
+ def test_unordered_lists(self):
824
+ # Simple list
825
+ self.assertEqual(self.parser.parse_line("- Item 1"), "<ul><li>Item 1")
826
+ self.assertEqual(self.parser.parse_line("- Item 2"), "</li><li>Item 2")
827
+ self.assertEqual(self.parser.parse_line(""), "</li></ul>")
828
+
829
+ # Reset parser state
830
+ self.parser = MarkdownParser()
831
+
832
+ # Different markers
833
+ self.assertEqual(self.parser.parse_line("* Item 1"), "<ul><li>Item 1")
834
+ self.assertEqual(self.parser.parse_line("+ Item 2"), "</li><li>Item 2")
835
+
836
+ # Reset parser state
837
+ self.parser = MarkdownParser()
838
+
839
+ # Nested list
840
+ self.assertEqual(self.parser.parse_line("- Item 1"), "<ul><li>Item 1")
841
+ self.assertEqual(self.parser.parse_line(" - Nested 1"), "</li><ul><li>Nested 1")
842
+ self.assertEqual(self.parser.parse_line("- Item 2"), "</li></ul></li><li>Item 2")
843
+
844
+ def test_ordered_lists(self):
845
+ # Simple ordered list
846
+ self.assertEqual(self.parser.parse_line("1. Item 1"), "<ol><li>Item 1")
847
+ self.assertEqual(self.parser.parse_line("2. Item 2"), "</li><li>Item 2")
848
+ self.assertEqual(self.parser.parse_line(""), "</li></ol>")
849
+
850
+ # Reset parser state
851
+ self.parser = MarkdownParser()
852
+
853
+ # Nested ordered list
854
+ self.assertEqual(self.parser.parse_line("1. Item 1"), "<ol><li>Item 1")
855
+ self.assertEqual(self.parser.parse_line(" 1. Nested 1"), "</li><ol><li>Nested 1")
856
+ self.assertEqual(self.parser.parse_line("2. Item 2"), "</li></ol></li><li>Item 2")
857
+
858
+ def test_mixed_lists(self):
859
+ # Unordered list with nested ordered list
860
+ self.assertEqual(self.parser.parse_line("- Item 1"), "<ul><li>Item 1")
861
+ self.assertEqual(self.parser.parse_line(" 1. Nested 1"), "</li><ol><li>Nested 1")
862
+ self.assertEqual(self.parser.parse_line("- Item 2"), "</li></ol></li><li>Item 2")
863
+
864
+ def test_complete_parse(self):
865
+ markdown = "# Header\n\n- Item 1\n- Item 2\n\n1. Number 1\n2. Number 2"
866
+ expected = "<h1>Header</h1>\n<ul><li>Item 1</li><li>Item 2</li></ul>\n<ol><li>Number 1</li><li>Number 2</li></ol>"
867
+ self.assertEqual(self.parser.parse(markdown), expected)
868
+
869
+ if __name__ == "__main__":
870
+ unittest.main()
871
+ =======
872
+ #!/usr/bin/env python3
873
+ """
874
+ Tests for the Markdown Parser TUI output
875
+ """
876
+
877
+ import unittest
878
+ from markdown_parser import MarkdownParser
879
+
880
+ class TestMarkdownParser(unittest.TestCase):
881
+ def setUp(self):
882
+ self.parser = MarkdownParser()
883
+
884
+ def test_headers(self):
885
+ h1_expected = "\n=============\n HEADER 1 \n============="
886
+ self.assertEqual(self.parser.parse_line("# Header 1"), h1_expected)
887
+
888
+ h2_expected = "\nHeader 2\n--------"
889
+ self.assertEqual(self.parser.parse_line("## Header 2"), h2_expected)
890
+
891
+ h3_expected = "\n• Header 3"
892
+ self.assertEqual(self.parser.parse_line("### Header 3"), h3_expected)
893
+
894
+ h4_expected = "\n > Header 4"
895
+ self.assertEqual(self.parser.parse_line("#### Header 4"), h4_expected)
896
+
897
+ h5_expected = "\n • Header 5"
898
+ self.assertEqual(self.parser.parse_line("##### Header 5"), h5_expected)
899
+
900
+ h6_expected = "\n > Header 6"
901
+ self.assertEqual(self.parser.parse_line("###### Header 6"), h6_expected)
902
+
903
+ # Invalid headers
904
+ self.assertEqual(self.parser.parse_line("####### Too many"), "####### Too many")
905
+ self.assertEqual(self.parser.parse_line("#No space"), "#No space")
906
+
907
+ def test_unordered_lists(self):
908
+ # Simple list
909
+ self.assertEqual(self.parser.parse_line("- Item 1"), "• Item 1")
910
+ self.assertEqual(self.parser.parse_line("- Item 2"), "• Item 2")
911
+ self.assertEqual(self.parser.parse_line(""), "")
912
+
913
+ # Reset parser state
914
+ self.parser = MarkdownParser()
915
+
916
+ # Different markers
917
+ self.assertEqual(self.parser.parse_line("* Item 1"), "• Item 1")
918
+ self.assertEqual(self.parser.parse_line("+ Item 2"), "• Item 2")
919
+
920
+ # Reset parser state
921
+ self.parser = MarkdownParser()
922
+
923
+ # Nested list
924
+ self.assertEqual(self.parser.parse_line("- Item 1"), "• Item 1")
925
+ self.assertEqual(self.parser.parse_line(" - Nested 1"), " ◦ Nested 1")
926
+ self.assertEqual(self.parser.parse_line("- Item 2"), "• Item 2")
927
+
928
+ def test_ordered_lists(self):
929
+ # Simple ordered list
930
+ self.assertEqual(self.parser.parse_line("1. Item 1"), "1. Item 1")
931
+ self.assertEqual(self.parser.parse_line("2. Item 2"), "2. Item 2")
932
+ self.assertEqual(self.parser.parse_line(""), "")
933
+
934
+ # Reset parser state
935
+ self.parser = MarkdownParser()
936
+
937
+ # Nested ordered list
938
+ self.assertEqual(self.parser.parse_line("1. Item 1"), "1. Item 1")
939
+ self.assertEqual(self.parser.parse_line(" 1. Nested 1"), " 1. Nested 1")
940
+ self.assertEqual(self.parser.parse_line("2. Item 2"), "2. Item 2")
941
+
942
+ def test_mixed_lists(self):
943
+ # Unordered list with nested ordered list
944
+ self.assertEqual(self.parser.parse_line("- Item 1"), "• Item 1")
945
+ self.assertEqual(self.parser.parse_line(" 1. Nested 1"), " 1. Nested 1")
946
+ self.assertEqual(self.parser.parse_line("- Item 2"), "• Item 2")
947
+
948
+ def test_complete_parse(self):
949
+ markdown = "# Header\n\n- Item 1\n- Item 2\n\n1. Number 1\n2. Number 2"
950
+ expected = "\n===========\n HEADER \n===========\n• Item 1\n• Item 2\n1. Number 1\n2. Number 2"
951
+ self.assertEqual(self.parser.parse(markdown), expected)
952
+
953
+ if __name__ == "__main__":
954
+ unittest.main()
955
+ >>>>>>> REPLACE
956
+ ```
957
+
958
+ Now you can run the example and tests with the following commands:
959
+
960
+ ```bash
961
+ python3 example.py
962
+ python3 test_markdown_parser.py
963
+ ```
964
+
965
+ You can also parse a markdown file directly by providing its path as an argument:
966
+
967
+ ```bash
968
+ python3 example.py your_markdown_file.md
969
+ ```
970
+
971
+ > Tokens: 7.6k sent, 4.7k received. Cost: $0.09 message, $0.14 session.
972
+ > Committing markdown_parser.py before applying edits.
973
+ > Committing example.py before applying edits.
974
+ > Committing test_markdown_parser.py before applying edits.
975
+ >
976
+ >
977
+ > ^C again to exit
978
+
979
+ # aider chat started at 2025-03-11 17:35:49
980
+
981
+ > /home/chris/.local/bin/aider
982
+ > Found OPENROUTER_API_KEY so using openrouter/anthropic/claude-3.7-sonnet since no --model was specified.
983
+ > Aider v0.76.2
984
+ > Main model: openrouter/anthropic/claude-3.7-sonnet with diff edit format, infinite output
985
+ > Weak model: openrouter/anthropic/claude-3-5-haiku
986
+ > Git repo: .git with 3 files
987
+ > Repo-map: using 4096 tokens, auto refresh
988
+ >
989
+ >
990
+ > ^C again to exit
991
+ >
992
+ >
993
+ > ^C KeyboardInterrupt
994
+
995
+ # aider chat started at 2025-03-11 17:36:10
996
+
997
+ > /home/chris/.local/bin/aider --model openrouter/google/gemini-2.0-flash-exp:free
998
+ > Warning for openrouter/google/gemini-2.0-flash-exp:free: Unknown context window size and costs, using sane defaults.
999
+ > Did you mean one of these?
1000
+ > - openrouter/google/gemini-2.0-flash-001
1001
+ > You can skip this check with --no-show-model-warnings
1002
+ > https://aider.chat/docs/llms/warnings.html
1003
+ > Open documentation url for more info? (Y)es/(N)o/(D)on't ask again [Yes]: d
1004
+ > Aider v0.76.2
1005
+ > Model: openrouter/google/gemini-2.0-flash-exp:free with whole edit format
1006
+ > Git repo: .git with 3 files
1007
+ > Repo-map: using 1024 tokens, auto refresh
1008
+
1009
+ #### /models
1010
+ > Please provide a partial model name to search for.
1011
+
1012
+ #### /models free
1013
+ > Models which match "free":
1014
+ > - openrouter/meta-llama/llama-3-8b-instruct:free
1015
+ > - openrouter/mistralai/mistral-7b-instruct:free
1016
+
1017
+ #### models gemini
1018
+
1019
+ Okay, I understand the format you want me to use.
1020
+
1021
+ Please provide the request you would like me to fulfill. I will then tell you which files I think need to be edited.
1022
+
1023
+ > Tokens: 990 sent, 37 received.
1024
+ >
1025
+ >
1026
+ > ^C again to exit
1027
+
1028
+ #### /models gemini
1029
+ > Models which match "gemini":
1030
+ > - gemini-1.0-pro
1031
+ > - gemini-1.0-pro-001
1032
+ > - gemini-1.0-pro-002
1033
+ > - gemini-1.0-pro-vision
1034
+ > - gemini-1.0-pro-vision-001
1035
+ > - gemini-1.0-ultra
1036
+ > - gemini-1.0-ultra-001
1037
+ > - gemini-1.5-flash
1038
+ > - gemini-1.5-flash-001
1039
+ > - gemini-1.5-flash-002
1040
+ > - gemini-1.5-flash-exp-0827
1041
+ > - gemini-1.5-flash-preview-0514
1042
+ > - gemini-1.5-pro
1043
+ > - gemini-1.5-pro-001
1044
+ > - gemini-1.5-pro-002
1045
+ > - gemini-1.5-pro-preview-0215
1046
+ > - gemini-1.5-pro-preview-0409
1047
+ > - gemini-1.5-pro-preview-0514
1048
+ > - gemini-2.0-flash-001
1049
+ > - gemini-2.0-flash-exp
1050
+ > - gemini-2.0-flash-thinking-exp
1051
+ > - gemini-2.0-flash-thinking-exp-01-21
1052
+ > - gemini-2.0-pro-exp-02-05
1053
+ > - gemini-flash-experimental
1054
+ > - gemini-pro
1055
+ > - gemini-pro-experimental
1056
+ > - gemini-pro-vision
1057
+ > - gemini/gemini-1.5-flash
1058
+ > - gemini/gemini-1.5-flash-001
1059
+ > - gemini/gemini-1.5-flash-002
1060
+ > - gemini/gemini-1.5-flash-8b
1061
+ > - gemini/gemini-1.5-flash-8b-exp-0827
1062
+ > - gemini/gemini-1.5-flash-8b-exp-0924
1063
+ > - gemini/gemini-1.5-flash-exp-0827
1064
+ > - gemini/gemini-1.5-flash-latest
1065
+ > - gemini/gemini-1.5-pro
1066
+ > - gemini/gemini-1.5-pro-001
1067
+ > - gemini/gemini-1.5-pro-002
1068
+ > - gemini/gemini-1.5-pro-exp-0801
1069
+ > - gemini/gemini-1.5-pro-exp-0827
1070
+ > - gemini/gemini-1.5-pro-latest
1071
+ > - gemini/gemini-2.0-flash
1072
+ > - gemini/gemini-2.0-flash-001
1073
+ > - gemini/gemini-2.0-flash-exp
1074
+ > - gemini/gemini-2.0-flash-lite-preview-02-05
1075
+ > - gemini/gemini-2.0-flash-thinking-exp
1076
+ > - gemini/gemini-2.0-pro-exp-02-05
1077
+ > - gemini/gemini-exp-1114
1078
+ > - gemini/gemini-exp-1206
1079
+ > - gemini/gemini-gemma-2-27b-it
1080
+ > - gemini/gemini-gemma-2-9b-it
1081
+ > - gemini/gemini-pro
1082
+ > - gemini/gemini-pro-vision
1083
+ > - openrouter/google/gemini-2.0-flash-001
1084
+ > - openrouter/google/gemini-pro-1.5
1085
+ > - openrouter/google/gemini-pro-vision
1086
+ > - vertex_ai-language-models/gemini-1.0-pro
1087
+ > - vertex_ai-language-models/gemini-1.0-pro-001
1088
+ > - vertex_ai-language-models/gemini-1.0-pro-002
1089
+ > - vertex_ai-language-models/gemini-1.0-ultra
1090
+ > - vertex_ai-language-models/gemini-1.0-ultra-001
1091
+ > - vertex_ai-language-models/gemini-1.5-flash
1092
+ > - vertex_ai-language-models/gemini-1.5-flash-001
1093
+ > - vertex_ai-language-models/gemini-1.5-flash-002
1094
+ > - vertex_ai-language-models/gemini-1.5-flash-exp-0827
1095
+ > - vertex_ai-language-models/gemini-1.5-flash-preview-0514
1096
+ > - vertex_ai-language-models/gemini-1.5-pro
1097
+ > - vertex_ai-language-models/gemini-1.5-pro-001
1098
+ > - vertex_ai-language-models/gemini-1.5-pro-002
1099
+ > - vertex_ai-language-models/gemini-1.5-pro-preview-0215
1100
+ > - vertex_ai-language-models/gemini-1.5-pro-preview-0409
1101
+ > - vertex_ai-language-models/gemini-1.5-pro-preview-0514
1102
+ > - vertex_ai-language-models/gemini-2.0-flash-001
1103
+ > - vertex_ai-language-models/gemini-2.0-flash-exp
1104
+ > - vertex_ai-language-models/gemini-2.0-flash-thinking-exp
1105
+ > - vertex_ai-language-models/gemini-2.0-flash-thinking-exp-01-21
1106
+ > - vertex_ai-language-models/gemini-2.0-pro-exp-02-05
1107
+ > - vertex_ai-language-models/gemini-flash-experimental
1108
+ > - vertex_ai-language-models/gemini-pro
1109
+ > - vertex_ai-language-models/gemini-pro-experimental
1110
+ > - vertex_ai-vision-models/gemini-1.0-pro-vision
1111
+ > - vertex_ai-vision-models/gemini-1.0-pro-vision-001
1112
+ > - vertex_ai-vision-models/gemini-pro-vision
1113
+
1114
+ #### /modesl free
1115
+ > Invalid command: /modesl
1116
+
1117
+ #### /models :free
1118
+ > Models which match ":free":
1119
+ > - openrouter/meta-llama/llama-3-8b-instruct:free
1120
+ > - openrouter/mistralai/mistral-7b-instruct:free
1121
+
1122
+ #### /model openrouter/google/gemini-2.0-pro-exp-02-05:free
1123
+ > Warning for openrouter/google/gemini-2.0-pro-exp-02-05:free: Unknown context window size and costs, using sane defaults.
1124
+ > Warning for openrouter/google/gemini-2.0-flash-exp:free: Unknown context window size and costs, using sane defaults.
1125
+ > Did you mean one of these?
1126
+ > - openrouter/google/gemini-2.0-flash-001
1127
+ > Aider v0.76.2
1128
+ > Main model: openrouter/google/gemini-2.0-pro-exp-02-05:free with whole edit format
1129
+ > Weak model: openrouter/google/gemini-2.0-flash-exp:free
1130
+ > Git repo: .git with 3 files
1131
+ > Repo-map: using 1024 tokens, auto refresh
1132
+ >
1133
+ >
1134
+ > ^C again to exit
1135
+ >
1136
+ >
1137
+ > ^C KeyboardInterrupt
1138
+
1139
+ # aider chat started at 2025-03-11 17:41:04
1140
+
1141
+ > /home/chris/.local/bin/aider --model openrouter/google/gemini-2.0-flash-exp:free --api-key openrouter=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
1142
+ > Warning for openrouter/google/gemini-2.0-flash-exp:free: Unknown context window size and costs, using sane defaults.
1143
+ > Did you mean one of these?
1144
+ > - openrouter/google/gemini-2.0-flash-001
1145
+ > You can skip this check with --no-show-model-warnings
1146
+ > https://aider.chat/docs/llms/warnings.html
1147
+ > Open documentation url for more info? (Y)es/(N)o/(D)on't ask again [Yes]: d
1148
+ > Aider v0.76.2
1149
+ > Model: openrouter/google/gemini-2.0-flash-exp:free with whole edit format
1150
+ > Git repo: .git with 3 files
1151
+ > Repo-map: using 1024 tokens, auto refresh
1152
+
1153
+ #### /models free
1154
+ > Models which match "free":
1155
+ > - openrouter/meta-llama/llama-3-8b-instruct:free
1156
+ > - openrouter/mistralai/mistral-7b-instruct:free
1157
+
1158
+ #### /model openrouter/google/gemini-2.0-pro-exp-02-05:free
1159
+ > Warning for openrouter/google/gemini-2.0-pro-exp-02-05:free: Unknown context window size and costs, using sane defaults.
1160
+ > Warning for openrouter/google/gemini-2.0-flash-exp:free: Unknown context window size and costs, using sane defaults.
1161
+ > Did you mean one of these?
1162
+ > - openrouter/google/gemini-2.0-flash-001
1163
+ > Aider v0.76.2
1164
+ > Main model: openrouter/google/gemini-2.0-pro-exp-02-05:free with whole edit format
1165
+ > Weak model: openrouter/google/gemini-2.0-flash-exp:free
1166
+ > Git repo: .git with 3 files
1167
+ > Repo-map: using 1024 tokens, auto refresh
1168
+
1169
+ #### so i want it as a tui output, not HTML
1170
+
1171
+ Okay, I understand. You want to change the output from HTML to a TUI (Text User Interface) output.
1172
+
1173
+ Based on the file summaries you provided earlier, and this new requirement, the files most likely to need changes are:
1174
+
1175
+ * **`markdown_parser.py`**: The core logic for parsing the Markdown will likely remain the same, but the output generation will need to change significantly. Instead of generating HTML tags, it will need to produce TUI-compatible output, possibly using a library like `rich` or `textual`.
1176
+
1177
+ I'll stop here and wait for you to add these files to the chat so I can make specific suggestions.
1178
+
1179
+ > Tokens: 999 sent, 131 received.
1180
+
1181
+ #### so rich doesn't cut it because it can't do streaming in a reasonable way. It shows an elipses and it's terrible. textual looks wrong too. I just want streaming output.
1182
+
1183
+ Okay, I understand. You want plain, streaming text output, and you've ruled out `rich` and `textual`. I also understand the formatting requirements for file changes.
1184
+
1185
+ Given this, the file that needs to change is still:
1186
+
1187
+ * **`markdown_parser.py`**: This file will need to be modified to output plain text instead of HTML. The parsing logic will stay mostly the same, but the output formatting will change.
1188
+
1189
+ Here's the updated `markdown_parser.py` file:
1190
+
1191
+ markdown_parser.py
1192
+ ```
1193
+ class MarkdownParser:
1194
+ def __init__(self):
1195
+ self.in_list = False
1196
+ self.list_type = None
1197
+ self.list_level = 0
1198
+
1199
+ def parse_line(self, line):
1200
+ output = ""
1201
+ stripped_line = line.lstrip()
1202
+ indent = len(line) - len(stripped_line)
1203
+
1204
+ if stripped_line.startswith("#"):
1205
+ level = 0
1206
+ for char in stripped_line:
1207
+ if char == "#":
1208
+ level += 1
1209
+ else:
1210
+ break
1211
+ output += f"Header {level}: {stripped_line[level:].strip()}\n"
1212
+ self._close_list()
1213
+ elif stripped_line.startswith("* ") or stripped_line.startswith("- "):
1214
+ output += self._handle_list_item("unordered", indent, stripped_line[2:])
1215
+ elif stripped_line.startswith("1. "): # Simplified ordered list check
1216
+ output += self._handle_list_item("ordered", indent, stripped_line[3:])
1217
+ else:
1218
+ if self.in_list:
1219
+ self._close_list()
1220
+ output += f"{stripped_line}\n"
1221
+
1222
+ return output
1223
+
1224
+ def _handle_list_item(self, list_type, indent, content):
1225
+ output = ""
1226
+ level = indent // 4 # Assume 4 spaces per indent level
1227
+
1228
+ if not self.in_list:
1229
+ self.in_list = True
1230
+ self.list_type = list_type
1231
+ self.list_level = level
1232
+ elif self.list_type != list_type or self.list_level != level:
1233
+ # type or level changed.
1234
+ output += self._close_list()
1235
+ self.in_list = True
1236
+ self.list_type = list_type
1237
+ self.list_level = level
1238
+
1239
+ if list_type == "unordered":
1240
+ output += f"{' ' * level}* {content.strip()}\n"
1241
+ elif list_type == "ordered":
1242
+ output += f"{' ' * level}1. {content.strip()}\n" # Simplified numbering
1243
+ return output;
1244
+
1245
+ def _close_list(self):
1246
+ output = ""
1247
+ if (self.in_list):
1248
+ output = "" # No newlines needed.
1249
+ self.in_list = False
1250
+ self.list_type = None
1251
+ self.list_level = 0
1252
+ return output
1253
+
1254
+ def parse(self, markdown_text):
1255
+ output = ""
1256
+ for line in markdown_text.splitlines():
1257
+ output += self.parse_line(line)
1258
+ output += self._close_list() # close any open lists
1259
+ return output
1260
+
1261
+ def parse_stream(self, stream):
1262
+ output = ""
1263
+ for line in stream:
1264
+ output += self.parse_line(line)
1265
+ output += self._close_list() # close any open lists at the end of stream
1266
+ return output
1267
+ ```
1268
+
1269
+ > Tokens: 1.2k sent, 667 received.
1270
+ > markdown_parser.py
1271
+ > Add file to the chat? (Y)es/(N)o/(D)on't ask again [Yes]: y
1272
+
1273
+ You've already added the `markdown_parser.py` file, and I've provided the updated version for TUI output in my previous response. There are no other files that need changes based on your current requirements. If you have further modifications or new features, then we might need to add or modify other files.
1274
+
1275
+ > Tokens: 2.8k sent, 62 received.
1276
+
1277
+ #### example.py still outputs html
1278
+ > example.py
1279
+ > Add file to the chat? (Y)es/(N)o/(D)on't ask again [Yes]: y
1280
+
1281
+ You are absolutely correct! I apologize for the oversight. The `example.py` file needs to be updated to reflect the changes in `markdown_parser.py`, specifically that it now outputs plain text instead of HTML.
1282
+
1283
+ Here's the corrected `example.py`:
1284
+
1285
+ example.py
1286
+ ```
1287
+ #!/usr/bin/env python3
1288
+ """
1289
+ Example usage of the Markdown Parser
1290
+ """
1291
+
1292
+ from markdown_parser import MarkdownParser
1293
+
1294
+ def main():
1295
+ # Example markdown text
1296
+ markdown_text = """# Heading 1
1297
+ ## Heading 2
1298
+ ### Heading 3
1299
+ #### Heading 4
1300
+ ##### Heading 5
1301
+ ###### Heading 6
1302
+
1303
+ - Unordered list item 1
1304
+ - Unordered list item 2
1305
+ - Nested unordered item 2.1
1306
+ - Nested unordered item 2.2
1307
+ - Deeply nested item 2.2.1
1308
+ - Unordered list item 3
1309
+
1310
+ 1. Ordered list item 1
1311
+ 2. Ordered list item 2
1312
+ 1. Nested ordered item 2.1
1313
+ 2. Nested ordered item 2.2
1314
+ 3. Ordered list item 3
1315
+
1316
+ - Mixed list example
1317
+ 1. Nested ordered in unordered
1318
+ 2. Another nested item
1319
+ - Back to unordered
1320
+ """
1321
+
1322
+ # Parse the complete text
1323
+ parser = MarkdownParser()
1324
+ text = parser.parse(markdown_text)
1325
+ print("Complete parsing result:")
1326
+ print(text)
1327
+ print("\n" + "-" * 50 + "\n")
1328
+
1329
+ # Demonstrate streaming parsing
1330
+ print("Streaming parsing result:")
1331
+ stream_parser = MarkdownParser()
1332
+ for line in markdown_text.split('\n'):
1333
+ text_fragment = stream_parser.parse_line(line)
1334
+ if text_fragment:
1335
+ print(text_fragment, end="") # Print without extra newline
1336
+
1337
+ # Don't forget to close any open tags
1338
+ closing_text = stream_parser._close_list()
1339
+ if closing_text:
1340
+ print(closing_text, end="")
1341
+
1342
+ if __name__ == "__main__":
1343
+ main()
1344
+ ```
1345
+
1346
+ Key changes in `example.py`:
1347
+
1348
+ * Changed `html` variable names to `text`.
1349
+ * Removed the `print(f"Line: {line}")` line in the streaming section, as it's redundant with the direct output.
1350
+ * Added `end=""` to the `print` calls in the streaming section to prevent extra newlines, ensuring correct output formatting since the parser now includes newlines.
1351
+ * The `_close_list` method is now called and its output printed.
1352
+
1353
+ I have now provided the complete, corrected `example.py` file in the required format.
1354
+
1355
+ > Tokens: 3.3k sent, 530 received.
1356
+ > Committing example.py before applying edits.
1357
+ >
1358
+ >
1359
+ > ^C again to exit
1360
+
1361
+ # aider chat started at 2025-03-11 18:42:24
1362
+
1363
+ > /home/chris/.local/bin/aider --model openrouter/google/gemini-2.0-flash-exp:free --api-key openrouter=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
1364
+ > Warning for openrouter/google/gemini-2.0-flash-exp:free: Unknown context window size and costs, using sane defaults.
1365
+ > Did you mean one of these?
1366
+ > - openrouter/google/gemini-2.0-flash-001
1367
+ > You can skip this check with --no-show-model-warnings
1368
+ > https://aider.chat/docs/llms/warnings.html
1369
+ > Open documentation url for more info? (Y)es/(N)o/(D)on't ask again [Yes]: d
1370
+ > Aider v0.76.2
1371
+ > Model: openrouter/google/gemini-2.0-flash-exp:free with whole edit format
1372
+ > Git repo: .git with 3 files
1373
+ > Repo-map: using 1024 tokens, auto refresh
1374
+
1375
+ #### look at tui_parser.py and example.md ... write a test to make sure that the example.md works correctly
1376
+
1377
+ Okay, I need to see the contents of `tui_parser.py` and `example.md` first. Please add them to the chat.
1378
+
1379
+ > Tokens: 1.0k sent, 29 received.