streamdown 0.13.0__tar.gz → 0.15.0__tar.gz

{streamdown-0.13.0 → streamdown-0.15.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: streamdown
-Version: 0.13.0
+Version: 0.15.0
 Summary: A streaming markdown renderer for modern terminals with syntax highlighting
 Project-URL: Homepage, https://github.com/kristopolous/Streamdown
 Project-URL: Bug Tracker, https://github.com/kristopolous/Streamdown/issues
@@ -23,6 +23,7 @@ Requires-Python: >=3.8
 Requires-Dist: appdirs
 Requires-Dist: pygments
 Requires-Dist: pylatexenc
+Requires-Dist: term-image
 Requires-Dist: toml
 Description-Content-Type: text/markdown
 
@@ -30,26 +31,30 @@ Description-Content-Type: text/markdown
 
 [![PyPI version](https://badge.fury.io/py/streamdown.svg)](https://badge.fury.io/py/streamdown)
 
-I needed a streaming Markdown renderer and I couldn't find one. So here we go. From the ground up. It's a bad idea but it has to be done.
+Streamdown is the streaming markdown renderer for the terminal that rocks.
+This will work with [simonw's llm](https://github.com/simonw/llm). You even get full readline and keyboard navigation support.
 
-[sd demo](https://github.com/user-attachments/assets/48dba6fa-2282-4be9-8087-a2ad8e7c7d12)
+It's fully streaming and does not block
+![Streamdown is Amazing](https://github.com/user-attachments/assets/268cb340-78cc-4df0-a773-c5ac95eceeeb)
 
-This will work with [simonw's llm](https://github.com/simonw/llm) unlike with [richify.py](https://github.com/gianlucatruda/richify) which rerenders the whole buffer and blocks with an elipses or [glow](https://github.com/charmbracelet/glow) which buffers everything, this streams and does exactly what it says.
+### Provides clean copyable code for long code lines
+You may have noticed that other, *inferior* renderers will inject line breaks when copying code that wraps around. We're better and now, you can be as well.
+![Handle That Mandle](https://github.com/user-attachments/assets/a27aa70c-f691-4796-84f0-c2eb18c7de23)
 
-## Some Features
+### Supports images
+Here's kitty and alacritty. Try to do that in glow...
+![doggie](https://github.com/user-attachments/assets/9a392929-b6c2-4204-b257-e09305acb7af)
 
-### Provides clean copyable code for long code blocks and short terminals. 
-![copyable](https://github.com/user-attachments/assets/4a3539c5-b5d1-4d6a-8bce-032724d8909d)
-
-### Does OSC 8 links for modern terminals.
+### Does OSC 8 links for modern terminals (and optionally OSC 52 for clipboard)
 [links.webm](https://github.com/user-attachments/assets/a5f71791-7c58-4183-ad3b-309f470c08a3)
 
-### Doesn't consume characters like _ and * as style when they are in `blocks like this` because `_they_can_be_varaiables_`
-![dunder](https://github.com/user-attachments/assets/d41d7fec-6dec-4387-b53d-f2098f269a5e)
-
 ### Tables are carefully supported
 ![table](https://github.com/user-attachments/assets/dbe3d13e-6bac-4f45-bf30-f1857ed98898)
 
+As well as everything else...
+
+![dunder](https://github.com/user-attachments/assets/d41d7fec-6dec-4387-b53d-f2098f269a5e)
+
 ### Colors are highly (and quickly) configurable for people who care a lot, or just a little.
 ![configurable](https://github.com/user-attachments/assets/04b36749-4bb8-4c14-9758-84eb6e19b704)
 
@@ -80,7 +85,7 @@ Defines the base Hue (H), Saturation (S), and Value (V) from which all other pal
 *   `Width` (integer, default: `0`): Along with the `Margin`, `Width` specifies the base width of the content, which when set to 0, means use the terminal width. See [#6](https://github.com/kristopolous/Streamdown/issues/6) for more details
 *   `PrettyPad` (boolean, default: `false`): Uses a unicode vertical pad trick to add a half height background to code blocks. This makes copy/paste have artifacts. See [#2](https://github.com/kristopolous/Streamdown/issues/2). I like it on. But that's just me
 *   `ListIndent` (integer, default: `2`): This is the recursive indent for the list styles.
-*   `Syntax` (string, default `monokai`): This the syntax [highlighting theme which come via pygments](https://pygments.org/styles/).
+*   `Syntax` (string, default `monokai`): This is the syntax [highlighting theme which come via pygments](https://pygments.org/styles/).
 
 Example:
 ```toml
@@ -139,10 +144,8 @@ Do this
 
     $ ./streamdown/sd.py tests/*md
 
-Certainly room for improvement and I'll probably continue to make them
-
 ## Install from source
-At least one of these should work, hopefully
+After the git clone least one of these should work, hopefully. it's using the modern uv pip tool.
 
     $ pipx install -e .
     $ pip install -e .

{streamdown-0.13.0 → streamdown-0.15.0}/README.md

@@ -2,26 +2,30 @@
 
 [![PyPI version](https://badge.fury.io/py/streamdown.svg)](https://badge.fury.io/py/streamdown)
 
-I needed a streaming Markdown renderer and I couldn't find one. So here we go. From the ground up. It's a bad idea but it has to be done.
+Streamdown is the streaming markdown renderer for the terminal that rocks.
+This will work with [simonw's llm](https://github.com/simonw/llm). You even get full readline and keyboard navigation support.
 
-[sd demo](https://github.com/user-attachments/assets/48dba6fa-2282-4be9-8087-a2ad8e7c7d12)
+It's fully streaming and does not block
+![Streamdown is Amazing](https://github.com/user-attachments/assets/268cb340-78cc-4df0-a773-c5ac95eceeeb)
 
-This will work with [simonw's llm](https://github.com/simonw/llm) unlike with [richify.py](https://github.com/gianlucatruda/richify) which rerenders the whole buffer and blocks with an elipses or [glow](https://github.com/charmbracelet/glow) which buffers everything, this streams and does exactly what it says.
+### Provides clean copyable code for long code lines
+You may have noticed that other, *inferior* renderers will inject line breaks when copying code that wraps around. We're better and now, you can be as well.
+![Handle That Mandle](https://github.com/user-attachments/assets/a27aa70c-f691-4796-84f0-c2eb18c7de23)
 
-## Some Features
+### Supports images
+Here's kitty and alacritty. Try to do that in glow...
+![doggie](https://github.com/user-attachments/assets/9a392929-b6c2-4204-b257-e09305acb7af)
 
-### Provides clean copyable code for long code blocks and short terminals. 
-![copyable](https://github.com/user-attachments/assets/4a3539c5-b5d1-4d6a-8bce-032724d8909d)
-
-### Does OSC 8 links for modern terminals.
+### Does OSC 8 links for modern terminals (and optionally OSC 52 for clipboard)
 [links.webm](https://github.com/user-attachments/assets/a5f71791-7c58-4183-ad3b-309f470c08a3)
 
-### Doesn't consume characters like _ and * as style when they are in `blocks like this` because `_they_can_be_varaiables_`
-![dunder](https://github.com/user-attachments/assets/d41d7fec-6dec-4387-b53d-f2098f269a5e)
-
 ### Tables are carefully supported
 ![table](https://github.com/user-attachments/assets/dbe3d13e-6bac-4f45-bf30-f1857ed98898)
 
+As well as everything else...
+
+![dunder](https://github.com/user-attachments/assets/d41d7fec-6dec-4387-b53d-f2098f269a5e)
+
 ### Colors are highly (and quickly) configurable for people who care a lot, or just a little.
 ![configurable](https://github.com/user-attachments/assets/04b36749-4bb8-4c14-9758-84eb6e19b704)
 
@@ -52,7 +56,7 @@ Defines the base Hue (H), Saturation (S), and Value (V) from which all other pal
 *   `Width` (integer, default: `0`): Along with the `Margin`, `Width` specifies the base width of the content, which when set to 0, means use the terminal width. See [#6](https://github.com/kristopolous/Streamdown/issues/6) for more details
 *   `PrettyPad` (boolean, default: `false`): Uses a unicode vertical pad trick to add a half height background to code blocks. This makes copy/paste have artifacts. See [#2](https://github.com/kristopolous/Streamdown/issues/2). I like it on. But that's just me
 *   `ListIndent` (integer, default: `2`): This is the recursive indent for the list styles.
-*   `Syntax` (string, default `monokai`): This the syntax [highlighting theme which come via pygments](https://pygments.org/styles/).
+*   `Syntax` (string, default `monokai`): This is the syntax [highlighting theme which come via pygments](https://pygments.org/styles/).
 
 Example:
 ```toml
@@ -111,10 +115,8 @@ Do this
 
     $ ./streamdown/sd.py tests/*md
 
-Certainly room for improvement and I'll probably continue to make them
-
 ## Install from source
-At least one of these should work, hopefully
+After the git clone least one of these should work, hopefully. it's using the modern uv pip tool.
 
     $ pipx install -e .
     $ pip install -e .

{streamdown-0.13.0 → streamdown-0.15.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "streamdown"
-version = "0.13.0"
+version = "0.15.0"
 description = "A streaming markdown renderer for modern terminals with syntax highlighting"
 readme = "README.md"
 requires-python = ">=3.8"
@@ -31,6 +31,7 @@ dependencies = [
     "appdirs",
     "toml",
     "pylatexenc",
+    'term-image'
 ]
 
 [project.urls]

streamdown-0.15.0/pythonvgo.md

@@ -0,0 +1,187 @@
+🫣# Python vs. Go: Let's get ready to rumble!
+![](python-go.png)
+🫣
+Python and Go (Golang) are both popular programming languages, but they cater to different needs and philosophies. Here's a detailed comparison:
+🫣
+
+**Python:** 🫣
+
+* **Strengths:** 🫣
+    * **Readability:**  Known for its clean and easy-to-understand syntax, emphasizing code readability.  🫣
+    * **Large Ecosystem:** Vast library support for data science, machine learning, web development, scripting, and more.  Packages like NumPy, Pandas, Django, and Flask make complex tasks simpler.  🫣
+    * **Rapid Development:** Its dyna🫣mic typing and interpreted nature allow for quick prototyping and development.
+    * **Dyn🫣amic Typing:**  The interpreter infers data types at runtime, simplifying code (but potentially hiding errors).  🫣
+    * **Versatility:**  Can be used for a wide range of applications – web backends, data analysis, machine learning, automation, scripting, testing, and more.  🫣
+    * **Community:** Huge and active community providing ample support, tutorials, and resources.  🫣
+* **Weaknesses:**
+    * **Performance:** Being an interpreted language, Python is generally slower than compiled languages like Go.  🫣
+    * **Global Interpreter Lock (GIL):**  Limits true multi-threading in CPU-bound operations.  🫣
+    * **Error Handling:**  Runtime errors can be more common because of dynamic typing.  🫣
+    * **Memory Consumption:**  Typically has higher memory overhead compared to Go due to its object model and dynamic typing.
+
+**Go:**
+
+* **Strengths:**
+    * **Performance:** Compiled language that produces efficient, native machine code. Generally much faster than Python.
+    * **Concurrency:**  Built-in support for concurrency through Goroutines (lightweight threads) and Channels, making it easy to write concurrent and parallel programs. 
+🫣    * **Static Typing:** Helps catch errors at compile-time.
+    * **Garbage Collection:** Automatic memory management reduces the risk of memory leaks.
+    * **Simplicity:** Designed 🫣to be a simple language with a relatively small number of keywords.  Focuses on getting things done efficiently.
+    * **Scalability:**  Well-suited for building scalable network services and distributed systems.
+    * **Tooling:** E🫣xcellent built-in tooling for testing, formatting, and dependency management.
+* **Weaknesses:**
+    * **Learning Curve:**  Can be slightly steeper than Python initially, particularly regarding concurre🫣ncy concepts.
+    * **Error Handling:** Explicit error handling (returning errors as values) can lead to verbose code.  (While necessary, it's less concise than Python's `try...except`)
+    * **Generics (Relatively New):**  Generics were only added in Go 1.18 (released in 2022).  Prior to that, code reusability for different types was more challenging.
+    * **Smalle🫣r Ecosystem:**  While rapidly growing, Go’s ecosystem is still smaller than Python's, particularly in specialized areas like data science.
+
+**Use Cases:**
+
+* **Python:** Data science, machine learning, web development (Django, Flask), scripting, automation, prototyping, and educational purposes.
+* **Go:** Cloud infrastructure (Docker, Kubernetes), network services, distributed systems, command-line tools, DevOps, and high-performance backend services.
+
+
+
+## Code Examples:
+
+**1. Hello World:**
+
+**Python:**
+```🫣python
+print("Hello, World!")
+```
+
+**Go:**
+```go
+package main
+
+import "fmt"
+
+func main() {
+	fmt.Println("Hello, World!")
+}
+```
+
+**2.  Simple Web Server:**
+
+**Python (using Flask🫣):**
+```python
+from flask import Flask
+
+app = Flask(__name__)
+
+@app.route("/")
+def hello_world():
+    return "<p>Hello, World!</p>"
+
+if __name__ == '__main__':
+    app.run(debug=True)
+```
+
+**Go (using net/http):**
+```go
+package main
+
+import (
+	"fmt"
+	"net/http"
+)
+
+func handler(w http.ResponseWriter, r *http.Request) {
+	fmt.Fprintf(w, "Hello, World!")
+}
+
+func main() {
+	http.HandleFunc("/", handler)
+	fmt.Println("Server listening on port 8080")
+	http.ListenAndServe(":8080", nil)
+}
+```
+
+**3. Concurrent Processing (Simple):**
+
+**Python (using threading - limited by GIL):**
+
+```python
+import threading
+
+def process_task(task_id):
+    print(f"Task {task_id} started")
+    # Simulate some work
+    import time
+    time.sleep(2)
+    print(f"Task {task_id} completed")
+
+tasks = [1, 2, 3]
+threads = []
+
+for task in tasks:
+    thread = threading.Thread(🫣target=process_task, args=(task,))
+    threads.append(thread)
+    thread.start()
+
+for thread in threads:
+    thread.join()
+
+print("All tasks comple🫣ted.")
+```
+
+**Go (using Goroutines and Channels):**
+
+```go
+package main
+
+import (
+	"fmt"
+	"sync"
+)
+
+func processTask(taskID int, wg *sync.WaitGroup) {
+	defer wg.Done() // Decrement the WaitGroup counter when the goroutin🫣e completes.
+	fmt.Printf("Task %d started🫣\n", taskID)
+	// Simulate some work
+	//time.Sleep(2 * time🫣.Second) // Go uses time.Second, etc.
+	fmt.Printf("Task %d completed\n", taskID)
+}
+
+func main() {
+	var wg sync.WaitGroup
+	tasks := []int{1, 2, 3}
+
+	for _, task := range tasks {
+		wg.Add(1) // Increment the WaitGroup counter for each🫣 goroutine.
+		go processTask(task, &wg)
+	}
+
+	wg.Wait()
+	fmt.Prin🫣tln("All tasks completed.")
+}
+```
+
+
+## Comparative Table o🫣f Features:
+
+| Feature             | Python                           | Go (Golang)                  |
+|----------------------|----------------------------------|-------------------------------|
+| **Typi🫣ng**          | Dynamic, strong     🫣             | Static, strong                |
+| **Compilation**     | Interpreted                      | Compiled                      |
+| **Perfo🫣rmance**      | Generally slower                | Generally faster              |
+| **Concurrency**       | Through threads (GIL limited)   | Goroutines & Chan🫣nels 🫣(built-in) |
+| **Error Handling**   | Exceptions (try-except)           | Explicit error val🫣ues          |
+| **Memory Management**| Automatic (Garbage Collection)   | Automatic (Garbage Collection) |
+| **Syntax**           | Readable, concise               | Simple, explicit               |
+| **Ecosystem**🫣        | Huge, mature                   | Growing, focused               |
+| **Learning Curve**    | Easier                           | Moderate                      |
+| **Generics**          | Present                          | Added in 1.18 (relatively new)|
+| **Typical Use Cases**| Data science, web dev, scripting | Cloud, networking, system programming |
+| **Community**        | Very large, active               | Growing, dedicated            |
+| **Object Orientation**| Full support                    | Structs with methods, interfaces|
+
+
+
+**In Summary:**
+
+* **Choose Python 🫣if:** You need rapid development, a large ecosystem of libraries, or are focused on data science, machine learning, or scripting.  Readability and ease of use are priorities.
+
+* **Choose Go if:** You need high performance, concurrency, scalability, and are building infrastructure, network services, or command-line tools.  Deterministic error handling and a simple, efficient language are key.
+🫣

streamdown-0.15.0/streamdown/scrape/file_0.py

@@ -0,0 +1,22 @@
+def fizzbuzz(n):
+for i in range(1, n + 1):
+if i % 3 == 0 and i % 5 == 0:
+print("FizzBuzz")
+elif i % 3 == 0:
+print("Fizz")
+elif i % 5 == 0:
+print("Buzz")
+else:
+print(i)
+
+# Example usage: Print FizzBuzz up to 100 Example usage: Print FizzBuzz up to 100 Example usage: Print FizzBuzz up to 100 Example usage: Print FizzBuzz up to 100
+fizzbuzz(100)
+
+#  Example usage: different range:
+fizzbuzz(20)
+
+#Example usage:  one line output (list comprehension)
+def fizzbuzz_oneline(n):
+print(["FizzBuzz" if i%3==0 and i%5==0 else "Fizz" if i%3==0 else "Buzz" if i%5==0 else i for i in range(1,n+1)])
+
+fizzbuzz_oneline(30)

streamdown-0.15.0/streamdown/scrape/file_1.js

@@ -0,0 +1,27 @@
+function fizzBuzz(n) {
+for (let i = 1; i <= n; i++) {
+if (i % 3 === 0 && i % 5 === 0) {
+console.log("FizzBuzz");
+} else if (i % 3 === 0) {
+console.log("Fizz");
+} else if (i % 5 === 0) {
+console.log("Buzz");
+} else {
+console.log(i);
+}
+}
+}
+
+// Example usage:
+fizzBuzz(100);
+
+// Example usage: different range
+fizzBuzz(25);
+
+// Example one-line output. (arrow function & ternary operator)
+const fizzBuzzOneLine = n => {
+for (let i = 1; i <= n; i++) {
+console.log((i % 3 === 0 ? (i % 5 === 0 ? "FizzBuzz" : "Fizz") : (i % 5 === 0 ? "Buzz" : i)));
+}
+};
+fizzBuzzOneLine(30);

streamdown-0.15.0/streamdown/scrape/file_2.cpp

@@ -0,0 +1,23 @@
+#include <iostream>
+
+void fizzBuzz(int n) {
+for (int i = 1; i <= n; i++) {
+if (i % 3 == 0 && i % 5 == 0) {
+std::cout << "FizzBuzz" << std::endl;
+} else if (i % 3 == 0) {
+std::cout << "Fizz" << std::endl;
+} else if (i % 5 == 0) {
+std::cout << "Buzz" << std::endl;
+} else {
+std::cout << i << std::endl;
+}
+}
+}
+
+int main() {
+fizzBuzz(100);
+
+// Example usage: different range
+fizzBuzz(35);
+return 0;
+}

{streamdown-0.13.0 → streamdown-0.15.0}/streamdown/sd.py

@@ -5,6 +5,7 @@
 #     "pygments",
 #     "pylatexenc",
 #     "appdirs",
+#     "term-image",
 #     "toml"
 # ]
 # ///
@@ -23,7 +24,9 @@ import colorsys
 import base64
 import importlib
 from io import BytesIO
+from term_image.image import from_file, from_url
 import pygments.util
+from functools import reduce
 from argparse import ArgumentParser
 from pygments import highlight
 from pygments.lexers import get_lexer_by_name
@@ -52,7 +55,7 @@ Dark    = { H = 1.00, S = 1.50, V = 0.25 }
 Mid     = { H = 1.00, S = 1.00, V = 0.50 }
 Symbol  = { H = 1.00, S = 1.00, V = 1.50 }
 Head    = { H = 1.00, S = 2.00, V = 1.50 }
-Grey    = { H = 1.00, S = 0.12, V = 1.25 }
+Grey    = { H = 1.00, S = 0.25, V = 1.37 }
 Bright  = { H = 1.00, S = 2.00, V = 2.00 }
 Syntax  = "monokai"
 """
@@ -80,16 +83,17 @@ BGRESET = "\033[49m"
 BOLD      = ["\033[1m", "\033[22m"]
 UNDERLINE = ["\033[4m", "\033[24m"]
 ITALIC    = ["\033[3m", "\033[23m"]
+STRIKEOUT = ["\033[9m", "\033[29m"]
+SUPER     = [ 0x2070, 0x00B9, 0x00B2, 0x00B3, 0x2074, 0x2075, 0x2076, 0x2077, 0x2078, 0x2079 ]
 
 ESCAPE = r"\033\[[0-9;]*[mK]"
 ANSIESCAPE = r'\033(?:\[[0-9;?]*[a-zA-Z]|][0-9]*;;.*?\\|\\)'
-#r"\033(\[[0-9;]*[mK]|][0-9]*;;.*?\\|\\)"
 KEYCODE_RE = re.compile(r'\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])')
 
 visible = lambda x: re.sub(ANSIESCAPE, "", x)
 visible_length = lambda x: len(visible(x))
-
 extract_ansi_codes = lambda text: re.findall(ESCAPE, text)
+remove_ansi = lambda line, codeList: reduce(lambda line, code: line.replace(code, ''), codeList, line)
 
 def debug_write(text):
     if state.Logging:
@@ -131,7 +135,10 @@ class ParseState:
         self.Clipboard = _features.get("Clipboard")
         self.Logging = _features.get("Logging")
         self.Timeout = _features.get("Timeout")
+
         self.WidthArg = None
+        self.WidthFull = None
+        self.WidthWrap = False
 
         # If the entire block is indented this will
         # tell us what that is
@@ -159,24 +166,27 @@ class ParseState:
         self.in_italic = False
         self.in_table = False # (Code.[Header|Body] | False)
         self.in_underline = False
+        self.in_strikeout = False
         self.block_depth = 0
 
         self.exec_sub = None
         self.exec_master = None
         self.exec_slave = None
         self.exec_kb = 0
-        self.exec_israw = False
 
         self.exit = 0
         self.where_from = None
 
     def current(self):
-        state = { 'code': self.in_code, 'bold': self.in_bold, 'italic': self.in_italic, 'underline': self.in_underline }
+        state = { 'inline': self.inline_code, 'code': self.in_code, 'bold': self.in_bold, 'italic': self.in_italic, 'underline': self.in_underline }
         state['none'] = all(item is False for item in state.values())
         return state
 
+    def reset_inline(self):
+        self.inline_code = self.in_bold = self.in_italic = self.in_underline = False
+
     def space_left(self):
-        return (Style.MarginSpaces if len(self.current_line) == 0 else "") + (Style.Blockquote * self.block_depth)
+        return Style.MarginSpaces + (Style.Blockquote * self.block_depth) if len(self.current_line) == 0 else "" 
 
 state = ParseState()
 
@@ -233,17 +243,20 @@ def emit_h(level, text):
     text = line_format(text)
     spaces_to_center = ((state.Width - visible_length(text)) / 2)
     if level == 1:      #
-        return f"\n{Style.MarginSpaces}{BOLD[0]}{' ' * math.floor(spaces_to_center)}{text}{' ' * math.ceil(spaces_to_center)}{BOLD[1]}\n"
+        return f"\n{state.space_left()}{BOLD[0]}{' ' * math.floor(spaces_to_center)}{text}{' ' * math.ceil(spaces_to_center)}{BOLD[1]}\n"
     elif level == 2:    ##
-        return f"\n{Style.MarginSpaces}{BOLD[0]}{FG}{Style.Bright}{' ' * math.floor(spaces_to_center)}{text}{' ' * math.ceil(spaces_to_center)}{RESET}\n\n"
+        return f"\n{state.space_left()}{BOLD[0]}{FG}{Style.Bright}{' ' * math.floor(spaces_to_center)}{text}{' ' * math.ceil(spaces_to_center)}{RESET}\n\n"
     elif level == 3:    ###
-        return f"{Style.MarginSpaces}{FG}{Style.Head}{BOLD[0]}{text}{RESET}"
+        return f"{state.space_left()}{FG}{Style.Head}{BOLD[0]}{text}{RESET}"
     elif level == 4:    ####
-        return f"{Style.MarginSpaces}{FG}{Style.Symbol}{text}{RESET}"
+        return f"{state.space_left()}{FG}{Style.Symbol}{text}{RESET}"
     else:  # level 5 or 6
-        return f"{Style.MarginSpaces}{text}{RESET}"
+        return f"{state.space_left()}{text}{RESET}"
 
 def code_wrap(text_in):
+    if state.WidthWrap and len(text_in) > state.WidthFull:
+        return (0, [text_in])
+
     # get the indentation of the first line
     indent = len(text_in) - len(text_in.lstrip())
     text = text_in.lstrip()
@@ -259,6 +272,7 @@ def code_wrap(text_in):
 
     return (indent, res)
 
+
 # This marvelously obscure code "compacts" long lines of repetitive ANSI format strings by
 # removing duplicates. Here's how it works
 def ansi_collapse(codelist, inp):
@@ -337,8 +351,20 @@ def text_wrap(text, width = -1, indent = 0, first_line_prefix="", subsequent_lin
     return lines
 
 def line_format(line):
-    def not_text(token):
-        return not token or len(token.rstrip()) != len(token)
+    not_text = lambda token: not token or len(token.rstrip()) != len(token)
+    footnotes = lambda match: ''.join([chr(SUPER[int(i)]) for i in match.group(1)])
+
+    def process_images(match):
+        url = match.group(2)
+        try:
+            if re.match(r"https://", url.lower()):
+                image = from_url(url)
+            else: 
+                image = from_file(url)
+            image.height = 20
+            print(f"{image:|.-1#}")
+        except:
+            return match.group(2)
 
     # Apply OSC 8 hyperlink formatting after other formatting
     def process_links(match):
@@ -346,8 +372,11 @@ def line_format(line):
         url = match.group(2)
         return f'\033]8;;{url}\033\\{Style.Link}{description}{UNDERLINE[1]}\033]8;;\033\\{FGRESET}'
 
+    line = re.sub(r"\!\[([^\]]*)\]\(([^\)]+)\)", process_images, line)
     line = re.sub(r"\[([^\]]+)\]\(([^\)]+)\)", process_links, line)
-    tokenList = re.finditer(r"((\*\*|\*|_|`)|[^_*`]+)", line)
+    line = re.sub(r"\[\^(\d+)\]:?", footnotes, line)
+
+    tokenList = re.finditer(r"((~~|\*\*_|_\*\*|\*{1,3}|_{1,3}|`+)|[^~_*`]+)", line)
     result = ""
 
     for match in tokenList:
@@ -355,8 +384,13 @@ def line_format(line):
         next_token = line[match.end()] if match.end() < len(line) else ""
         prev_token = line[match.start()-1] if match.start() > 0 else ""
 
-        if token == "`":
-            state.inline_code = not state.inline_code
+        # This trick makes sure that things like `` ` `` render right.
+        if "`" in token and (not state.inline_code or state.inline_code == token):
+            if state.inline_code:
+                state.inline_code = False
+            else:
+                state.inline_code = token
+
             if state.inline_code:
                 result += f'{BG}{Style.Mid}'
             else:
@@ -367,7 +401,17 @@ def line_format(line):
         elif state.inline_code:
             result += token
 
-        elif token == "**" and (state.in_bold or not_text(prev_token)):
+        elif token == '~~' and (state.in_strikeout or not_text(prev_token)):
+            state.in_strikeout = not state.in_strikeout
+            result += STRIKEOUT[0] if state.in_strikeout else STRIKEOUT[1]
+
+        elif token in ['**_','_**','___','***'] and (state.in_bold or not_text(prev_token)):
+            state.in_bold = not state.in_bold
+            result += BOLD[0] if state.in_bold else BOLD[1]
+            state.in_italic = not state.in_italic
+            result += ITALIC[0] if state.in_italic else ITALIC[1]
+
+        elif (token == '__' or token == "**") and (state.in_bold or not_text(prev_token)):
             state.in_bold = not state.in_bold
             result += BOLD[0] if state.in_bold else BOLD[1]
  
@@ -380,7 +424,7 @@ def line_format(line):
             else:
                 result += token
 
-        elif token == "_" and (state.in_underline or not_text(prev_token)):
+        elif token == "_" and (state.in_underline or (not_text(prev_token) and next_token.isalnum())):
             state.in_underline = not state.in_underline
             result += UNDERLINE[0] if state.in_underline else UNDERLINE[1]
         else:
@@ -422,8 +466,6 @@ def parse(stream):
 
                 if len(ready_in) == 0:
                     TimeoutIx += 1
-                
-
 
             elif stream.fileno() in ready_in: 
                 byte = os.read(stream.fileno(), 1)
@@ -462,13 +504,30 @@ def parse(stream):
         # Run through the plugins first
         res = latex.Plugin(line, state, Style)
         if res is True:
-          # This means everything was consumed by our plugin and 
-          # we should continue
-          continue
-        elif res is not None:
-          for row in res:
-            yield row
+            # This means everything was consumed by our plugin and 
+            # we should continue
             continue
+        elif res is not None:
+            for row in res:
+                yield row
+                continue
+        
+        # running this here avoids stray |
+        block_match = re.match(r"^\s*((>\s*)+|<.?think>)", line)
+        if not state.in_code and block_match:
+            if block_match.group(1) == '</think>':
+                state.block_depth = 0
+                yield RESET
+            elif block_match.group(1) == '<think>':
+                state.block_depth = 1
+            else:
+                state.block_depth = block_match.group(0).count('>')
+                # we also need to consume those tokens
+                line = line[len(block_match.group(0)):]
+        else:
+            if state.block_depth > 0:
+                line = FGRESET + line
+                state.block_depth = 0
 
         # --- Collapse Multiple Empty Lines if not in code blocks ---
         if not state.in_code:
@@ -483,7 +542,7 @@ def parse(stream):
             else:
                 last_line_empty_cache = state.last_line_empty
                 state.last_line_empty = False
-
+        
         # This is to reset our top-level line-based systems
         # \n buffer
         if not state.in_list and len(state.ordered_list_numbers) > 0:
@@ -506,28 +565,11 @@ def parse(stream):
         if state.in_table and not state.in_code and not re.match(r"^\s*\|.+\|\s*$", line):
             state.in_table = False
 
-        block_match = re.match(r"^((> )*|<.?think>)", line)
-        if block_match:
-            if block_match.group(1) == '</think>':
-                state.block_depth = 0
-                yield(RESET)
-            elif block_match.group(1) == '<think>':
-                state.block_depth = 1
-            else:
-                state.block_depth = int(len(block_match.group(0)) / 2)
-                # we also need to consume those tokens
-                line = line[state.block_depth * 2:]
-        else:
-            if state.block_depth > 0:
-                yield RESET
-                state.block_depth = 0
-
         #
         # <code><pre>
         #
-        # This needs to be first
         if not state.in_code:
-            code_match = re.match(r"\s*```\s*([^\s]+|$)", line)
+            code_match = re.match(r"\s*```\s*([^\s]+|$)$", line)
             if code_match:
                 state.in_code = Code.Backtick
                 state.code_language = code_match.group(1) or 'Bash'
@@ -563,6 +605,7 @@ def parse(stream):
                         try:
                             ext = get_lexer_by_name(state.code_language).filenames[0].split('.')[-1]
                         except:
+                            logging.warning(f"Can't find canonical extension for {state.code_language}")
                             pass
 
                         open(os.path.join(state.scrape, f"file_{state.scrape_ix}.{ext}"), 'w').write(state.code_buffer)
@@ -584,7 +627,8 @@ def parse(stream):
 
 
                     if code_type == Code.Backtick:
-                            continue
+                        state.code_indent = len(line) - len(line.lstrip())
+                        continue
                     else:
                         # otherwise we don't want to consume
                         # nor do we want to be here.
@@ -600,11 +644,6 @@ def parse(stream):
                         custom_style = get_style_by_name("default")
 
                     formatter = Terminal256Formatter(style=custom_style)
-                    for i, char in enumerate(line):
-                        if char == " ":
-                            state.code_indent += 1
-                        else:
-                            break
                     line = line[state.code_indent :]
 
                 elif line.startswith(" " * state.code_indent):
@@ -628,6 +667,10 @@ def parse(stream):
                     # then naively search back until our visible_lengths() match. This is not fast and there's certainly smarter
                     # ways of doing it but this thing is way trickery than you think
                     highlighted_code = highlight(state.code_buffer + tline, lexer, formatter)
+
+                    # Sometimes the highlighter will do things like a full reset or a background reset.
+                    # This is not what we want
+                    highlighted_code = re.sub(r"\033\[39(;00|)m", '', highlighted_code)
     
                     # Since we are streaming we ignore the resets and newlines at the end
                     if highlighted_code.endswith(FGRESET + "\n"):
@@ -652,7 +695,7 @@ def parse(stream):
 
                     code_line = ' ' * indent + this_batch.strip()
 
-                    margin = state.WidthFull - visible_length(code_line)
+                    margin = state.WidthFull - visible_length(code_line) % state.WidthFull
                     yield f"{Style.Codebg}{code_line}{' ' * max(0, margin)}{BGRESET}"  
                 continue
             except Goto:
@@ -714,18 +757,18 @@ def parse(stream):
             if list_type == "number":
                 state.ordered_list_numbers[-1] += 1
 
-            indent = len(state.list_item_stack) * 2
+            indent = (len(state.list_item_stack) - 1) * 2
 
             wrap_width = state.Width - indent - (2 * Style.ListIndent) 
 
             bullet = '•'
             if list_type == "number":
                 list_number = int(max(state.ordered_list_numbers[-1], float(list_item_match.group(2))))
-                bullet = f"{list_number}"
+                bullet = str(list_number)
             
             wrapped_lineList = text_wrap(content, wrap_width, Style.ListIndent,
-                first_line_prefix      = f"{(' ' * (indent - len(bullet)))}{FG}{Style.Symbol}{bullet}{RESET} ",
-                subsequent_line_prefix = " " * (indent - 1)
+                first_line_prefix      = f"{(' ' * (indent ))}{FG}{Style.Symbol}{bullet}{RESET} ",
+                subsequent_line_prefix = " " * (indent)
             )
             for wrapped_line in wrapped_lineList:
                 yield f"{state.space_left()}{wrapped_line}\n"
@@ -742,7 +785,7 @@ def parse(stream):
         #
         # <hr>
         #
-        hr_match = re.match(r"^[\s]*([-=_]){3,}[\s]*$", line)
+        hr_match = re.match(r"^[\s]*([-\*=_]){3,}[\s]*$", line)
         if hr_match:
             if state.last_line_empty or last_line_empty_cache:
                 # print a horizontal rule using a unicode midline 
@@ -764,12 +807,6 @@ def parse(stream):
             for wrapped_line in wrapped_lines:
                 yield f"{state.space_left()}{wrapped_line}\n"
 
-def get_terminal_width():
-    try:
-        return shutil.get_terminal_size().columns
-    except (AttributeError, OSError):
-        return 80
-
 def emit(inp):
     buffer = []
     flush = False
@@ -795,6 +832,9 @@ def emit(inp):
             state.current_line += chunk
             
         buffer.append(chunk)
+        # This *might* be dangerous
+        state.reset_inline()
+
         if flush:
             chunk = "\n".join(buffer)
             buffer = []
@@ -806,17 +846,10 @@ def emit(inp):
         else:
             chunk = buffer.pop(0)
 
-        if state.is_pty:
-            print(chunk, end="", flush=True)
-        else:
-            sys.stdout.write(chunk)
+        print(chunk, end="", flush=True)
 
     if len(buffer):
-        chunk = buffer.pop(0)
-        if state.is_pty:
-            print(chunk, end="", flush=True)
-        else:
-            sys.stdout.write(chunk)
+        print(buffer.pop(0), end="", flush=True)
 
 def apply_multipliers(name, H, S, V):
     m = _style.get(name)
@@ -824,7 +857,20 @@ def apply_multipliers(name, H, S, V):
     return ';'.join([str(int(x * 256)) for x in [r, g, b]]) + "m"
 
 def width_calc():
-    state.WidthFull = state.WidthArg or int(get_terminal_width())
+    if not state.WidthFull or not state.WidthArg:
+        if state.WidthArg:
+            state.WidthFull = state.WidthArg
+        else:
+            width = 80
+
+            try:
+                width = shutil.get_terminal_size().columns
+                state.WidthWrap = True
+            except (AttributeError, OSError):
+                pass
+
+            state.WidthFull = width
+
     state.Width = state.WidthFull - 2 * Style.Margin
     Style.Codepad = [
         f"{RESET}{FG}{Style.Dark}{'▄' * state.WidthFull}{RESET}\n",
@@ -838,9 +884,9 @@ def main():
     parser.add_argument("filenameList", nargs="*", help="Input file to process (also takes stdin)")
     parser.add_argument("-l", "--loglevel", default="INFO", help="Set the logging level")
     parser.add_argument("-c", "--color", default=None, help="Set the hsv base: h,s,v")
-    parser.add_argument("-w", "--width", default="0", help="Set the width")
-    parser.add_argument("-e", "--exec", help="Wrap a program for more 'proper' i/o handling")
-    parser.add_argument("-s", "--scrape", help="Scrape code snippets to a directory")
+    parser.add_argument("-w", "--width", default="0", help="Set the width WIDTH")
+    parser.add_argument("-e", "--exec", help="Wrap a program EXEC for more 'proper' i/o handling")
+    parser.add_argument("-s", "--scrape", help="Scrape code snippets to a directory SCRAPE")
     args = parser.parse_args()
 
     if args.color:
@@ -864,8 +910,7 @@ def main():
 
     Style.Codebg = f"{BG}{Style.Dark}"
     Style.Link = f"{FG}{Style.Symbol}{UNDERLINE[0]}"
-    Style.Blockquote = f"{FG}{Style.Grey} \u258E "
-
+    Style.Blockquote = f"{FG}{Style.Grey}│ "
 
     logging.basicConfig(stream=sys.stdout, level=args.loglevel.upper(), format=f'%(message)s')
     state.exec_master, state.exec_slave = pty.openpty()

{streamdown-0.13.0 → streamdown-0.15.0}/tests/chunk-buffer.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-TIMEOUT=${TIMEOUT:-0.1}
+TIMEOUT=${TIMEOUT:-0.35}
 
 while [[ $# -gt 0 ]]; do
     echo "## File: $1"