python-fastllm 0.0.50__tar.gz → 0.0.51__tar.gz

{python_fastllm-0.0.50 → python_fastllm-0.0.51}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: python-fastllm
-Version: 0.0.50
+Version: 0.0.51
 Author-email: Kerem Turgutlu <keremturgutlu@gmail.com>
 License: Apache-2.0
 Project-URL: Repository, https://github.com/AnswerDotAI/fastllm
@@ -10,7 +10,7 @@ Classifier: Programming Language :: Python :: 3 :: Only
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 Requires-Dist: fastcore>=2.1.18
-Requires-Dist: aidialog>=0.0.24
+Requires-Dist: aidialog>=0.0.25
 Requires-Dist: fasttransport>=0.0.2
 Requires-Dist: pillow
 
@@ -28,7 +28,7 @@ Run `pip install python-fastllm` or clone from github and install locally.
 ``` python
 from aidialog.msg_parts import Msg, Part, Text, Thinking, ToolUse, InputImage, mk_tool_res_msg, Completion
 from fastllm.acomplete import acomplete
-from fastllm.types import sample_img_url
+from fastllm.types import sample_img_url, sample_doms
 import asyncio, json
 
 # Helpers
@@ -57,7 +57,7 @@ The same `acomplete` call works with Claude, GPT, Gemini, and Kimi. Just change
 
 ``` python
 models = [
-    ('claude-sonnet-4-20250514', {}),
+    ('claude-sonnet-5', {}),
     ('gpt-4o-mini', {}),
     ('models/gemini-3-flash-preview', {}),
     ('accounts/fireworks/models/kimi-k2p5', dict(vendor_name='fireworks_ai'))
@@ -81,46 +81,6 @@ for name, kw in models:
 
     The response should be simple and direct.
 
-## Multi-Turn — Swap Providers Mid-Conversation
-
-Build a conversation with one provider, then seamlessly continue it with another. `fastllm` translates between every provider’s native format automatically:
-
-``` python
-# Turn 1: Claude starts the conversation
-msgs = [user("Name the 3 largest planets in our solar system. One sentence.")]
-print("Claude: ", end='')
-r1 = await stream(msgs, model='claude-sonnet-4-20250514', max_tokens=mtok)
-```
-
-    Claude: The three largest planets in our solar system are Jupiter, Saturn, and Neptune.
-
-``` python
-# Turn 2: Switch to GPT — just change the model string
-msgs += [r1.message, user("Which one has the most moons?")]
-print("GPT: ", end='')
-r2 = await stream(msgs, model='gpt-4o-mini', max_tokens=mtok)
-```
-
-    GPT: As of now, Saturn has the most moons, with over 80 confirmed moons.
-
-``` python
-# Turn 3: Switch to Gemini — same msgs, different provider
-msgs += [r2.message, user("Summarize our conversation in one sentence.")]
-print("Gemini: ", end='')
-r3 = await stream(msgs, model='models/gemini-3-flash-preview', max_tokens=mtok)
-```
-
-    Gemini: The conversation identified the three largest planets in the solar system and noted that Saturn currently has the most moons.
-
-``` python
-# Turn 4: Switch to Kimi — works the same way
-msgs += [r3.message, user("Thanks! What's one surprising fact about Saturn?")]
-print("Kimi: ", end='')
-r4 = await stream(msgs, model='accounts/fireworks/models/kimi-k2p5', vendor_name='fireworks_ai', max_tokens=mtok)
-```
-
-    Kimi: 🧠🧠🧠🧠🧠🧠🧠🧠🧠🧠Saturn is less dense than water, meaning it would theoretically float if you could find a bathtub large enough to hold it.
-
 ## System Prompts
 
 Pass a system prompt to any provider — `fastllm` maps it to each API’s native mechanism (Anthropic `system`, OpenAI `instructions`, Gemini `system_instruction`):
@@ -129,7 +89,7 @@ Pass a system prompt to any provider — `fastllm` maps it to each API’s nativ
 sys = "You are a pirate chef. Always respond in pirate speak and mention food."
 
 print("Claude: ", end='')
-r = await stream([user("What should I do today?")], model='claude-sonnet-4-20250514', system=sys, max_tokens=mtok)
+r = await stream([user("What should I do today?")], model='claude-sonnet-5', system=sys, max_tokens=mtok)
 
 print("Gemini: ", end='')
 r = await stream([user("What should I do today?")], model='models/gemini-3-flash-preview', system=sys, max_tokens=mtok)
@@ -162,7 +122,7 @@ r = await stream([user("What should I do today?")], model='models/gemini-3-flash
 
 ## Tool Calling — Define Once, Use Anywhere
 
-Define tools in a single canonical format. `fastllm` translates to each provider’s native tool schema automatically. Here we start a tool-use conversation with Claude, provide the result, then switch to GPT and Gemini to continue:
+Define tools in a single canonical format. `fastllm` translates to each provider’s native tool schema automatically. Here Claude requests the tool, receives the result, and answers:
 
 ``` python
 tools = [{"type": "function", "function": {
@@ -173,7 +133,7 @@ tools = [{"type": "function", "function": {
 
 # Turn 1: Claude requests the tool
 msgs = [user("What's the weather in Paris?")]
-r1 = await stream(msgs, model='claude-sonnet-4-20250514', tools=tools, max_tokens=mtok)
+r1 = await stream(msgs, model='claude-sonnet-5', tools=tools, max_tokens=mtok)
 print("Tool calls:", r1.tool_calls)
 ```
 
@@ -183,44 +143,28 @@ print("Tool calls:", r1.tool_calls)
     Tool calls: [ToolUse(raw={'caller': {'type': 'direct'}}, cache_control=None, id='toolu_01RMN1WM7vPBT3ovv5Ex6VzC', name='get_weather', arguments={'city': 'Paris'}, server=False, text=None)]
 
 ``` python
-# Provide the tool result
-msgs += [r1.message, mk_tool_res_msg(r1.tool_calls, ['22°C, sunny with light clouds'])]
-
-# Turn 2: Switch to GPT to interpret the result
-msgs.append(user("Should I bring a jacket?"))
-print("GPT: ", end='')
-r2 = await stream(msgs, model='gpt-4o-mini', tools=tools, max_tokens=mtok)
+# Provide the tool result, then continue the same conversation
+msgs += [r1.message, mk_tool_res_msg(r1.tool_calls, ['22°C, sunny with light clouds']), user("Should I bring a jacket?")]
+print("Claude: ", end='')
+r2 = await stream(msgs, model='claude-sonnet-5', tools=tools, max_tokens=mtok)
 ```
 
     GPT: With the temperature at 22°C and sunny with light clouds, a jacket isn't necessary for most people. However, if you tend to feel cold easily or if you plan to be out in the evening when it might cool down a bit, it could be a good idea to bring a light jacket.
 
-``` python
-# Turn 3: Gemini sees the full cross-provider tool history
-msgs += [r2.message, user("How about tomorrow — will it rain?")]
-r3 = await stream(msgs, model='models/gemini-3-flash-preview', tools=tools, max_tokens=mtok, web_search_options={})
-```
-
-    Tomorrow in Paris (Saturday, June 13), the weather is expected to be mostly pleasant and sunny. 
-
-    While there is a small chance of light, passing showers (around a 20-25% chance), most forecasts indicate a dry day with mostly clear or scattered clouds. Temperatures will likely reach a comfortable high of around 22°C to 27°C (72°F to 80°F), making it a great day for being outdoors. 
-
-    So, while you might see a stray shower, you likely won't need to worry about heavy rain!
-
 ## Tool Choice
 
 Control whether the model must use tools, can’t use tools, or decides on its own:
 
 ``` python
 # Force the model to call a tool (even for a greeting)
-r = await stream([user("Hello there!")], model='claude-sonnet-4-20250514', tools=tools, tool_choice='required', max_tokens=mtok)
+r = await stream([user("Hello there!")], model='claude-sonnet-5', tools=tools, tool_choice='required', max_tokens=mtok)
 print("Forced:", r.tool_calls)
 
 # Prevent tool use (model must answer directly)
 print("\nNo tools: ", end='')
-r = await stream([user("What's the weather?")], model='claude-sonnet-4-20250514', tools=tools, tool_choice='none', max_tokens=mtok)
+r = await stream([user("What's the weather?")], model='claude-sonnet-5', tools=tools, tool_choice='none', max_tokens=mtok)
 ```
 
-
     - ⏳ `get_weather(city="<UNKNOWN>")` ⏳
 
     Forced: [ToolUse(raw={'caller': {'type': 'direct'}}, cache_control=None, id='toolu_01U7tAXjXAtwjp6xNSMbsyPU', name='get_weather', arguments={'city': '<UNKNOWN>'}, server=False, text=None)]
@@ -329,11 +273,11 @@ long_ctx = "You are an expert on the solar system. " * 200
 system = Text(long_ctx, cache_control={'type': 'ephemeral'})
 
 print("Call 1: ", end='')
-r1 = await stream([user("What is Jupiter's mass?")], model='claude-sonnet-4-20250514', system=system, max_tokens=mtok)
+r1 = await stream([user("What is Jupiter's mass?")], model='claude-sonnet-5', system=system, max_tokens=mtok)
 print(f"Usage: {r1.usage}")
 
 print("Call 2: ", end='')
-r2 = await stream([user("How about Saturn?")], model='claude-sonnet-4-20250514', system=system, max_tokens=mtok)
+r2 = await stream([user("How about Saturn?")], model='claude-sonnet-5', system=system, max_tokens=mtok)
 print(f"Usage: {r2.usage}")
 ```
 
@@ -388,7 +332,7 @@ img_msg = Msg(role='user', content=[
     Text("What do you see in this image?")
 ])
 
-for name, kw in [('claude-sonnet-4-20250514', {}), ('gpt-4o-mini', {}), ('models/gemini-3-flash-preview', {})]:
+for name, kw in [('claude-sonnet-5', {}), ('gpt-4o-mini', {}), ('models/gemini-3-flash-preview', {})]:
     print(f"{name:>30s}: ", end='')
     r = await stream([img_msg], model=name, max_tokens=mtok, **kw)
 ```

{python_fastllm-0.0.50 → python_fastllm-0.0.51}/README.md

@@ -12,7 +12,7 @@ Run `pip install python-fastllm` or clone from github and install locally.
 ``` python
 from aidialog.msg_parts import Msg, Part, Text, Thinking, ToolUse, InputImage, mk_tool_res_msg, Completion
 from fastllm.acomplete import acomplete
-from fastllm.types import sample_img_url
+from fastllm.types import sample_img_url, sample_doms
 import asyncio, json
 
 # Helpers
@@ -41,7 +41,7 @@ The same `acomplete` call works with Claude, GPT, Gemini, and Kimi. Just change
 
 ``` python
 models = [
-    ('claude-sonnet-4-20250514', {}),
+    ('claude-sonnet-5', {}),
     ('gpt-4o-mini', {}),
     ('models/gemini-3-flash-preview', {}),
     ('accounts/fireworks/models/kimi-k2p5', dict(vendor_name='fireworks_ai'))
@@ -65,46 +65,6 @@ for name, kw in models:
 
     The response should be simple and direct.
 
-## Multi-Turn — Swap Providers Mid-Conversation
-
-Build a conversation with one provider, then seamlessly continue it with another. `fastllm` translates between every provider’s native format automatically:
-
-``` python
-# Turn 1: Claude starts the conversation
-msgs = [user("Name the 3 largest planets in our solar system. One sentence.")]
-print("Claude: ", end='')
-r1 = await stream(msgs, model='claude-sonnet-4-20250514', max_tokens=mtok)
-```
-
-    Claude: The three largest planets in our solar system are Jupiter, Saturn, and Neptune.
-
-``` python
-# Turn 2: Switch to GPT — just change the model string
-msgs += [r1.message, user("Which one has the most moons?")]
-print("GPT: ", end='')
-r2 = await stream(msgs, model='gpt-4o-mini', max_tokens=mtok)
-```
-
-    GPT: As of now, Saturn has the most moons, with over 80 confirmed moons.
-
-``` python
-# Turn 3: Switch to Gemini — same msgs, different provider
-msgs += [r2.message, user("Summarize our conversation in one sentence.")]
-print("Gemini: ", end='')
-r3 = await stream(msgs, model='models/gemini-3-flash-preview', max_tokens=mtok)
-```
-
-    Gemini: The conversation identified the three largest planets in the solar system and noted that Saturn currently has the most moons.
-
-``` python
-# Turn 4: Switch to Kimi — works the same way
-msgs += [r3.message, user("Thanks! What's one surprising fact about Saturn?")]
-print("Kimi: ", end='')
-r4 = await stream(msgs, model='accounts/fireworks/models/kimi-k2p5', vendor_name='fireworks_ai', max_tokens=mtok)
-```
-
-    Kimi: 🧠🧠🧠🧠🧠🧠🧠🧠🧠🧠Saturn is less dense than water, meaning it would theoretically float if you could find a bathtub large enough to hold it.
-
 ## System Prompts
 
 Pass a system prompt to any provider — `fastllm` maps it to each API’s native mechanism (Anthropic `system`, OpenAI `instructions`, Gemini `system_instruction`):
@@ -113,7 +73,7 @@ Pass a system prompt to any provider — `fastllm` maps it to each API’s nativ
 sys = "You are a pirate chef. Always respond in pirate speak and mention food."
 
 print("Claude: ", end='')
-r = await stream([user("What should I do today?")], model='claude-sonnet-4-20250514', system=sys, max_tokens=mtok)
+r = await stream([user("What should I do today?")], model='claude-sonnet-5', system=sys, max_tokens=mtok)
 
 print("Gemini: ", end='')
 r = await stream([user("What should I do today?")], model='models/gemini-3-flash-preview', system=sys, max_tokens=mtok)
@@ -146,7 +106,7 @@ r = await stream([user("What should I do today?")], model='models/gemini-3-flash
 
 ## Tool Calling — Define Once, Use Anywhere
 
-Define tools in a single canonical format. `fastllm` translates to each provider’s native tool schema automatically. Here we start a tool-use conversation with Claude, provide the result, then switch to GPT and Gemini to continue:
+Define tools in a single canonical format. `fastllm` translates to each provider’s native tool schema automatically. Here Claude requests the tool, receives the result, and answers:
 
 ``` python
 tools = [{"type": "function", "function": {
@@ -157,7 +117,7 @@ tools = [{"type": "function", "function": {
 
 # Turn 1: Claude requests the tool
 msgs = [user("What's the weather in Paris?")]
-r1 = await stream(msgs, model='claude-sonnet-4-20250514', tools=tools, max_tokens=mtok)
+r1 = await stream(msgs, model='claude-sonnet-5', tools=tools, max_tokens=mtok)
 print("Tool calls:", r1.tool_calls)
 ```
 
@@ -167,44 +127,28 @@ print("Tool calls:", r1.tool_calls)
     Tool calls: [ToolUse(raw={'caller': {'type': 'direct'}}, cache_control=None, id='toolu_01RMN1WM7vPBT3ovv5Ex6VzC', name='get_weather', arguments={'city': 'Paris'}, server=False, text=None)]
 
 ``` python
-# Provide the tool result
-msgs += [r1.message, mk_tool_res_msg(r1.tool_calls, ['22°C, sunny with light clouds'])]
-
-# Turn 2: Switch to GPT to interpret the result
-msgs.append(user("Should I bring a jacket?"))
-print("GPT: ", end='')
-r2 = await stream(msgs, model='gpt-4o-mini', tools=tools, max_tokens=mtok)
+# Provide the tool result, then continue the same conversation
+msgs += [r1.message, mk_tool_res_msg(r1.tool_calls, ['22°C, sunny with light clouds']), user("Should I bring a jacket?")]
+print("Claude: ", end='')
+r2 = await stream(msgs, model='claude-sonnet-5', tools=tools, max_tokens=mtok)
 ```
 
     GPT: With the temperature at 22°C and sunny with light clouds, a jacket isn't necessary for most people. However, if you tend to feel cold easily or if you plan to be out in the evening when it might cool down a bit, it could be a good idea to bring a light jacket.
 
-``` python
-# Turn 3: Gemini sees the full cross-provider tool history
-msgs += [r2.message, user("How about tomorrow — will it rain?")]
-r3 = await stream(msgs, model='models/gemini-3-flash-preview', tools=tools, max_tokens=mtok, web_search_options={})
-```
-
-    Tomorrow in Paris (Saturday, June 13), the weather is expected to be mostly pleasant and sunny. 
-
-    While there is a small chance of light, passing showers (around a 20-25% chance), most forecasts indicate a dry day with mostly clear or scattered clouds. Temperatures will likely reach a comfortable high of around 22°C to 27°C (72°F to 80°F), making it a great day for being outdoors. 
-
-    So, while you might see a stray shower, you likely won't need to worry about heavy rain!
-
 ## Tool Choice
 
 Control whether the model must use tools, can’t use tools, or decides on its own:
 
 ``` python
 # Force the model to call a tool (even for a greeting)
-r = await stream([user("Hello there!")], model='claude-sonnet-4-20250514', tools=tools, tool_choice='required', max_tokens=mtok)
+r = await stream([user("Hello there!")], model='claude-sonnet-5', tools=tools, tool_choice='required', max_tokens=mtok)
 print("Forced:", r.tool_calls)
 
 # Prevent tool use (model must answer directly)
 print("\nNo tools: ", end='')
-r = await stream([user("What's the weather?")], model='claude-sonnet-4-20250514', tools=tools, tool_choice='none', max_tokens=mtok)
+r = await stream([user("What's the weather?")], model='claude-sonnet-5', tools=tools, tool_choice='none', max_tokens=mtok)
 ```
 
-
     - ⏳ `get_weather(city="<UNKNOWN>")` ⏳
 
     Forced: [ToolUse(raw={'caller': {'type': 'direct'}}, cache_control=None, id='toolu_01U7tAXjXAtwjp6xNSMbsyPU', name='get_weather', arguments={'city': '<UNKNOWN>'}, server=False, text=None)]
@@ -313,11 +257,11 @@ long_ctx = "You are an expert on the solar system. " * 200
 system = Text(long_ctx, cache_control={'type': 'ephemeral'})
 
 print("Call 1: ", end='')
-r1 = await stream([user("What is Jupiter's mass?")], model='claude-sonnet-4-20250514', system=system, max_tokens=mtok)
+r1 = await stream([user("What is Jupiter's mass?")], model='claude-sonnet-5', system=system, max_tokens=mtok)
 print(f"Usage: {r1.usage}")
 
 print("Call 2: ", end='')
-r2 = await stream([user("How about Saturn?")], model='claude-sonnet-4-20250514', system=system, max_tokens=mtok)
+r2 = await stream([user("How about Saturn?")], model='claude-sonnet-5', system=system, max_tokens=mtok)
 print(f"Usage: {r2.usage}")
 ```
 
@@ -372,7 +316,7 @@ img_msg = Msg(role='user', content=[
     Text("What do you see in this image?")
 ])
 
-for name, kw in [('claude-sonnet-4-20250514', {}), ('gpt-4o-mini', {}), ('models/gemini-3-flash-preview', {})]:
+for name, kw in [('claude-sonnet-5', {}), ('gpt-4o-mini', {}), ('models/gemini-3-flash-preview', {})]:
     print(f"{name:>30s}: ", end='')
     r = await stream([img_msg], model=name, max_tokens=mtok, **kw)
 ```

{python_fastllm-0.0.50 → python_fastllm-0.0.51}/fastllm/__init__.py

@@ -2,4 +2,4 @@
 
 - `fastllm.responses`: Provider-independent Responses resources and streaming over FastLLM"""
 
-__version__ = "0.0.50"
+__version__ = "0.0.51"

{python_fastllm-0.0.50 → python_fastllm-0.0.51}/fastllm/_modidx.py

@@ -51,7 +51,8 @@ d = { 'settings': { 'branch': 'main',
                                    'fastllm.anthropic.norm_tool_calls': ('anthropic.html#norm_tool_calls', 'fastllm/anthropic.py'),
                                    'fastllm.anthropic.norm_tr_parts': ('anthropic.html#norm_tr_parts', 'fastllm/anthropic.py'),
                                    'fastllm.anthropic.norm_usage': ('anthropic.html#norm_usage', 'fastllm/anthropic.py'),
-                                   'fastllm.anthropic.raw_msg': ('anthropic.html#raw_msg', 'fastllm/anthropic.py')},
+                                   'fastllm.anthropic.raw_msg': ('anthropic.html#raw_msg', 'fastllm/anthropic.py'),
+                                   'fastllm.anthropic.search_hits': ('anthropic.html#search_hits', 'fastllm/anthropic.py')},
             'fastllm.chat': { 'fastllm.chat.AsyncChat': ('chat.html#asyncchat', 'fastllm/chat.py'),
                               'fastllm.chat.AsyncChat.__call__': ('chat.html#asyncchat.__call__', 'fastllm/chat.py'),
                               'fastllm.chat.AsyncChat.__init__': ('chat.html#asyncchat.__init__', 'fastllm/chat.py'),
@@ -255,11 +256,15 @@ d = { 'settings': { 'branch': 'main',
                                           'fastllm.openai_responses.raw_msg': ( 'oai_responses.html#raw_msg',
                                                                                 'fastllm/openai_responses.py')},
             'fastllm.responses': { 'fastllm.responses.AsyncResponses': ('responses.html#asyncresponses', 'fastllm/responses.py'),
+                                   'fastllm.responses.AsyncResponses.__init__': ( 'responses.html#asyncresponses.__init__',
+                                                                                  'fastllm/responses.py'),
                                    'fastllm.responses.AsyncResponses.call': ('responses.html#asyncresponses.call', 'fastllm/responses.py'),
                                    'fastllm.responses.AsyncResponses.events': ( 'responses.html#asyncresponses.events',
                                                                                 'fastllm/responses.py'),
                                    'fastllm.responses.AsyncResponses.prepare': ( 'responses.html#asyncresponses.prepare',
                                                                                  'fastllm/responses.py'),
+                                   'fastllm.responses.AsyncResponses.serve': ( 'responses.html#asyncresponses.serve',
+                                                                               'fastllm/responses.py'),
                                    'fastllm.responses.AsyncResponses.server_results': ( 'responses.html#asyncresponses.server_results',
                                                                                         'fastllm/responses.py'),
                                    'fastllm.responses.AsyncResponses.server_turn': ( 'responses.html#asyncresponses.server_turn',

{python_fastllm-0.0.50 → python_fastllm-0.0.51}/fastllm/anthropic.py

@@ -1,9 +1,9 @@
 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/04_anthropic.ipynb.
 
 # %% auto #0
-__all__ = ['ant_tc_types', 'norm_tool_call', 'norm_tool_calls', 'norm_usage', 'finalize_usage', 'norm_finish', 'norm_parts',
-           'raw_msg', 'norm_tr_parts', 'norm_sse_event', 'delta_index_fn', 'collate_raw', 'acollect_stream',
-           'denorm_tool_use', 'denorm_assistant', 'denorm_tool', 'denorm_msgs', 'denorm_tool_schs',
+__all__ = ['ant_tc_types', 'norm_tool_call', 'norm_tool_calls', 'norm_usage', 'finalize_usage', 'norm_finish', 'search_hits',
+           'norm_parts', 'raw_msg', 'norm_tr_parts', 'norm_sse_event', 'delta_index_fn', 'collate_raw',
+           'acollect_stream', 'denorm_tool_use', 'denorm_assistant', 'denorm_tool', 'denorm_msgs', 'denorm_tool_schs',
            'denorm_tool_choice', 'denorm_reasoning', 'denorm_web_search', 'denorm_system', 'denorm_user',
            'denorm_image', 'denorm_file', 'denorm_tool_result', 'apply_cache_idxs', 'mk_payload', 'get_hdrs', 'cost']
 
@@ -15,7 +15,7 @@ from fastcore.meta import *
 from fasttransport.errors import api_error_from_event
 
 from .types import *
-from aidialog.msg_parts import (Part, PartType, mk_part, Msg, Text, Thinking, ToolUse, ToolResult, tool_text,
+from aidialog.msg_parts import (Part, PartType, mk_part, Msg, Completion, Text, Thinking, ToolUse, ToolResult, tool_text,
                                 InputImage, InputAudio, InputVideo, InputFile, data_url)
 from .streaming import *
 from .streaming import mk_acollect_stream
@@ -69,8 +69,16 @@ def _ant_part_type(typ):
     ant_parts_mapping = dict(text=PartType.text, thinking=PartType.thinking, redacted_thinking=PartType.thinking, tool_use=PartType.tool_use, server_tool_use=PartType.tool_use, mcp_tool_use=PartType.tool_use)
     return ant_parts_mapping.get(typ, typ)
 
+def search_hits(parts, blocks):
+    "Set each server call's `text` to the `title: url` lines of its result block, when the response has one"
+    calls = {p.id: p for p in parts if isinstance(p, ToolUse)}
+    for b in blocks:
+        if b.get('type', '').endswith('_tool_result') and (tc := calls.get(b.get('tool_use_id'))):
+            tc.text = '\n'.join(f"{r.get('title', '')}: {r.get('url', '')}" for r in b.get('content') or [] if isinstance(r, dict) and r.get('url'))
+    return parts
+
 def norm_parts(resp):
-    "Canonical parts of a response; a server tool's result block lives only in the raw message"
+    "Canonical parts of a response; a server tool's result block lives in the raw message, and as the call's `text`"
     parts = []
     for b in resp.get("content", []):
         typ = _ant_part_type(b.get("type", "text"))
@@ -78,7 +86,7 @@ def norm_parts(resp):
         elif typ == PartType.tool_use:
             if tc := norm_tool_call(b): parts.append(tc)
         elif typ == PartType.text: parts.append(Text(b.get("text", ""), raw=b))
-    return parts
+    return search_hits(parts, resp.get("content", []))
 
 # %% ../nbs/04_anthropic.ipynb #f24e52db
 def raw_msg(resp):
@@ -150,7 +158,9 @@ def collate_raw(deltas):
 @delegates(mk_acollect_stream, but=['index_fn', 'api_name'])
 async def acollect_stream(resp, **kwargs):
     res = mk_acollect_stream(norm_and_yield(resp, norm_sse_event), index_fn=delta_index_fn, api_name='anthropic', **kwargs)
-    async for o in res: yield o
+    async for o in res:
+        if isinstance(o, Completion): search_hits(o.message.content, o.message.raw['content'])
+        yield o
 
 # %% ../nbs/04_anthropic.ipynb #bb11ab62
 def _ant_cc(block, p):

{python_fastllm-0.0.50 → python_fastllm-0.0.51}/fastllm/responses.py

@@ -160,6 +160,7 @@ class ResponseTurn(BasicRepr):
 # %% ../nbs/06a_responses.ipynb #a648297f
 class AsyncResponses:
     "Run individual Responses turns over FastLLM."
+    def __init__(self): self.tasks = set()   # served turns still running
     def prepare(self, body, previous=None, server_tools=None, **kwargs):
         if not isinstance(body, dict): raise ResponsesError('Request body must be an object')
         model = body.get('model')
@@ -343,3 +344,21 @@ async def events(self:AsyncResponses, turn, finalize=None, stream=None):
     except ResponsesError as exc: yield state.event('error', code=exc.code, message=str(exc), param=exc.param)
     except APIError as exc: yield state.event('error', code=exc.code or 'provider_error', message=str(exc), param=None)
 
+
+# %% ../nbs/06a_responses.ipynb #cc32ce40
+@patch
+def serve(self:AsyncResponses, turn, finalize=None, stream=None, timeout=None):
+    "The frames of `events`, from a task that outlives its reader, so a disconnected client still gets its turn finalized"
+    q = asyncio.Queue()
+    async def _run():
+        try:
+            async with asyncio.timeout(timeout):
+                async for frame in self.events(turn, finalize=finalize, stream=stream): q.put_nowait(frame)
+        except TimeoutError: q.put_nowait(response_event('error', 0, code='timeout', message=f'turn exceeded {timeout}s', param=None))
+        finally: q.put_nowait(None)
+    t = asyncio.create_task(_run())
+    self.tasks.add(t)
+    t.add_done_callback(self.tasks.discard)
+    async def _frames():
+        while (frame := await q.get()) is not None: yield frame
+    return _frames()

{python_fastllm-0.0.50 → python_fastllm-0.0.51}/fastllm/types.py

@@ -3,14 +3,14 @@
 # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/00_types.ipynb.
 
 # %% auto #0
-__all__ = ['FinishReason', 'api_registry', 'model_prices_url', 'sample_img_url', 'haik45', 'sonn45', 'sonn46', 'sonn', 'sonn5',
-           'opus46', 'opus48', 'opus', 'opus5', 'fable', 'fable5', 'fable51', 'gpt54', 'gpt54m', 'gpt55', 'codex54',
-           'codex54m', 'codex55', 'codex53spark', 'model_info_registry', 'modern_llm', 'effort_codes',
-           'deepseek_v4_common', 'deepseek_v4_flash_prices', 'mimo_v25_common', 'codex_pricing', 'sol', 'terra', 'luna',
-           'gpt56s', 'Usage', 'APIRegistry', 'mk_completion', 'fn_schema', 'payload_kwargs', 'provider_req',
-           'get_api_key', 'wrap_typed', 'unwrap_typed', 'resize_b64', 'model_prices_meta', 'infer_api_name',
-           'get_model_meta', 'register_model_info', 'get_model_info', 'effort_code', 'effort_levels', 'resolve_effort',
-           'get_model_pricing', 'approx_pricing', 'is_deepseek_peak_hour', 'price_tier', 'tier_rate']
+__all__ = ['FinishReason', 'api_registry', 'model_prices_url', 'sample_img_url', 'sample_doms', 'haik45', 'sonn45', 'sonn46',
+           'sonn', 'sonn5', 'opus46', 'opus48', 'opus', 'opus5', 'fable', 'fable5', 'fable51', 'gpt54', 'gpt54m',
+           'gpt55', 'codex54', 'codex54m', 'codex55', 'codex53spark', 'model_info_registry', 'modern_llm',
+           'effort_codes', 'deepseek_v4_common', 'deepseek_v4_flash_prices', 'mimo_v25_common', 'codex_pricing', 'sol',
+           'terra', 'luna', 'gpt56s', 'Usage', 'APIRegistry', 'mk_completion', 'fn_schema', 'payload_kwargs',
+           'provider_req', 'get_api_key', 'wrap_typed', 'unwrap_typed', 'resize_b64', 'model_prices_meta',
+           'infer_api_name', 'get_model_meta', 'register_model_info', 'get_model_info', 'effort_code', 'effort_levels',
+           'resolve_effort', 'get_model_pricing', 'approx_pricing', 'is_deepseek_peak_hour', 'price_tier', 'tier_rate']
 
 # %% ../nbs/00_types.ipynb #b4d047fd
 import httpx2, base64, io
@@ -170,6 +170,7 @@ def model_prices_meta():
 
 # %% ../nbs/00_types.ipynb #019882a3
 sample_img_url = "https://img.magnific.com/free-photo/mountain-range-body-water_53876-139760.jpg?semt=ais_hybrid&w=740&q=80"
+sample_doms = ('img.magnific.com', 'arxiv.org', 'openaiassets.blob.core.windows.net')   # hosts of the sample media the lessons fetch, for cachy
 
 # %% ../nbs/00_types.ipynb #68e488d8
 def infer_api_name(model):

{python_fastllm-0.0.50 → python_fastllm-0.0.51}/pyproject.toml

@@ -15,7 +15,7 @@ classifiers = [
     "Programming Language :: Python :: 3",
     "Programming Language :: Python :: 3 :: Only",
 ]
-dependencies = ['fastcore>=2.1.18', 'aidialog>=0.0.24', 'fasttransport>=0.0.2', 'pillow']
+dependencies = ['fastcore>=2.1.18', 'aidialog>=0.0.25', 'fasttransport>=0.0.2', 'pillow']
 
 [project.scripts]
 fastllm-auth = "fastllm.codex:main"

{python_fastllm-0.0.50 → python_fastllm-0.0.51}/python_fastllm.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: python-fastllm
-Version: 0.0.50
+Version: 0.0.51
 Author-email: Kerem Turgutlu <keremturgutlu@gmail.com>
 License: Apache-2.0
 Project-URL: Repository, https://github.com/AnswerDotAI/fastllm
@@ -10,7 +10,7 @@ Classifier: Programming Language :: Python :: 3 :: Only
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 Requires-Dist: fastcore>=2.1.18
-Requires-Dist: aidialog>=0.0.24
+Requires-Dist: aidialog>=0.0.25
 Requires-Dist: fasttransport>=0.0.2
 Requires-Dist: pillow
 
@@ -28,7 +28,7 @@ Run `pip install python-fastllm` or clone from github and install locally.
 ``` python
 from aidialog.msg_parts import Msg, Part, Text, Thinking, ToolUse, InputImage, mk_tool_res_msg, Completion
 from fastllm.acomplete import acomplete
-from fastllm.types import sample_img_url
+from fastllm.types import sample_img_url, sample_doms
 import asyncio, json
 
 # Helpers
@@ -57,7 +57,7 @@ The same `acomplete` call works with Claude, GPT, Gemini, and Kimi. Just change
 
 ``` python
 models = [
-    ('claude-sonnet-4-20250514', {}),
+    ('claude-sonnet-5', {}),
     ('gpt-4o-mini', {}),
     ('models/gemini-3-flash-preview', {}),
     ('accounts/fireworks/models/kimi-k2p5', dict(vendor_name='fireworks_ai'))
@@ -81,46 +81,6 @@ for name, kw in models:
 
     The response should be simple and direct.
 
-## Multi-Turn — Swap Providers Mid-Conversation
-
-Build a conversation with one provider, then seamlessly continue it with another. `fastllm` translates between every provider’s native format automatically:
-
-``` python
-# Turn 1: Claude starts the conversation
-msgs = [user("Name the 3 largest planets in our solar system. One sentence.")]
-print("Claude: ", end='')
-r1 = await stream(msgs, model='claude-sonnet-4-20250514', max_tokens=mtok)
-```
-
-    Claude: The three largest planets in our solar system are Jupiter, Saturn, and Neptune.
-
-``` python
-# Turn 2: Switch to GPT — just change the model string
-msgs += [r1.message, user("Which one has the most moons?")]
-print("GPT: ", end='')
-r2 = await stream(msgs, model='gpt-4o-mini', max_tokens=mtok)
-```
-
-    GPT: As of now, Saturn has the most moons, with over 80 confirmed moons.
-
-``` python
-# Turn 3: Switch to Gemini — same msgs, different provider
-msgs += [r2.message, user("Summarize our conversation in one sentence.")]
-print("Gemini: ", end='')
-r3 = await stream(msgs, model='models/gemini-3-flash-preview', max_tokens=mtok)
-```
-
-    Gemini: The conversation identified the three largest planets in the solar system and noted that Saturn currently has the most moons.
-
-``` python
-# Turn 4: Switch to Kimi — works the same way
-msgs += [r3.message, user("Thanks! What's one surprising fact about Saturn?")]
-print("Kimi: ", end='')
-r4 = await stream(msgs, model='accounts/fireworks/models/kimi-k2p5', vendor_name='fireworks_ai', max_tokens=mtok)
-```
-
-    Kimi: 🧠🧠🧠🧠🧠🧠🧠🧠🧠🧠Saturn is less dense than water, meaning it would theoretically float if you could find a bathtub large enough to hold it.
-
 ## System Prompts
 
 Pass a system prompt to any provider — `fastllm` maps it to each API’s native mechanism (Anthropic `system`, OpenAI `instructions`, Gemini `system_instruction`):
@@ -129,7 +89,7 @@ Pass a system prompt to any provider — `fastllm` maps it to each API’s nativ
 sys = "You are a pirate chef. Always respond in pirate speak and mention food."
 
 print("Claude: ", end='')
-r = await stream([user("What should I do today?")], model='claude-sonnet-4-20250514', system=sys, max_tokens=mtok)
+r = await stream([user("What should I do today?")], model='claude-sonnet-5', system=sys, max_tokens=mtok)
 
 print("Gemini: ", end='')
 r = await stream([user("What should I do today?")], model='models/gemini-3-flash-preview', system=sys, max_tokens=mtok)
@@ -162,7 +122,7 @@ r = await stream([user("What should I do today?")], model='models/gemini-3-flash
 
 ## Tool Calling — Define Once, Use Anywhere
 
-Define tools in a single canonical format. `fastllm` translates to each provider’s native tool schema automatically. Here we start a tool-use conversation with Claude, provide the result, then switch to GPT and Gemini to continue:
+Define tools in a single canonical format. `fastllm` translates to each provider’s native tool schema automatically. Here Claude requests the tool, receives the result, and answers:
 
 ``` python
 tools = [{"type": "function", "function": {
@@ -173,7 +133,7 @@ tools = [{"type": "function", "function": {
 
 # Turn 1: Claude requests the tool
 msgs = [user("What's the weather in Paris?")]
-r1 = await stream(msgs, model='claude-sonnet-4-20250514', tools=tools, max_tokens=mtok)
+r1 = await stream(msgs, model='claude-sonnet-5', tools=tools, max_tokens=mtok)
 print("Tool calls:", r1.tool_calls)
 ```
 
@@ -183,44 +143,28 @@ print("Tool calls:", r1.tool_calls)
     Tool calls: [ToolUse(raw={'caller': {'type': 'direct'}}, cache_control=None, id='toolu_01RMN1WM7vPBT3ovv5Ex6VzC', name='get_weather', arguments={'city': 'Paris'}, server=False, text=None)]
 
 ``` python
-# Provide the tool result
-msgs += [r1.message, mk_tool_res_msg(r1.tool_calls, ['22°C, sunny with light clouds'])]
-
-# Turn 2: Switch to GPT to interpret the result
-msgs.append(user("Should I bring a jacket?"))
-print("GPT: ", end='')
-r2 = await stream(msgs, model='gpt-4o-mini', tools=tools, max_tokens=mtok)
+# Provide the tool result, then continue the same conversation
+msgs += [r1.message, mk_tool_res_msg(r1.tool_calls, ['22°C, sunny with light clouds']), user("Should I bring a jacket?")]
+print("Claude: ", end='')
+r2 = await stream(msgs, model='claude-sonnet-5', tools=tools, max_tokens=mtok)
 ```
 
     GPT: With the temperature at 22°C and sunny with light clouds, a jacket isn't necessary for most people. However, if you tend to feel cold easily or if you plan to be out in the evening when it might cool down a bit, it could be a good idea to bring a light jacket.
 
-``` python
-# Turn 3: Gemini sees the full cross-provider tool history
-msgs += [r2.message, user("How about tomorrow — will it rain?")]
-r3 = await stream(msgs, model='models/gemini-3-flash-preview', tools=tools, max_tokens=mtok, web_search_options={})
-```
-
-    Tomorrow in Paris (Saturday, June 13), the weather is expected to be mostly pleasant and sunny. 
-
-    While there is a small chance of light, passing showers (around a 20-25% chance), most forecasts indicate a dry day with mostly clear or scattered clouds. Temperatures will likely reach a comfortable high of around 22°C to 27°C (72°F to 80°F), making it a great day for being outdoors. 
-
-    So, while you might see a stray shower, you likely won't need to worry about heavy rain!
-
 ## Tool Choice
 
 Control whether the model must use tools, can’t use tools, or decides on its own:
 
 ``` python
 # Force the model to call a tool (even for a greeting)
-r = await stream([user("Hello there!")], model='claude-sonnet-4-20250514', tools=tools, tool_choice='required', max_tokens=mtok)
+r = await stream([user("Hello there!")], model='claude-sonnet-5', tools=tools, tool_choice='required', max_tokens=mtok)
 print("Forced:", r.tool_calls)
 
 # Prevent tool use (model must answer directly)
 print("\nNo tools: ", end='')
-r = await stream([user("What's the weather?")], model='claude-sonnet-4-20250514', tools=tools, tool_choice='none', max_tokens=mtok)
+r = await stream([user("What's the weather?")], model='claude-sonnet-5', tools=tools, tool_choice='none', max_tokens=mtok)
 ```
 
-
     - ⏳ `get_weather(city="<UNKNOWN>")` ⏳
 
     Forced: [ToolUse(raw={'caller': {'type': 'direct'}}, cache_control=None, id='toolu_01U7tAXjXAtwjp6xNSMbsyPU', name='get_weather', arguments={'city': '<UNKNOWN>'}, server=False, text=None)]
@@ -329,11 +273,11 @@ long_ctx = "You are an expert on the solar system. " * 200
 system = Text(long_ctx, cache_control={'type': 'ephemeral'})
 
 print("Call 1: ", end='')
-r1 = await stream([user("What is Jupiter's mass?")], model='claude-sonnet-4-20250514', system=system, max_tokens=mtok)
+r1 = await stream([user("What is Jupiter's mass?")], model='claude-sonnet-5', system=system, max_tokens=mtok)
 print(f"Usage: {r1.usage}")
 
 print("Call 2: ", end='')
-r2 = await stream([user("How about Saturn?")], model='claude-sonnet-4-20250514', system=system, max_tokens=mtok)
+r2 = await stream([user("How about Saturn?")], model='claude-sonnet-5', system=system, max_tokens=mtok)
 print(f"Usage: {r2.usage}")
 ```
 
@@ -388,7 +332,7 @@ img_msg = Msg(role='user', content=[
     Text("What do you see in this image?")
 ])
 
-for name, kw in [('claude-sonnet-4-20250514', {}), ('gpt-4o-mini', {}), ('models/gemini-3-flash-preview', {})]:
+for name, kw in [('claude-sonnet-5', {}), ('gpt-4o-mini', {}), ('models/gemini-3-flash-preview', {})]:
     print(f"{name:>30s}: ", end='')
     r = await stream([img_msg], model=name, max_tokens=mtok, **kw)
 ```

{python_fastllm-0.0.50 → python_fastllm-0.0.51}/python_fastllm.egg-info/requires.txt

@@ -1,4 +1,4 @@
 fastcore>=2.1.18
-aidialog>=0.0.24
+aidialog>=0.0.25
 fasttransport>=0.0.2
 pillow