abstractcore 2.4.6__tar.gz → 2.4.7__tar.gz

{abstractcore-2.4.6 → abstractcore-2.4.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: abstractcore
-Version: 2.4.6
+Version: 2.4.7
 Summary: Unified interface to all LLM providers with essential infrastructure for tool calling, streaming, and model management
 Author-email: Laurent-Philippe Albou <contact@abstractcore.ai>
 Maintainer-email: Laurent-Philippe Albou <contact@abstractcore.ai>
@@ -29,6 +29,7 @@ License-File: LICENSE
 Requires-Dist: pydantic<3.0.0,>=2.0.0
 Requires-Dist: httpx<1.0.0,>=0.24.0
 Requires-Dist: tiktoken<1.0.0,>=0.5.0
+Requires-Dist: requests<3.0.0,>=2.25.0
 Provides-Extra: openai
 Requires-Dist: openai<2.0.0,>=1.0.0; extra == "openai"
 Provides-Extra: anthropic
@@ -46,6 +47,11 @@ Provides-Extra: embeddings
 Requires-Dist: sentence-transformers<4.0.0,>=2.7.0; extra == "embeddings"
 Requires-Dist: numpy<2.0.0,>=1.20.0; extra == "embeddings"
 Provides-Extra: processing
+Provides-Extra: tools
+Requires-Dist: beautifulsoup4<5.0.0,>=4.12.0; extra == "tools"
+Requires-Dist: lxml<6.0.0,>=4.9.0; extra == "tools"
+Requires-Dist: duckduckgo-search<4.0.0,>=3.8.0; extra == "tools"
+Requires-Dist: psutil<6.0.0,>=5.9.0; extra == "tools"
 Provides-Extra: media
 Requires-Dist: Pillow<12.0.0,>=10.0.0; extra == "media"
 Requires-Dist: pymupdf4llm<1.0.0,>=0.0.20; extra == "media"
@@ -60,9 +66,9 @@ Requires-Dist: abstractcore[huggingface]; extra == "heavy-providers"
 Provides-Extra: all-providers
 Requires-Dist: abstractcore[anthropic,embeddings,huggingface,lmstudio,mlx,ollama,openai]; extra == "all-providers"
 Provides-Extra: all
-Requires-Dist: abstractcore[anthropic,dev,docs,embeddings,huggingface,lmstudio,media,mlx,ollama,openai,processing,server,test]; extra == "all"
+Requires-Dist: abstractcore[anthropic,dev,docs,embeddings,huggingface,lmstudio,media,mlx,ollama,openai,processing,server,test,tools]; extra == "all"
 Provides-Extra: lightweight
-Requires-Dist: abstractcore[anthropic,embeddings,lmstudio,media,ollama,openai,processing,server]; extra == "lightweight"
+Requires-Dist: abstractcore[anthropic,embeddings,lmstudio,media,ollama,openai,processing,server,tools]; extra == "lightweight"
 Provides-Extra: dev
 Requires-Dist: pytest>=7.0.0; extra == "dev"
 Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
@@ -89,7 +95,7 @@ Requires-Dist: mkdocs-material>=9.0.0; extra == "docs"
 Requires-Dist: mkdocstrings[python]>=0.22.0; extra == "docs"
 Requires-Dist: mkdocs-autorefs>=0.4.0; extra == "docs"
 Provides-Extra: full-dev
-Requires-Dist: abstractcore[all-providers,dev,docs,test]; extra == "full-dev"
+Requires-Dist: abstractcore[all-providers,dev,docs,test,tools]; extra == "full-dev"
 Dynamic: license-file
 
 # AbstractCore
@@ -155,6 +161,45 @@ response = llm.generate(
 print(response.content)
 ```
 
+### Response Object (GenerateResponse)
+
+Every LLM generation returns a **GenerateResponse** object with consistent structure across all providers:
+
+```python
+from abstractcore import create_llm
+
+llm = create_llm("openai", model="gpt-4o-mini")
+response = llm.generate("Explain quantum computing in simple terms")
+
+# Core response data
+print(f"Content: {response.content}")               # Generated text
+print(f"Model: {response.model}")                   # Model used
+print(f"Finish reason: {response.finish_reason}")   # Why generation stopped
+
+# Consistent token access across ALL providers (NEW in v2.4.7)
+print(f"Input tokens: {response.input_tokens}")     # Always available
+print(f"Output tokens: {response.output_tokens}")   # Always available  
+print(f"Total tokens: {response.total_tokens}")     # Always available
+
+# Generation time tracking (NEW in v2.4.7)
+print(f"Generation time: {response.gen_time}ms")    # Always available (rounded to 1 decimal)
+
+# Advanced access
+print(f"Tool calls: {response.tool_calls}")         # Tools executed (if any)
+print(f"Raw usage: {response.usage}")               # Provider-specific token data
+print(f"Metadata: {response.metadata}")             # Additional context
+
+# Comprehensive summary
+print(f"Summary: {response.get_summary()}")         # "Model: gpt-4o-mini | Tokens: 117 | Time: 1234.5ms"
+```
+
+**Token Count Sources:**
+- **Provider APIs**: OpenAI, Anthropic, LMStudio (native API token counts)
+- **AbstractCore Calculation**: MLX, HuggingFace, Mock (using `token_utils.py`)
+- **Mixed Sources**: Ollama (combination of provider and calculated tokens)
+
+**Backward Compatibility**: Legacy `prompt_tokens` and `completion_tokens` keys remain available in `response.usage` dictionary.
+
 ### Built-in Tools
 
 AbstractCore includes a comprehensive set of ready-to-use tools for common tasks:
@@ -271,6 +316,7 @@ response = llm.generate(
 - **Session Management**: Persistent conversations with metadata, analytics, and complete serialization
 - **Structured Responses**: Clean, predictable output formats with Pydantic
 - **Streaming Support**: Real-time token generation for interactive experiences
+- **Consistent Token Terminology**: Unified `input_tokens`, `output_tokens`, `total_tokens` across all providers
 - **Embeddings**: Built-in support for semantic search and RAG applications
 - **Universal Server**: Optional OpenAI-compatible API server with `/v1/responses` endpoint
 

{abstractcore-2.4.6 → abstractcore-2.4.7}/README.md

@@ -61,6 +61,45 @@ response = llm.generate(
 print(response.content)
 ```
 
+### Response Object (GenerateResponse)
+
+Every LLM generation returns a **GenerateResponse** object with consistent structure across all providers:
+
+```python
+from abstractcore import create_llm
+
+llm = create_llm("openai", model="gpt-4o-mini")
+response = llm.generate("Explain quantum computing in simple terms")
+
+# Core response data
+print(f"Content: {response.content}")               # Generated text
+print(f"Model: {response.model}")                   # Model used
+print(f"Finish reason: {response.finish_reason}")   # Why generation stopped
+
+# Consistent token access across ALL providers (NEW in v2.4.7)
+print(f"Input tokens: {response.input_tokens}")     # Always available
+print(f"Output tokens: {response.output_tokens}")   # Always available  
+print(f"Total tokens: {response.total_tokens}")     # Always available
+
+# Generation time tracking (NEW in v2.4.7)
+print(f"Generation time: {response.gen_time}ms")    # Always available (rounded to 1 decimal)
+
+# Advanced access
+print(f"Tool calls: {response.tool_calls}")         # Tools executed (if any)
+print(f"Raw usage: {response.usage}")               # Provider-specific token data
+print(f"Metadata: {response.metadata}")             # Additional context
+
+# Comprehensive summary
+print(f"Summary: {response.get_summary()}")         # "Model: gpt-4o-mini | Tokens: 117 | Time: 1234.5ms"
+```
+
+**Token Count Sources:**
+- **Provider APIs**: OpenAI, Anthropic, LMStudio (native API token counts)
+- **AbstractCore Calculation**: MLX, HuggingFace, Mock (using `token_utils.py`)
+- **Mixed Sources**: Ollama (combination of provider and calculated tokens)
+
+**Backward Compatibility**: Legacy `prompt_tokens` and `completion_tokens` keys remain available in `response.usage` dictionary.
+
 ### Built-in Tools
 
 AbstractCore includes a comprehensive set of ready-to-use tools for common tasks:
@@ -177,6 +216,7 @@ response = llm.generate(
 - **Session Management**: Persistent conversations with metadata, analytics, and complete serialization
 - **Structured Responses**: Clean, predictable output formats with Pydantic
 - **Streaming Support**: Real-time token generation for interactive experiences
+- **Consistent Token Terminology**: Unified `input_tokens`, `output_tokens`, `total_tokens` across all providers
 - **Embeddings**: Built-in support for semantic search and RAG applications
 - **Universal Server**: Optional OpenAI-compatible API server with `/v1/responses` endpoint
 

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore/__init__.py

@@ -44,6 +44,9 @@ except ImportError:
 from .processing import BasicSummarizer, SummaryStyle, SummaryLength, BasicExtractor
 _has_processing = True
 
+# Tools module (core functionality)
+from .tools import tool
+
 __all__ = [
     'create_llm',
     'BasicSession',
@@ -54,7 +57,8 @@ __all__ = [
     'MessageRole',
     'ModelNotFoundError',
     'ProviderAPIError',
-    'AuthenticationError'
+    'AuthenticationError',
+    'tool'
 ]
 
 if _has_embeddings:

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore/assets/session_schema.json

@@ -109,7 +109,7 @@
                 "tokens_before": { "type": "integer" },
                 "tokens_after": { "type": "integer" },
                 "compression_ratio": { "type": "number" },
-                "generation_time_ms": { "type": "number" }
+                "gen_time": { "type": "number" }
               }
             }
           },

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore/core/session.py

@@ -760,7 +760,7 @@ class BasicSession:
                 "tokens_before": original_tokens,
                 "tokens_after": self._estimate_tokens_for_summary(summary_result.summary),
                 "compression_ratio": self._calculate_compression_ratio(original_tokens, summary_result.summary),
-                "generation_time_ms": duration_ms
+                "gen_time": duration_ms
             }
         }
         

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore/core/types.py

@@ -91,6 +91,7 @@ class GenerateResponse:
     usage: Optional[Dict[str, int]] = None
     tool_calls: Optional[List[Dict[str, Any]]] = None
     metadata: Optional[Dict[str, Any]] = None
+    gen_time: Optional[float] = None  # Generation time in milliseconds
 
     def has_tool_calls(self) -> bool:
         """Check if response contains tool calls"""
@@ -109,6 +110,29 @@ class GenerateResponse:
             parts.append(f"Model: {self.model}")
         if self.usage:
             parts.append(f"Tokens: {self.usage.get('total_tokens', 'unknown')}")
+        if self.gen_time:
+            parts.append(f"Time: {self.gen_time:.1f}ms")
         if self.tool_calls:
             parts.append(f"Tools: {len(self.tool_calls)} executed")
-        return " | ".join(parts)
+        return " | ".join(parts)
+    
+    @property
+    def input_tokens(self) -> Optional[int]:
+        """Get input tokens with consistent terminology (prompt_tokens or input_tokens)."""
+        if not self.usage:
+            return None
+        return self.usage.get('input_tokens') or self.usage.get('prompt_tokens')
+    
+    @property
+    def output_tokens(self) -> Optional[int]:
+        """Get output tokens with consistent terminology (completion_tokens or output_tokens)."""
+        if not self.usage:
+            return None
+        return self.usage.get('output_tokens') or self.usage.get('completion_tokens')
+    
+    @property
+    def total_tokens(self) -> Optional[int]:
+        """Get total tokens."""
+        if not self.usage:
+            return None
+        return self.usage.get('total_tokens')

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore/providers/anthropic_provider.py

@@ -186,8 +186,14 @@ class AnthropicProvider(BaseProvider):
             if stream:
                 return self._stream_response(call_params, tools)
             else:
+                # Track generation time
+                start_time = time.time()
                 response = self.client.messages.create(**call_params)
+                gen_time = round((time.time() - start_time) * 1000, 1)
+                
                 formatted = self._format_response(response)
+                # Add generation time to response
+                formatted.gen_time = gen_time
 
                 # Handle tool execution for Anthropic responses
                 if tools and (formatted.has_tool_calls() or

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore/providers/huggingface_provider.py

@@ -863,6 +863,9 @@ class HuggingFaceProvider(BaseProvider):
                 if torch.cuda.is_available():
                     torch.cuda.manual_seed_all(seed)
 
+            # Track generation time
+            start_time = time.time()
+            
             outputs = self.pipeline(
                 input_text,
                 max_new_tokens=max_new_tokens,
@@ -874,6 +877,8 @@ class HuggingFaceProvider(BaseProvider):
                 truncation=True,
                 return_full_text=False
             )
+            
+            gen_time = round((time.time() - start_time) * 1000, 1)
 
             if outputs and len(outputs) > 0:
                 response_text = outputs[0]['generated_text'].strip()
@@ -885,34 +890,41 @@ class HuggingFaceProvider(BaseProvider):
                     content=response_text,
                     model=self.model,
                     finish_reason="stop",
-                    usage=usage
+                    usage=usage,
+                    gen_time=gen_time
                 )
             else:
                 return GenerateResponse(
                     content="",
                     model=self.model,
-                    finish_reason="stop"
+                    finish_reason="stop",
+                    gen_time=gen_time
                 )
 
         except Exception as e:
+            gen_time = round((time.time() - start_time) * 1000, 1) if 'start_time' in locals() else 0.0
             return GenerateResponse(
                 content=f"Error: {str(e)}",
                 model=self.model,
-                finish_reason="error"
+                finish_reason="error",
+                gen_time=gen_time
             )
 
     def _calculate_usage(self, prompt: str, response: str) -> Dict[str, int]:
         """Calculate token usage using centralized token utilities."""
         from ..utils.token_utils import TokenUtils
 
-        prompt_tokens = TokenUtils.estimate_tokens(prompt, self.model)
-        completion_tokens = TokenUtils.estimate_tokens(response, self.model)
-        total_tokens = prompt_tokens + completion_tokens
+        input_tokens = TokenUtils.estimate_tokens(prompt, self.model)
+        output_tokens = TokenUtils.estimate_tokens(response, self.model)
+        total_tokens = input_tokens + output_tokens
 
         return {
-            "prompt_tokens": prompt_tokens,
-            "completion_tokens": completion_tokens,
-            "total_tokens": total_tokens
+            "input_tokens": input_tokens,
+            "output_tokens": output_tokens,
+            "total_tokens": total_tokens,
+            # Keep legacy keys for backward compatibility
+            "prompt_tokens": input_tokens,
+            "completion_tokens": output_tokens
         }
 
     def _stream_generate_transformers(self, input_text: str, max_new_tokens: int,

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore/providers/lmstudio_provider.py

@@ -4,6 +4,7 @@ LM Studio provider implementation (OpenAI-compatible API).
 
 import httpx
 import json
+import time
 from typing import List, Dict, Any, Optional, Union, Iterator, Type
 
 try:
@@ -225,12 +226,15 @@ class LMStudioProvider(BaseProvider):
             if not hasattr(self, 'client') or self.client is None:
                 raise ProviderAPIError("HTTP client not initialized")
 
+            # Track generation time
+            start_time = time.time()
             response = self.client.post(
                 f"{self.base_url}/chat/completions",
                 json=payload,
                 headers={"Content-Type": "application/json"}
             )
             response.raise_for_status()
+            gen_time = round((time.time() - start_time) * 1000, 1)
 
             result = response.json()
 
@@ -252,10 +256,14 @@ class LMStudioProvider(BaseProvider):
                 finish_reason=finish_reason,
                 raw_response=result,
                 usage={
+                    "input_tokens": usage.get("prompt_tokens", 0),
+                    "output_tokens": usage.get("completion_tokens", 0),
+                    "total_tokens": usage.get("total_tokens", 0),
+                    # Keep legacy keys for backward compatibility
                     "prompt_tokens": usage.get("prompt_tokens", 0),
-                    "completion_tokens": usage.get("completion_tokens", 0),
-                    "total_tokens": usage.get("total_tokens", 0)
-                }
+                    "completion_tokens": usage.get("completion_tokens", 0)
+                },
+                gen_time=gen_time
             )
 
         except AttributeError as e:

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore/providers/mlx_provider.py

@@ -266,6 +266,9 @@ class MLXProvider(BaseProvider):
             mx.random.seed(seed)
             self.logger.debug(f"Set MLX random seed to {seed} for deterministic generation")
 
+        # Track generation time
+        start_time = time.time()
+        
         # Try different MLX API signatures
         try:
             # Try new mlx-lm API
@@ -288,6 +291,8 @@ class MLXProvider(BaseProvider):
                 # Fallback to basic response
                 response_text = prompt + " I am an AI assistant powered by MLX on Apple Silicon."
 
+        gen_time = round((time.time() - start_time) * 1000, 1)
+        
         # Use the full response as-is - preserve all content including thinking
         generated = response_text.strip()
 
@@ -295,21 +300,25 @@ class MLXProvider(BaseProvider):
             content=generated,
             model=self.model,
             finish_reason="stop",
-            usage=self._calculate_usage(prompt, generated)
+            usage=self._calculate_usage(prompt, generated),
+            gen_time=gen_time
         )
 
     def _calculate_usage(self, prompt: str, response: str) -> Dict[str, int]:
         """Calculate token usage using centralized token utilities."""
         from ..utils.token_utils import TokenUtils
         
-        prompt_tokens = TokenUtils.estimate_tokens(prompt, self.model)
-        completion_tokens = TokenUtils.estimate_tokens(response, self.model)
-        total_tokens = prompt_tokens + completion_tokens
+        input_tokens = TokenUtils.estimate_tokens(prompt, self.model)
+        output_tokens = TokenUtils.estimate_tokens(response, self.model)
+        total_tokens = input_tokens + output_tokens
         
         return {
-            "prompt_tokens": prompt_tokens,
-            "completion_tokens": completion_tokens,
-            "total_tokens": total_tokens
+            "input_tokens": input_tokens,
+            "output_tokens": output_tokens,
+            "total_tokens": total_tokens,
+            # Keep legacy keys for backward compatibility
+            "prompt_tokens": input_tokens,
+            "completion_tokens": output_tokens
         }
 
     def _stream_generate(self, prompt: str, max_tokens: int, temperature: float, top_p: float, tool_call_tags: Optional[str] = None, seed: Optional[int] = None) -> Iterator[GenerateResponse]:

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore/providers/mock_provider.py

@@ -48,6 +48,12 @@ class MockProvider(BaseProvider):
 
     def _single_response(self, prompt: str, response_model: Optional[Type[BaseModel]] = None) -> GenerateResponse:
         """Generate single mock response"""
+        import time
+        
+        # Simulate generation time (10-100ms for mock)
+        start_time = time.time()
+        time.sleep(0.01 + (len(prompt) % 10) * 0.01)  # 10-100ms based on prompt length
+        gen_time = round((time.time() - start_time) * 1000, 1)
 
         if response_model and PYDANTIC_AVAILABLE:
             # Generate valid JSON for structured output
@@ -59,21 +65,25 @@ class MockProvider(BaseProvider):
             content=content,
             model=self.model,
             finish_reason="stop",
-            usage=self._calculate_mock_usage(prompt, content)
+            usage=self._calculate_mock_usage(prompt, content),
+            gen_time=gen_time
         )
 
     def _calculate_mock_usage(self, prompt: str, response: str) -> Dict[str, int]:
         """Calculate mock token usage using centralized token utilities."""
         from ..utils.token_utils import TokenUtils
         
-        prompt_tokens = TokenUtils.estimate_tokens(prompt, self.model)
-        completion_tokens = TokenUtils.estimate_tokens(response, self.model)
-        total_tokens = prompt_tokens + completion_tokens
+        input_tokens = TokenUtils.estimate_tokens(prompt, self.model)
+        output_tokens = TokenUtils.estimate_tokens(response, self.model)
+        total_tokens = input_tokens + output_tokens
         
         return {
-            "prompt_tokens": prompt_tokens,
-            "completion_tokens": completion_tokens, 
-            "total_tokens": total_tokens
+            "input_tokens": input_tokens,
+            "output_tokens": output_tokens,
+            "total_tokens": total_tokens,
+            # Keep legacy keys for backward compatibility
+            "prompt_tokens": input_tokens,
+            "completion_tokens": output_tokens
         }
 
     def _stream_response(self, prompt: str) -> Iterator[GenerateResponse]:

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore/providers/ollama_provider.py

@@ -225,11 +225,14 @@ class OllamaProvider(BaseProvider):
     def _single_generate(self, endpoint: str, payload: Dict[str, Any], tools: Optional[List[Dict[str, Any]]] = None) -> GenerateResponse:
         """Generate single response"""
         try:
+            # Track generation time
+            start_time = time.time()
             response = self.client.post(
                 f"{self.base_url}{endpoint}",
                 json=payload
             )
             response.raise_for_status()
+            gen_time = round((time.time() - start_time) * 1000, 1)
 
             result = response.json()
 
@@ -246,10 +249,14 @@ class OllamaProvider(BaseProvider):
                 finish_reason="stop",
                 raw_response=result,
                 usage={
+                    "input_tokens": result.get("prompt_eval_count", 0),
+                    "output_tokens": result.get("eval_count", 0),
+                    "total_tokens": result.get("prompt_eval_count", 0) + result.get("eval_count", 0),
+                    # Keep legacy keys for backward compatibility
                     "prompt_tokens": result.get("prompt_eval_count", 0),
-                    "completion_tokens": result.get("eval_count", 0),
-                    "total_tokens": result.get("prompt_eval_count", 0) + result.get("eval_count", 0)
-                }
+                    "completion_tokens": result.get("eval_count", 0)
+                },
+                gen_time=gen_time
             )
 
             # Execute tools if enabled and tools are present

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore/providers/openai_provider.py

@@ -169,8 +169,14 @@ class OpenAIProvider(BaseProvider):
             if stream:
                 return self._stream_response(call_params, tools)
             else:
+                # Track generation time
+                start_time = time.time()
                 response = self.client.chat.completions.create(**call_params)
+                gen_time = round((time.time() - start_time) * 1000, 1)
+                
                 formatted = self._format_response(response)
+                # Add generation time to response
+                formatted.gen_time = gen_time
 
                 # Handle tool execution for OpenAI native responses
                 if tools and formatted.has_tool_calls():
@@ -216,13 +222,16 @@ class OpenAIProvider(BaseProvider):
                     "arguments": tc.function.arguments
                 })
 
-        # Build usage dict with detailed breakdown
+        # Build usage dict with consistent terminology
         usage = None
         if hasattr(response, 'usage'):
             usage = {
+                "input_tokens": response.usage.prompt_tokens,
+                "output_tokens": response.usage.completion_tokens,
+                "total_tokens": response.usage.total_tokens,
+                # Keep legacy keys for backward compatibility
                 "prompt_tokens": response.usage.prompt_tokens,
-                "completion_tokens": response.usage.completion_tokens,
-                "total_tokens": response.usage.total_tokens
+                "completion_tokens": response.usage.completion_tokens
             }
 
             # Add detailed token breakdown for reasoning models

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore/utils/version.py

@@ -11,4 +11,4 @@ including when the package is installed from PyPI where pyproject.toml is not av
 
 # Package version - update this when releasing new versions
 # This must be manually synchronized with the version in pyproject.toml
-__version__ = "2.4.6"
+__version__ = "2.4.7"

{abstractcore-2.4.6 → abstractcore-2.4.7}/abstractcore.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: abstractcore
-Version: 2.4.6
+Version: 2.4.7
 Summary: Unified interface to all LLM providers with essential infrastructure for tool calling, streaming, and model management
 Author-email: Laurent-Philippe Albou <contact@abstractcore.ai>
 Maintainer-email: Laurent-Philippe Albou <contact@abstractcore.ai>
@@ -29,6 +29,7 @@ License-File: LICENSE
 Requires-Dist: pydantic<3.0.0,>=2.0.0
 Requires-Dist: httpx<1.0.0,>=0.24.0
 Requires-Dist: tiktoken<1.0.0,>=0.5.0
+Requires-Dist: requests<3.0.0,>=2.25.0
 Provides-Extra: openai
 Requires-Dist: openai<2.0.0,>=1.0.0; extra == "openai"
 Provides-Extra: anthropic
@@ -46,6 +47,11 @@ Provides-Extra: embeddings
 Requires-Dist: sentence-transformers<4.0.0,>=2.7.0; extra == "embeddings"
 Requires-Dist: numpy<2.0.0,>=1.20.0; extra == "embeddings"
 Provides-Extra: processing
+Provides-Extra: tools
+Requires-Dist: beautifulsoup4<5.0.0,>=4.12.0; extra == "tools"
+Requires-Dist: lxml<6.0.0,>=4.9.0; extra == "tools"
+Requires-Dist: duckduckgo-search<4.0.0,>=3.8.0; extra == "tools"
+Requires-Dist: psutil<6.0.0,>=5.9.0; extra == "tools"
 Provides-Extra: media
 Requires-Dist: Pillow<12.0.0,>=10.0.0; extra == "media"
 Requires-Dist: pymupdf4llm<1.0.0,>=0.0.20; extra == "media"
@@ -60,9 +66,9 @@ Requires-Dist: abstractcore[huggingface]; extra == "heavy-providers"
 Provides-Extra: all-providers
 Requires-Dist: abstractcore[anthropic,embeddings,huggingface,lmstudio,mlx,ollama,openai]; extra == "all-providers"
 Provides-Extra: all
-Requires-Dist: abstractcore[anthropic,dev,docs,embeddings,huggingface,lmstudio,media,mlx,ollama,openai,processing,server,test]; extra == "all"
+Requires-Dist: abstractcore[anthropic,dev,docs,embeddings,huggingface,lmstudio,media,mlx,ollama,openai,processing,server,test,tools]; extra == "all"
 Provides-Extra: lightweight
-Requires-Dist: abstractcore[anthropic,embeddings,lmstudio,media,ollama,openai,processing,server]; extra == "lightweight"
+Requires-Dist: abstractcore[anthropic,embeddings,lmstudio,media,ollama,openai,processing,server,tools]; extra == "lightweight"
 Provides-Extra: dev
 Requires-Dist: pytest>=7.0.0; extra == "dev"
 Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
@@ -89,7 +95,7 @@ Requires-Dist: mkdocs-material>=9.0.0; extra == "docs"
 Requires-Dist: mkdocstrings[python]>=0.22.0; extra == "docs"
 Requires-Dist: mkdocs-autorefs>=0.4.0; extra == "docs"
 Provides-Extra: full-dev
-Requires-Dist: abstractcore[all-providers,dev,docs,test]; extra == "full-dev"
+Requires-Dist: abstractcore[all-providers,dev,docs,test,tools]; extra == "full-dev"
 Dynamic: license-file
 
 # AbstractCore
@@ -155,6 +161,45 @@ response = llm.generate(
 print(response.content)
 ```
 
+### Response Object (GenerateResponse)
+
+Every LLM generation returns a **GenerateResponse** object with consistent structure across all providers:
+
+```python
+from abstractcore import create_llm
+
+llm = create_llm("openai", model="gpt-4o-mini")
+response = llm.generate("Explain quantum computing in simple terms")
+
+# Core response data
+print(f"Content: {response.content}")               # Generated text
+print(f"Model: {response.model}")                   # Model used
+print(f"Finish reason: {response.finish_reason}")   # Why generation stopped
+
+# Consistent token access across ALL providers (NEW in v2.4.7)
+print(f"Input tokens: {response.input_tokens}")     # Always available
+print(f"Output tokens: {response.output_tokens}")   # Always available  
+print(f"Total tokens: {response.total_tokens}")     # Always available
+
+# Generation time tracking (NEW in v2.4.7)
+print(f"Generation time: {response.gen_time}ms")    # Always available (rounded to 1 decimal)
+
+# Advanced access
+print(f"Tool calls: {response.tool_calls}")         # Tools executed (if any)
+print(f"Raw usage: {response.usage}")               # Provider-specific token data
+print(f"Metadata: {response.metadata}")             # Additional context
+
+# Comprehensive summary
+print(f"Summary: {response.get_summary()}")         # "Model: gpt-4o-mini | Tokens: 117 | Time: 1234.5ms"
+```
+
+**Token Count Sources:**
+- **Provider APIs**: OpenAI, Anthropic, LMStudio (native API token counts)
+- **AbstractCore Calculation**: MLX, HuggingFace, Mock (using `token_utils.py`)
+- **Mixed Sources**: Ollama (combination of provider and calculated tokens)
+
+**Backward Compatibility**: Legacy `prompt_tokens` and `completion_tokens` keys remain available in `response.usage` dictionary.
+
 ### Built-in Tools
 
 AbstractCore includes a comprehensive set of ready-to-use tools for common tasks:
@@ -271,6 +316,7 @@ response = llm.generate(
 - **Session Management**: Persistent conversations with metadata, analytics, and complete serialization
 - **Structured Responses**: Clean, predictable output formats with Pydantic
 - **Streaming Support**: Real-time token generation for interactive experiences
+- **Consistent Token Terminology**: Unified `input_tokens`, `output_tokens`, `total_tokens` across all providers
 - **Embeddings**: Built-in support for semantic search and RAG applications
 - **Universal Server**: Optional OpenAI-compatible API server with `/v1/responses` endpoint