abstractcore 2.9.1__py3-none-any.whl → 2.11.4__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (85) hide show
  1. abstractcore/__init__.py +7 -27
  2. abstractcore/apps/deepsearch.py +9 -4
  3. abstractcore/apps/extractor.py +33 -100
  4. abstractcore/apps/intent.py +19 -0
  5. abstractcore/apps/judge.py +20 -1
  6. abstractcore/apps/summarizer.py +20 -1
  7. abstractcore/architectures/detection.py +34 -1
  8. abstractcore/architectures/response_postprocessing.py +313 -0
  9. abstractcore/assets/architecture_formats.json +38 -8
  10. abstractcore/assets/model_capabilities.json +882 -160
  11. abstractcore/compression/__init__.py +1 -2
  12. abstractcore/compression/glyph_processor.py +6 -4
  13. abstractcore/config/main.py +52 -20
  14. abstractcore/config/manager.py +390 -12
  15. abstractcore/config/vision_config.py +5 -5
  16. abstractcore/core/interface.py +151 -3
  17. abstractcore/core/session.py +16 -10
  18. abstractcore/download.py +1 -1
  19. abstractcore/embeddings/manager.py +20 -6
  20. abstractcore/endpoint/__init__.py +2 -0
  21. abstractcore/endpoint/app.py +458 -0
  22. abstractcore/mcp/client.py +3 -1
  23. abstractcore/media/__init__.py +52 -17
  24. abstractcore/media/auto_handler.py +42 -22
  25. abstractcore/media/base.py +44 -1
  26. abstractcore/media/capabilities.py +12 -33
  27. abstractcore/media/enrichment.py +105 -0
  28. abstractcore/media/handlers/anthropic_handler.py +19 -28
  29. abstractcore/media/handlers/local_handler.py +124 -70
  30. abstractcore/media/handlers/openai_handler.py +19 -31
  31. abstractcore/media/processors/__init__.py +4 -2
  32. abstractcore/media/processors/audio_processor.py +57 -0
  33. abstractcore/media/processors/office_processor.py +8 -3
  34. abstractcore/media/processors/pdf_processor.py +46 -3
  35. abstractcore/media/processors/text_processor.py +22 -24
  36. abstractcore/media/processors/video_processor.py +58 -0
  37. abstractcore/media/types.py +97 -4
  38. abstractcore/media/utils/image_scaler.py +20 -2
  39. abstractcore/media/utils/video_frames.py +219 -0
  40. abstractcore/media/vision_fallback.py +136 -22
  41. abstractcore/processing/__init__.py +32 -3
  42. abstractcore/processing/basic_deepsearch.py +15 -10
  43. abstractcore/processing/basic_intent.py +3 -2
  44. abstractcore/processing/basic_judge.py +3 -2
  45. abstractcore/processing/basic_summarizer.py +1 -1
  46. abstractcore/providers/__init__.py +3 -1
  47. abstractcore/providers/anthropic_provider.py +95 -8
  48. abstractcore/providers/base.py +1516 -81
  49. abstractcore/providers/huggingface_provider.py +546 -69
  50. abstractcore/providers/lmstudio_provider.py +30 -916
  51. abstractcore/providers/mlx_provider.py +382 -35
  52. abstractcore/providers/model_capabilities.py +5 -1
  53. abstractcore/providers/ollama_provider.py +99 -15
  54. abstractcore/providers/openai_compatible_provider.py +406 -180
  55. abstractcore/providers/openai_provider.py +188 -44
  56. abstractcore/providers/openrouter_provider.py +76 -0
  57. abstractcore/providers/registry.py +61 -5
  58. abstractcore/providers/streaming.py +138 -33
  59. abstractcore/providers/vllm_provider.py +92 -817
  60. abstractcore/server/app.py +478 -28
  61. abstractcore/server/audio_endpoints.py +139 -0
  62. abstractcore/server/vision_endpoints.py +1319 -0
  63. abstractcore/structured/handler.py +316 -41
  64. abstractcore/tools/common_tools.py +5501 -2012
  65. abstractcore/tools/comms_tools.py +1641 -0
  66. abstractcore/tools/core.py +37 -7
  67. abstractcore/tools/handler.py +4 -9
  68. abstractcore/tools/parser.py +49 -2
  69. abstractcore/tools/tag_rewriter.py +2 -1
  70. abstractcore/tools/telegram_tdlib.py +407 -0
  71. abstractcore/tools/telegram_tools.py +261 -0
  72. abstractcore/utils/cli.py +1085 -72
  73. abstractcore/utils/structured_logging.py +29 -8
  74. abstractcore/utils/token_utils.py +2 -0
  75. abstractcore/utils/truncation.py +29 -0
  76. abstractcore/utils/version.py +3 -4
  77. abstractcore/utils/vlm_token_calculator.py +12 -2
  78. abstractcore-2.11.4.dist-info/METADATA +562 -0
  79. abstractcore-2.11.4.dist-info/RECORD +133 -0
  80. {abstractcore-2.9.1.dist-info → abstractcore-2.11.4.dist-info}/WHEEL +1 -1
  81. {abstractcore-2.9.1.dist-info → abstractcore-2.11.4.dist-info}/entry_points.txt +1 -0
  82. abstractcore-2.9.1.dist-info/METADATA +0 -1190
  83. abstractcore-2.9.1.dist-info/RECORD +0 -119
  84. {abstractcore-2.9.1.dist-info → abstractcore-2.11.4.dist-info}/licenses/LICENSE +0 -0
  85. {abstractcore-2.9.1.dist-info → abstractcore-2.11.4.dist-info}/top_level.txt +0 -0
@@ -1,1190 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: abstractcore
3
- Version: 2.9.1
4
- Summary: Unified interface to all LLM providers with essential infrastructure for tool calling, streaming, and model management
5
- Author-email: Laurent-Philippe Albou <contact@abstractcore.ai>
6
- Maintainer-email: Laurent-Philippe Albou <contact@abstractcore.ai>
7
- License: MIT
8
- Project-URL: Homepage, https://lpalbou.github.io/AbstractCore
9
- Project-URL: Documentation, https://github.com/lpalbou/AbstractCore#readme
10
- Project-URL: Repository, https://github.com/lpalbou/AbstractCore
11
- Project-URL: Bug Tracker, https://github.com/lpalbou/AbstractCore/issues
12
- Project-URL: Changelog, https://github.com/lpalbou/AbstractCore/blob/main/CHANGELOG.md
13
- Keywords: llm,openai,anthropic,ollama,lmstudio,huggingface,mlx,ai,machine-learning,natural-language-processing,tool-calling,streaming
14
- Classifier: Development Status :: 4 - Beta
15
- Classifier: Intended Audience :: Developers
16
- Classifier: License :: OSI Approved :: MIT License
17
- Classifier: Operating System :: OS Independent
18
- Classifier: Programming Language :: Python :: 3
19
- Classifier: Programming Language :: Python :: 3.9
20
- Classifier: Programming Language :: Python :: 3.10
21
- Classifier: Programming Language :: Python :: 3.11
22
- Classifier: Programming Language :: Python :: 3.12
23
- Classifier: Topic :: Software Development :: Libraries :: Python Modules
24
- Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
25
- Classifier: Topic :: Internet :: WWW/HTTP :: HTTP Servers
26
- Requires-Python: >=3.9
27
- Description-Content-Type: text/markdown
28
- License-File: LICENSE
29
- Requires-Dist: pydantic<3.0.0,>=2.0.0
30
- Requires-Dist: httpx<1.0.0,>=0.24.0
31
- Requires-Dist: tiktoken<1.0.0,>=0.5.0
32
- Requires-Dist: requests<3.0.0,>=2.25.0
33
- Requires-Dist: beautifulsoup4<5.0.0,>=4.12.0
34
- Requires-Dist: Pillow<12.0.0,>=10.0.0
35
- Provides-Extra: openai
36
- Requires-Dist: openai<2.0.0,>=1.0.0; extra == "openai"
37
- Provides-Extra: anthropic
38
- Requires-Dist: anthropic<1.0.0,>=0.25.0; extra == "anthropic"
39
- Provides-Extra: ollama
40
- Provides-Extra: lmstudio
41
- Provides-Extra: huggingface
42
- Requires-Dist: transformers<5.0.0,>=4.57.1; extra == "huggingface"
43
- Requires-Dist: torch<3.0.0,>=2.6.0; extra == "huggingface"
44
- Requires-Dist: torchvision>=0.17.0; extra == "huggingface"
45
- Requires-Dist: torchaudio>=2.1.0; extra == "huggingface"
46
- Requires-Dist: llama-cpp-python<1.0.0,>=0.2.0; extra == "huggingface"
47
- Requires-Dist: outlines>=0.1.0; extra == "huggingface"
48
- Provides-Extra: mlx
49
- Requires-Dist: mlx<1.0.0,>=0.15.0; extra == "mlx"
50
- Requires-Dist: mlx-lm<1.0.0,>=0.15.0; extra == "mlx"
51
- Requires-Dist: outlines>=0.1.0; extra == "mlx"
52
- Provides-Extra: vllm
53
- Requires-Dist: vllm<1.0.0,>=0.6.0; extra == "vllm"
54
- Provides-Extra: embeddings
55
- Requires-Dist: sentence-transformers<6.0.0,>=5.1.0; extra == "embeddings"
56
- Requires-Dist: numpy<2.0.0,>=1.20.0; extra == "embeddings"
57
- Provides-Extra: processing
58
- Provides-Extra: tools
59
- Requires-Dist: beautifulsoup4<5.0.0,>=4.12.0; extra == "tools"
60
- Requires-Dist: lxml<6.0.0,>=4.9.0; extra == "tools"
61
- Requires-Dist: ddgs<10.0.0,>=9.10.0; python_version >= "3.10" and extra == "tools"
62
- Requires-Dist: duckduckgo-search<4.0.0,>=3.8.0; python_version < "3.10" and extra == "tools"
63
- Requires-Dist: psutil<6.0.0,>=5.9.0; extra == "tools"
64
- Provides-Extra: tool
65
- Requires-Dist: beautifulsoup4<5.0.0,>=4.12.0; extra == "tool"
66
- Requires-Dist: lxml<6.0.0,>=4.9.0; extra == "tool"
67
- Requires-Dist: ddgs<10.0.0,>=9.10.0; python_version >= "3.10" and extra == "tool"
68
- Requires-Dist: duckduckgo-search<4.0.0,>=3.8.0; python_version < "3.10" and extra == "tool"
69
- Requires-Dist: psutil<6.0.0,>=5.9.0; extra == "tool"
70
- Provides-Extra: media
71
- Requires-Dist: Pillow<12.0.0,>=10.0.0; extra == "media"
72
- Requires-Dist: pymupdf4llm<1.0.0,>=0.0.20; extra == "media"
73
- Requires-Dist: unstructured[office]<1.0.0,>=0.10.0; extra == "media"
74
- Requires-Dist: pandas<3.0.0,>=1.0.0; extra == "media"
75
- Provides-Extra: compression
76
- Requires-Dist: abstractcore[media]; extra == "compression"
77
- Requires-Dist: pdf2image<2.0.0,>=1.16.0; extra == "compression"
78
- Provides-Extra: api-providers
79
- Requires-Dist: abstractcore[anthropic,openai]; extra == "api-providers"
80
- Provides-Extra: local-providers
81
- Requires-Dist: abstractcore[lmstudio,mlx,ollama]; extra == "local-providers"
82
- Provides-Extra: local-providers-apple
83
- Requires-Dist: abstractcore[lmstudio,mlx,ollama]; extra == "local-providers-apple"
84
- Provides-Extra: local-providers-gpu
85
- Requires-Dist: abstractcore[lmstudio,ollama,vllm]; extra == "local-providers-gpu"
86
- Provides-Extra: gpu-providers
87
- Requires-Dist: abstractcore[huggingface,vllm]; extra == "gpu-providers"
88
- Provides-Extra: heavy-providers
89
- Requires-Dist: abstractcore[huggingface]; extra == "heavy-providers"
90
- Provides-Extra: all-providers
91
- Requires-Dist: abstractcore[anthropic,embeddings,huggingface,lmstudio,mlx,ollama,openai,vllm]; extra == "all-providers"
92
- Provides-Extra: all-providers-apple
93
- Requires-Dist: abstractcore[anthropic,embeddings,huggingface,lmstudio,mlx,ollama,openai]; extra == "all-providers-apple"
94
- Provides-Extra: all-providers-gpu
95
- Requires-Dist: abstractcore[anthropic,embeddings,huggingface,lmstudio,ollama,openai,vllm]; extra == "all-providers-gpu"
96
- Provides-Extra: all-providers-non-mlx
97
- Requires-Dist: abstractcore[anthropic,embeddings,huggingface,lmstudio,ollama,openai]; extra == "all-providers-non-mlx"
98
- Provides-Extra: local-providers-non-mlx
99
- Requires-Dist: abstractcore[lmstudio,ollama]; extra == "local-providers-non-mlx"
100
- Provides-Extra: all
101
- Requires-Dist: abstractcore[anthropic,compression,dev,docs,embeddings,huggingface,lmstudio,media,mlx,ollama,openai,processing,server,test,tools,vllm]; extra == "all"
102
- Provides-Extra: all-apple
103
- Requires-Dist: abstractcore[anthropic,compression,dev,docs,embeddings,huggingface,lmstudio,media,mlx,ollama,openai,processing,server,test,tools]; extra == "all-apple"
104
- Provides-Extra: all-gpu
105
- Requires-Dist: abstractcore[anthropic,compression,dev,docs,embeddings,huggingface,lmstudio,media,ollama,openai,processing,server,test,tools,vllm]; extra == "all-gpu"
106
- Provides-Extra: all-non-mlx
107
- Requires-Dist: abstractcore[anthropic,compression,dev,docs,embeddings,huggingface,lmstudio,media,ollama,openai,processing,server,test,tools]; extra == "all-non-mlx"
108
- Provides-Extra: lightweight
109
- Requires-Dist: abstractcore[anthropic,compression,embeddings,lmstudio,media,ollama,openai,processing,server,tools]; extra == "lightweight"
110
- Provides-Extra: dev
111
- Requires-Dist: pytest>=7.0.0; extra == "dev"
112
- Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
113
- Requires-Dist: pytest-mock>=3.10.0; extra == "dev"
114
- Requires-Dist: black>=23.0.0; extra == "dev"
115
- Requires-Dist: isort>=5.12.0; extra == "dev"
116
- Requires-Dist: mypy>=1.5.0; extra == "dev"
117
- Requires-Dist: ruff>=0.1.0; extra == "dev"
118
- Requires-Dist: pre-commit>=3.0.0; extra == "dev"
119
- Provides-Extra: server
120
- Requires-Dist: fastapi<1.0.0,>=0.100.0; extra == "server"
121
- Requires-Dist: uvicorn[standard]<1.0.0,>=0.23.0; extra == "server"
122
- Requires-Dist: sse-starlette<2.0.0,>=1.6.0; extra == "server"
123
- Provides-Extra: test
124
- Requires-Dist: pytest>=7.0.0; extra == "test"
125
- Requires-Dist: pytest-asyncio>=0.21.0; extra == "test"
126
- Requires-Dist: pytest-mock>=3.10.0; extra == "test"
127
- Requires-Dist: pytest-cov>=4.0.0; extra == "test"
128
- Requires-Dist: responses>=0.23.0; extra == "test"
129
- Requires-Dist: httpx>=0.24.0; extra == "test"
130
- Provides-Extra: docs
131
- Requires-Dist: mkdocs>=1.5.0; extra == "docs"
132
- Requires-Dist: mkdocs-material>=9.0.0; extra == "docs"
133
- Requires-Dist: mkdocstrings[python]>=0.22.0; extra == "docs"
134
- Requires-Dist: mkdocs-autorefs>=0.4.0; extra == "docs"
135
- Provides-Extra: full-dev
136
- Requires-Dist: abstractcore[all-providers,dev,docs,test,tools]; extra == "full-dev"
137
- Dynamic: license-file
138
-
139
- # AbstractCore
140
-
141
- [![PyPI version](https://img.shields.io/pypi/v/abstractcore.svg)](https://pypi.org/project/abstractcore/)
142
- [![Python Version](https://img.shields.io/pypi/pyversions/abstractcore)](https://pypi.org/project/abstractcore/)
143
- [![license](https://img.shields.io/github/license/lpalbou/abstractcore)](https://github.com/lpalbou/abstractcore/blob/main/LICENSE)
144
- [![GitHub stars](https://img.shields.io/github/stars/lpalbou/abstractcore?style=social)](https://github.com/lpalbou/abstractcore/stargazers)
145
-
146
- A unified Python library for interaction with multiple Large Language Model (LLM) providers.
147
-
148
- **Write once, run everywhere.**
149
-
150
- ## Quick Start
151
-
152
- ### Installation
153
-
154
- ```bash
155
- # macOS/Apple Silicon (includes MLX)
156
- pip install abstractcore[all]
157
-
158
- # Linux/Windows (excludes MLX)
159
- pip install abstractcore[all-non-mlx]
160
- ```
161
-
162
- ### Basic Usage
163
-
164
- ```python
165
- from abstractcore import create_llm
166
-
167
- # Works with any provider - just change the provider name
168
- llm = create_llm("anthropic", model="claude-3-5-haiku-latest")
169
- response = llm.generate("What is the capital of France?")
170
- print(response.content)
171
- ```
172
-
173
- ### Deterministic Generation
174
-
175
- ```python
176
- from abstractcore import create_llm
177
-
178
- # Deterministic outputs with seed + temperature=0
179
- llm = create_llm("openai", model="gpt-3.5-turbo", seed=42, temperature=0.0)
180
-
181
- # These will produce identical outputs
182
- response1 = llm.generate("Write exactly 3 words about coding")
183
- response2 = llm.generate("Write exactly 3 words about coding")
184
- print(f"Response 1: {response1.content}") # "Innovative, challenging, rewarding."
185
- print(f"Response 2: {response2.content}") # "Innovative, challenging, rewarding."
186
- ```
187
-
188
- ### Tool Calling
189
-
190
- ```python
191
- from abstractcore import create_llm, tool
192
-
193
- @tool
194
- def get_current_weather(city: str):
195
- """Fetch current weather for a given city."""
196
- return f"Weather in {city}: 72°F, Sunny"
197
-
198
- llm = create_llm("openai", model="gpt-4o-mini")
199
- response = llm.generate(
200
- "What's the weather like in San Francisco?",
201
- tools=[get_current_weather]
202
- )
203
- print(response.content)
204
- ```
205
-
206
- ### Tool Execution Modes
207
-
208
- AbstractCore supports two tool execution modes:
209
-
210
- **Mode 1: Passthrough (Default)** - Returns raw tool call tags for downstream processing
211
-
212
- ```python
213
- from abstractcore import create_llm
214
- from abstractcore.tools import tool
215
-
216
- @tool(name="get_weather", description="Get weather for a city")
217
- def get_weather(city: str) -> str:
218
- return f"Weather in {city}: Sunny, 22°C"
219
-
220
- llm = create_llm("ollama", model="qwen3:4b") # execute_tools=False by default
221
- response = llm.generate("What's the weather in Paris?", tools=[get_weather])
222
- # response.content contains raw tool call tags: <|tool_call|>...
223
- # Downstream runtime (AbstractRuntime, Codex, Claude Code) parses and executes
224
- ```
225
-
226
- **Use case**: Agent loops, AbstractRuntime, Codex, Claude Code, custom orchestration
227
-
228
- **Mode 2: Direct Execution** - AbstractCore executes tools and returns results
229
-
230
- ```python
231
- from abstractcore import create_llm
232
- from abstractcore.tools import tool
233
- from abstractcore.tools.registry import register_tool
234
-
235
- @tool(name="get_weather", description="Get weather for a city")
236
- def get_weather(city: str) -> str:
237
- return f"Weather in {city}: Sunny, 22°C"
238
-
239
- register_tool(get_weather) # Required for direct execution
240
-
241
- llm = create_llm("ollama", model="qwen3:4b", execute_tools=True)
242
- response = llm.generate("What's the weather in Paris?", tools=[get_weather])
243
- # response.content contains executed tool results
244
- ```
245
-
246
- **Use case**: Simple scripts, single-turn tool use
247
-
248
- > **Note**: The `@tool` decorator creates metadata but does NOT register globally. Tools are passed explicitly to `generate()`. Use `register_tool()` only when using direct execution mode.
249
-
250
- ### Response Object (GenerateResponse)
251
-
252
- Every LLM generation returns a **GenerateResponse** object with consistent structure across all providers:
253
-
254
- ```python
255
- from abstractcore import create_llm
256
-
257
- llm = create_llm("openai", model="gpt-4o-mini")
258
- response = llm.generate("Explain quantum computing in simple terms")
259
-
260
- # Core response data
261
- print(f"Content: {response.content}") # Generated text
262
- print(f"Model: {response.model}") # Model used
263
- print(f"Finish reason: {response.finish_reason}") # Why generation stopped
264
-
265
- # Consistent token access across ALL providers (NEW in v2.4.7)
266
- print(f"Input tokens: {response.input_tokens}") # Always available
267
- print(f"Output tokens: {response.output_tokens}") # Always available
268
- print(f"Total tokens: {response.total_tokens}") # Always available
269
-
270
- # Generation time tracking (NEW in v2.4.7)
271
- print(f"Generation time: {response.gen_time}ms") # Always available (rounded to 1 decimal)
272
-
273
- # Advanced access
274
- print(f"Tool calls: {response.tool_calls}") # Tools executed (if any)
275
- print(f"Raw usage: {response.usage}") # Provider-specific token data
276
- print(f"Metadata: {response.metadata}") # Additional context
277
-
278
- # Comprehensive summary
279
- print(f"Summary: {response.get_summary()}") # "Model: gpt-4o-mini | Tokens: 117 | Time: 1234.5ms"
280
- ```
281
-
282
- **Token Count Sources:**
283
- - **Provider APIs**: OpenAI, Anthropic, LMStudio (native API token counts)
284
- - **AbstractCore Calculation**: MLX, HuggingFace (using `token_utils.py`)
285
- - **Mixed Sources**: Ollama (combination of provider and calculated tokens)
286
-
287
- **Backward Compatibility**: Legacy `prompt_tokens` and `completion_tokens` keys remain available in `response.usage` dictionary.
288
-
289
- ### Built-in Tools
290
-
291
- AbstractCore includes a comprehensive set of ready-to-use tools for common tasks:
292
-
293
- > Note: `abstractcore.tools.common_tools` requires `abstractcore[tools]` (BeautifulSoup, lxml, web search backends, etc.).
294
-
295
- ```python
296
- from abstractcore.tools.common_tools import fetch_url, search_files, read_file
297
-
298
- # Intelligent web content fetching with automatic parsing
299
- result = fetch_url("https://api.github.com/repos/python/cpython")
300
- # Automatically detects JSON, HTML, images, PDFs, etc. and provides structured analysis
301
-
302
- # File system operations
303
- files = search_files("def.*fetch", ".", file_pattern="*.py") # Find function definitions
304
- content = read_file("config.json") # Read file contents
305
-
306
- # Use with any LLM
307
- llm = create_llm("anthropic", model="claude-3-5-haiku-latest")
308
- response = llm.generate(
309
- "Analyze this API response and summarize the key information",
310
- tools=[fetch_url]
311
- )
312
- ```
313
-
314
- **Available Tools:**
315
- - `fetch_url` - Intelligent web content fetching with automatic content type detection and parsing
316
- - `search_files` - Search for text patterns inside files using regex
317
- - `list_files` - Find and list files by names/paths using glob patterns
318
- - `read_file` - Read file contents with optional line range selection
319
- - `write_file` - Write content to files with directory creation
320
- - `edit_file` - Edit files using pattern matching and replacement
321
- - `web_search` - Search the web using DuckDuckGo
322
- - `execute_command` - Execute shell commands safely with security controls
323
-
324
- ### Session Management
325
-
326
- ```python
327
- from abstractcore import BasicSession, create_llm
328
-
329
- # Create a persistent conversation session
330
- llm = create_llm("openai", model="gpt-4o-mini")
331
- session = BasicSession(llm, system_prompt="You are a helpful assistant.")
332
-
333
- # Add messages with metadata
334
- session.add_message('user', 'Hello!', name='alice', location='Paris')
335
- response = session.generate('What is Python?', name='bob')
336
-
337
- # Save complete conversation with optional analytics
338
- session.save('conversation.json') # Basic save
339
- session.save('analyzed.json', summary=True, assessment=True, facts=True) # With analytics
340
-
341
- # Load and continue conversation
342
- loaded_session = BasicSession.load('conversation.json', provider=llm)
343
- ```
344
-
345
- [Learn more about Session](docs/session.md)
346
-
347
- ### Interaction Tracing (Observability)
348
-
349
- Enable complete observability of LLM interactions for debugging, compliance, and transparency:
350
-
351
- ```python
352
- from abstractcore import create_llm
353
- from abstractcore.core.session import BasicSession
354
- from abstractcore.utils import export_traces
355
-
356
- # Enable tracing on provider
357
- llm = create_llm('openai', model='gpt-4o-mini', enable_tracing=True, max_traces=100)
358
-
359
- # Or on session for automatic correlation
360
- session = BasicSession(provider=llm, enable_tracing=True)
361
-
362
- # Generate with custom metadata
363
- response = session.generate(
364
- "Write Python code",
365
- step_type='code_generation',
366
- attempt_number=1
367
- )
368
-
369
- # Access complete trace
370
- trace_id = response.metadata['trace_id']
371
- trace = llm.get_traces(trace_id=trace_id)
372
-
373
- # Full interaction context
374
- print(f"Prompt: {trace['prompt']}")
375
- print(f"Response: {trace['response']['content']}")
376
- print(f"Tokens: {trace['response']['usage']['total_tokens']}")
377
- print(f"Time: {trace['response']['generation_time_ms']}ms")
378
- print(f"Custom metadata: {trace['metadata']}")
379
-
380
- # Get all session traces
381
- traces = session.get_interaction_history()
382
-
383
- # Export to JSONL, JSON, or Markdown
384
- export_traces(traces, format='markdown', file_path='workflow_trace.md')
385
- ```
386
-
387
- **What's captured:**
388
- - All prompts, system prompts, and conversation history
389
- - Complete responses with token usage and timing
390
- - Generation parameters (temperature, tokens, seed, etc.)
391
- - Custom metadata for workflow tracking
392
- - Tool calls and results
393
-
394
- [Learn more about Interaction Tracing](docs/interaction-tracing.md)
395
-
396
- ### Async/Await Support
397
-
398
- Execute concurrent LLM requests for batch operations, multi-provider comparisons, or non-blocking web applications. **Production-ready with validated 6-7.5x performance improvement** for concurrent requests.
399
-
400
- ```python
401
- import asyncio
402
- from abstractcore import create_llm
403
-
404
- async def main():
405
- llm = create_llm("openai", model="gpt-4o-mini")
406
-
407
- # Execute 3 requests concurrently (6-7x faster!)
408
- tasks = [
409
- llm.agenerate(f"Summarize {topic}")
410
- for topic in ["Python", "JavaScript", "Rust"]
411
- ]
412
- responses = await asyncio.gather(*tasks)
413
-
414
- for response in responses:
415
- print(response.content)
416
-
417
- asyncio.run(main())
418
- ```
419
-
420
- **Performance (Validated with Real Testing):**
421
- - **Ollama**: 7.5x faster for concurrent requests
422
- - **LMStudio**: 6.5x faster for concurrent requests
423
- - **OpenAI**: 6.0x faster for concurrent requests
424
- - **Anthropic**: 7.4x faster for concurrent requests
425
- - **Average**: ~7x speedup across all providers
426
-
427
- **Native Async vs Fallback:**
428
- - **Native async** (httpx.AsyncClient): Ollama, LMStudio, OpenAI, Anthropic
429
- - **Fallback** (asyncio.to_thread): MLX, HuggingFace
430
- - All providers work seamlessly - fallback keeps event loop responsive
431
-
432
- **Use Cases:**
433
- - Batch operations with 6-7x speedup via parallel execution
434
- - Multi-provider comparisons (query OpenAI and Anthropic simultaneously)
435
- - FastAPI/async web frameworks integration
436
- - Session async for conversation management
437
-
438
- **Works with:**
439
- - All 6 providers (OpenAI, Anthropic, Ollama, LMStudio, MLX, HuggingFace)
440
- - Streaming via `async for chunk in llm.agenerate(..., stream=True)`
441
- - Sessions via `await session.agenerate(...)`
442
- - Zero breaking changes to sync API
443
-
444
- **Learn async patterns:**
445
-
446
- AbstractCore includes an educational [async CLI demo](examples/async_cli_demo.py) that demonstrates 8 core async/await patterns:
447
- - Event-driven progress with GlobalEventBus
448
- - Parallel tool execution with asyncio.gather()
449
- - Proper async streaming pattern (await first, then async for)
450
- - Non-blocking animations and user input
451
-
452
- ```bash
453
- # Try the educational async demo
454
- python examples/async_cli_demo.py --provider ollama --model qwen3:4b --stream
455
- ```
456
-
457
- [Learn more in CLI docs](docs/acore-cli.md#async-cli-demo-educational-reference)
458
-
459
- ### Media Handling
460
-
461
- AbstractCore provides unified media handling across all providers with automatic resolution optimization. Upload images, PDFs, and documents using the same simple API regardless of your provider.
462
-
463
- ```python
464
- from abstractcore import create_llm
465
-
466
- # Vision analysis - works with any vision model
467
- # Images automatically processed at maximum supported resolution
468
- llm = create_llm("openai", model="gpt-4o")
469
- response = llm.generate(
470
- "What's in this image?",
471
- media=["photo.jpg"] # Auto-resized to model's maximum capability
472
- )
473
-
474
- # Document analysis - works with any model
475
- llm = create_llm("anthropic", model="claude-3.5-sonnet")
476
- response = llm.generate(
477
- "Summarize this research paper",
478
- media=["research_paper.pdf"]
479
- )
480
-
481
- # Multiple files - mix images, PDFs, spreadsheets
482
- response = llm.generate(
483
- "Analyze these business documents",
484
- media=["report.pdf", "chart.png", "data.xlsx"]
485
- )
486
-
487
- # Same code works with local models
488
- llm = create_llm("ollama", model="qwen3-vl:8b")
489
- response = llm.generate(
490
- "Describe this screenshot",
491
- media=["screenshot.png"] # Auto-optimized for qwen3-vl
492
- )
493
- ```
494
-
495
- **Key Features:**
496
- - **Smart Resolution**: Automatically uses maximum resolution supported by each model
497
- - **Format Support**: PNG, JPEG, GIF, WEBP, BMP, TIFF images; PDF, TXT, MD, CSV, TSV, JSON documents
498
- - **Office Documents**: DOCX, XLSX, PPT (with `pip install abstractcore[all]`)
499
- - **Vision Optimization**: Model-specific image processing for vision results
500
-
501
- **Provider compatibility:**
502
- - **High-resolution vision**: GPT-4o (up to 4096x4096), Claude 3.5 Sonnet (up to 1568x1568)
503
- - **Local models**: qwen3-vl (up to 3584x3584), gemma3:4b, llama3.2-vision
504
- - **All models**: Automatic text extraction for non-vision models
505
-
506
- [Learn more about Media Handling](docs/media-handling-system.md)
507
-
508
- ### Glyph Visual-Text Compression (🧪 EXPERIMENTAL)
509
-
510
- > ⚠️ **Vision Model Requirement**: This feature ONLY works with vision-capable models (e.g., gpt-4o, claude-3-5-sonnet, llama3.2-vision)
511
-
512
- Achieve **3-4x token compression** and **faster inference** with Glyph's revolutionary visual-text compression:
513
-
514
- ```python
515
- from abstractcore import create_llm
516
-
517
- # IMPORTANT: Requires a vision-capable model
518
- llm = create_llm("ollama", model="llama3.2-vision:11b") # ✓ Vision model
519
-
520
- # Large documents are automatically compressed for efficiency
521
- response = llm.generate(
522
- "Analyze the key findings in this research paper",
523
- media=["large_research_paper.pdf"] # Automatically compressed if beneficial
524
- )
525
-
526
- # Force compression (raises error if model lacks vision)
527
- response = llm.generate(
528
- "Summarize this document",
529
- media=["document.pdf"],
530
- glyph_compression="always" # "auto" | "always" | "never"
531
- )
532
-
533
- # Non-vision models will raise UnsupportedFeatureError
534
- # llm_no_vision = create_llm("openai", model="gpt-4") # ✗ No vision
535
- # response = llm_no_vision.generate("...", glyph_compression="always") # Error!
536
-
537
- # Check compression stats
538
- if response.metadata and response.metadata.get('compression_used'):
539
- stats = response.metadata.get('compression_stats', {})
540
- print(f"Compression ratio: {stats.get('compression_ratio')}x")
541
- print(f"Processing speedup: 14% faster, 79% less memory")
542
- ```
543
-
544
- **Validated Performance:**
545
- - **14% faster processing** with real-world documents
546
- - **79% lower memory usage** during processing
547
- - **100% quality preservation** - no loss of analytical accuracy
548
- - **Transparent operation** - works with existing code
549
-
550
- [Learn more about Glyph Compression](docs/glyphs.md)
551
-
552
- ## Key Features
553
-
554
- - **Offline-First Design**: Built primarily for open source LLMs with full offline capability. Download once, run forever without internet access
555
- - **Provider Agnostic**: Seamlessly switch between OpenAI, Anthropic, Ollama, LMStudio, MLX, HuggingFace, vLLM, and any OpenAI-compatible endpoint
556
- - **Async/Await Support** ⭐ NEW in v2.6.0: Native async support for concurrent requests with `asyncio.gather()` - works with all providers
557
- - **Dynamic Endpoint Configuration** ⭐ NEW in v2.6.5: Pass `base_url` in POST requests to connect to custom OpenAI-compatible endpoints without environment variables
558
- - **Interaction Tracing**: Complete LLM observability with programmatic access to prompts, responses, tokens, timing, and trace correlation for debugging, trust, and compliance
559
- - **Glyph Visual-Text Compression**: Revolutionary compression system that renders text as optimized images for 3-4x token compression and faster inference
560
- - **Centralized Configuration**: Global defaults and app-specific preferences at `~/.abstractcore/config/abstractcore.json`
561
- - **Intelligent Media Handling**: Upload images, PDFs, and documents with automatic maximum resolution optimization
562
- - **Vision Model Support**: Smart image processing at each model's maximum capability
563
- - **Document Processing**: PDF extraction (PyMuPDF4LLM), Office documents (DOCX/XLSX/PPT), CSV/TSV analysis
564
- - **Unified Tools**: Consistent tool calling across all providers
565
- - **Session Management**: Persistent conversations with metadata, analytics, and complete serialization
566
- - **Native Structured Output**: Server-side schema enforcement for Ollama and LMStudio (OpenAI and Anthropic also supported)
567
- - **Streaming Support**: Real-time token generation for interactive experiences
568
- - **Consistent Token Terminology**: Unified `input_tokens`, `output_tokens`, `total_tokens` across all providers
569
- - **Embeddings**: Built-in support for semantic search and RAG applications
570
- - **Universal Server**: Optional OpenAI-compatible API server with `/v1/responses` endpoint
571
-
572
- ## Supported Providers
573
-
574
- | Provider | Status | SEED Support | Hardware | Setup |
575
- |----------|--------|-------------|----------|-------|
576
- | **OpenAI** | Full | Native | Any | [Get API key](docs/prerequisites.md#openai-setup) |
577
- | **Anthropic** | Full | Warning* | Any | [Get API key](docs/prerequisites.md#anthropic-setup) |
578
- | **Ollama** | Full | Native | Any | [Install guide](docs/prerequisites.md#ollama-setup) |
579
- | **LMStudio** | Full | Native | Any | [Install guide](docs/prerequisites.md#lmstudio-setup) |
580
- | **MLX** | Full | Native | **Apple Silicon only** | [Setup guide](docs/prerequisites.md#mlx-setup) |
581
- | **HuggingFace** | Full | Native | Any | [Setup guide](docs/prerequisites.md#huggingface-setup) |
582
- | **vLLM** | Full | Native | **NVIDIA CUDA only** | [Setup guide](docs/prerequisites.md#vllm-setup) |
583
- | **OpenAI-Compatible** ⭐ NEW | Full | Native | Any | Works with llama.cpp, text-generation-webui, LocalAI, etc. |
584
-
585
- *Anthropic doesn't support seed parameters but issues a warning when provided. Use `temperature=0.0` for more consistent outputs.
586
-
587
- ## Server Mode (Optional HTTP REST API)
588
-
589
- AbstractCore is **primarily a Python library**. The server is an **optional component** that provides OpenAI-compatible HTTP endpoints:
590
-
591
- ```bash
592
- # Install with server support
593
- pip install abstractcore[server]
594
-
595
- # Start the server
596
- uvicorn abstractcore.server.app:app --host 0.0.0.0 --port 8000
597
- ```
598
-
599
- Use with any OpenAI-compatible client:
600
-
601
- ```python
602
- from openai import OpenAI
603
-
604
- client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
605
- response = client.chat.completions.create(
606
- model="anthropic/claude-3-5-haiku-latest",
607
- messages=[{"role": "user", "content": "Hello!"}]
608
- )
609
- ```
610
-
611
- **Server Features:**
612
- - OpenAI-compatible REST endpoints (`/v1/chat/completions`, `/v1/embeddings`, `/v1/responses`)
613
- - **NEW in v2.5.0**: OpenAI Responses API (`/v1/responses`) with native `input_file` support
614
- - Multi-provider support through one HTTP API
615
- - Comprehensive media processing (images, PDFs, Office documents, CSV/TSV)
616
- - Agentic CLI integration (Codex, Crush, Gemini CLI)
617
- - Streaming responses with optional opt-in
618
- - Tool call format conversion
619
- - Enhanced debug logging with `--debug` flag
620
- - Interactive API docs at `/docs` (Swagger UI)
621
-
622
- **When to use the server:**
623
- - Integrating with existing OpenAI-compatible tools
624
- - Using agentic CLIs (Codex, Crush, Gemini CLI)
625
- - Building web applications that need HTTP API
626
- - Multi-language access (not just Python)
627
-
628
- ## AbstractCore CLI (Optional Interactive Testing Tool)
629
-
630
- AbstractCore includes a **built-in CLI** for interactive testing, development, and conversation management. This is an internal testing tool, distinct from external agentic CLIs.
631
-
632
- ```bash
633
- # Start interactive CLI
634
- python -m abstractcore.utils.cli --provider ollama --model qwen3-coder:30b
635
-
636
- # With streaming enabled
637
- python -m abstractcore.utils.cli --provider openai --model gpt-4o-mini --stream
638
-
639
- # Single prompt execution
640
- python -m abstractcore.utils.cli --provider anthropic --model claude-3-5-haiku-latest --prompt "What is Python?"
641
- ```
642
-
643
- **Key Features:**
644
- - Interactive REPL with conversation history
645
- - Chat history compaction and management
646
- - Fact extraction from conversations
647
- - Conversation quality evaluation (LLM-as-a-judge)
648
- - Intent analysis and deception detection
649
- - Tool call testing and debugging
650
- - System prompt management
651
- - Multiple provider support
652
-
653
- **Popular Commands:**
654
- - `/compact` - Compress chat history while preserving context
655
- - `/facts [file]` - Extract structured facts from conversation
656
- - `/judge` - Evaluate conversation quality with feedback
657
- - `/intent [participant]` - Analyze psychological intents and detect deception
658
- - `/history [n]` - View conversation history
659
- - `/stream` - Toggle real-time streaming
660
- - `/system [prompt]` - Show or change system prompt
661
- - `/status` - Show current provider, model, and capabilities
662
-
663
- **Full Documentation:** [AbstractCore CLI Guide](docs/acore-cli.md)
664
-
665
- **When to use the CLI:**
666
- - Interactive development and testing
667
- - Debugging tool calls and provider behavior
668
- - Conversation management experiments
669
- - Quick prototyping with different models
670
- - Learning AbstractCore capabilities
671
-
672
- ## Built-in Applications (Ready-to-Use CLI Tools)
673
-
674
- AbstractCore includes **five specialized command-line applications** for common LLM tasks. These are production-ready tools that can be used directly from the terminal without any Python programming.
675
-
676
- ### Available Applications
677
-
678
- | Application | Purpose | Direct Command |
679
- |-------------|---------|----------------|
680
- | **Summarizer** | Document summarization | `summarizer` |
681
- | **Extractor** | Entity and relationship extraction | `extractor` |
682
- | **Judge** | Text evaluation and scoring | `judge` |
683
- | **Intent Analyzer** | Psychological intent analysis & deception detection | `intent` |
684
- | **DeepSearch** | Autonomous multi-stage research with web search | `deepsearch` |
685
-
686
- ### Quick Usage Examples
687
-
688
- ```bash
689
- # Document summarization with different styles and lengths
690
- summarizer document.pdf --style executive --length brief
691
- summarizer report.txt --focus "technical details" --output summary.txt
692
- summarizer large_doc.txt --chunk-size 15000 --provider openai --model gpt-4o-mini
693
-
694
- # Entity extraction with various formats and options
695
- extractor research_paper.pdf --format json-ld --focus technology
696
- extractor article.txt --entity-types person,organization,location --output entities.jsonld
697
- extractor doc.txt --iterate 3 --mode thorough --verbose
698
-
699
- # Text evaluation with custom criteria and contexts
700
- judge essay.txt --criteria clarity,accuracy,coherence --context "academic writing"
701
- judge code.py --context "code review" --format plain --verbose
702
- judge proposal.md --custom-criteria has_examples,covers_risks --output assessment.json
703
-
704
- # Intent analysis with psychological insights and deception detection
705
- intent conversation.txt --focus-participant user --depth comprehensive
706
- intent email.txt --format plain --context document --verbose
707
- intent chat_log.json --conversation-mode --provider lmstudio --model qwen/qwen3-30b-a3b-2507
708
-
709
- # Autonomous research with web search and reflexive refinement
710
- deepsearch "What are the latest advances in quantum computing?" --depth comprehensive
711
- deepsearch "AI impact on healthcare" --focus "diagnosis,treatment,ethics" --reflexive
712
- deepsearch "sustainable energy 2025" --max-sources 25 --provider openai --model gpt-4o-mini
713
- ```
714
-
715
- ### Installation & Setup
716
-
717
- Apps are automatically available after installing AbstractCore:
718
-
719
- ```bash
720
- # Install with all features
721
- pip install abstractcore[all]
722
-
723
- # Apps are immediately available
724
- summarizer --help
725
- extractor --help
726
- judge --help
727
- intent --help
728
- deepsearch --help
729
- ```
730
-
731
- ### Alternative Usage Methods
732
-
733
- ```bash
734
- # Method 1: Direct commands (recommended)
735
- summarizer document.txt
736
- extractor report.pdf
737
- judge essay.md
738
- intent conversation.txt
739
- deepsearch "your research query"
740
-
741
- # Method 2: Via Python module
742
- python -m abstractcore.apps summarizer document.txt
743
- python -m abstractcore.apps extractor report.pdf
744
- python -m abstractcore.apps judge essay.md
745
- python -m abstractcore.apps intent conversation.txt
746
- python -m abstractcore.apps deepsearch "your research query"
747
- ```
748
-
749
- ### Key Parameters
750
-
751
- **Common Parameters (all apps):**
752
- - `--provider` + `--model` - Use different LLM providers (OpenAI, Anthropic, Ollama, etc.)
753
- - `--output` - Save results to file instead of console
754
- - `--verbose` - Show detailed progress information
755
- - `--timeout` - HTTP timeout for LLM requests (default: 300s)
756
-
757
- **Summarizer Parameters:**
758
- - `--style` - Summary style: `structured`, `narrative`, `objective`, `analytical`, `executive`, `conversational`
759
- - `--length` - Summary length: `brief`, `standard`, `detailed`, `comprehensive`
760
- - `--focus` - Specific focus area for summarization
761
- - `--chunk-size` - Chunk size for large documents (1000-32000, default: 8000)
762
-
763
- **Extractor Parameters:**
764
- - `--format` - Output format: `json-ld`, `triples`, `json`, `yaml`
765
- - `--entity-types` - Focus on specific entities: `person,organization,location,technology,etc.`
766
- - `--mode` - Extraction mode: `fast`, `balanced`, `thorough`
767
- - `--iterate` - Number of refinement iterations (1-10, default: 1)
768
- - `--minified` - Output compact JSON without indentation
769
-
770
- **Judge Parameters:**
771
- - `--context` - Evaluation context (e.g., "code review", "academic writing")
772
- - `--criteria` - Standard criteria: `clarity,soundness,effectiveness,etc.`
773
- - `--custom-criteria` - Custom evaluation criteria
774
- - `--format` - Output format: `json`, `plain`, `yaml`
775
- - `--include-criteria` - Include detailed criteria explanations
776
-
777
- ### Key Features
778
-
779
- - **Provider Agnostic**: Works with any configured LLM provider (OpenAI, Anthropic, Ollama, etc.)
780
- - **Multiple Formats**: Support for PDF, TXT, MD, DOCX, and more
781
- - **Flexible Output**: JSON, JSON-LD, YAML, plain text formats
782
- - **Batch Processing**: Process multiple files at once
783
- - **Configurable**: Custom prompts, criteria, and evaluation rubrics
784
- - **Production Ready**: Robust error handling and logging
785
-
786
- ### Full Documentation
787
-
788
- Each application has documentation with examples and usage information:
789
-
790
- - **[Summarizer Guide](docs/apps/basic-summarizer.md)** - Document summarization with multiple strategies
791
- - **[Extractor Guide](docs/apps/basic-extractor.md)** - Entity and relationship extraction
792
- - **[Intent Analyzer Guide](docs/apps/basic-intent.md)** - Psychological intent analysis and deception detection
793
- - **[Judge Guide](docs/apps/basic-judge.md)** - Text evaluation and scoring systems
794
- - **[DeepSearch Guide](docs/apps/basic-deepsearch.md)** - Autonomous multi-stage research with web search
795
-
796
- **When to use the apps:**
797
- - Processing documents without writing code
798
- - Batch text analysis workflows
799
- - Quick prototyping of text processing pipelines
800
- - Integration with shell scripts and automation
801
- - Standardized text processing tasks
802
-
803
- ## Configuration
804
-
805
- AbstractCore provides a **centralized configuration system** that manages default models, cache directories, and logging settings from a single location. This eliminates the need to specify `--provider` and `--model` parameters repeatedly.
806
-
807
- ### Quick Setup
808
-
809
- ```bash
810
- # Check current configuration (shows how to change each setting)
811
- abstractcore --status
812
-
813
- # Set defaults for all applications
814
- abstractcore --set-global-default ollama/llama3:8b
815
-
816
- # Or configure specific applications (examples of customization)
817
- abstractcore --set-app-default summarizer openai gpt-4o-mini
818
- abstractcore --set-app-default extractor ollama qwen3:4b-instruct
819
- abstractcore --set-app-default judge anthropic claude-3-5-haiku
820
-
821
- # Configure logging (common examples)
822
- abstractcore --set-console-log-level WARNING # Reduce console output
823
- abstractcore --set-console-log-level NONE # Disable console logging
824
- abstractcore --enable-file-logging # Save logs to files
825
- abstractcore --enable-debug-logging # Full debug mode
826
-
827
- # Configure vision for image analysis with text-only models
828
- abstractcore --set-vision-provider ollama qwen2.5vl:7b
829
- abstractcore --set-vision-provider lmstudio qwen/qwen3-vl-4b
830
-
831
- # Set API keys as needed
832
- abstractcore --set-api-key openai sk-your-key-here
833
- abstractcore --set-api-key anthropic your-anthropic-key
834
-
835
- # Verify configuration (includes change commands for each setting)
836
- abstractcore --status
837
- ```
838
-
839
- ### Priority System
840
-
841
- AbstractCore uses a clear priority system where explicit parameters always override defaults:
842
-
843
- 1. **Explicit parameters** (highest priority): `summarizer doc.txt --provider openai --model gpt-4o-mini`
844
- 2. **App-specific config**: `abstractcore --set-app-default summarizer openai gpt-4o-mini`
845
- 3. **Global config**: `abstractcore --set-global-default openai/gpt-4o-mini`
846
- 4. **Built-in defaults** (lowest priority): `huggingface/unsloth/Qwen3-4B-Instruct-2507-GGUF`
847
-
848
- ### Usage After Configuration
849
-
850
- Once configured, apps use your defaults automatically:
851
-
852
- ```bash
853
- # Before configuration (requires explicit parameters)
854
- summarizer document.pdf --provider openai --model gpt-4o-mini
855
-
856
- # After configuration (uses configured defaults)
857
- summarizer document.pdf
858
-
859
- # Explicit parameters still override when needed
860
- summarizer document.pdf --provider anthropic --model claude-3-5-sonnet
861
- ```
862
-
863
- ### Configuration Features
864
-
865
- - **Application defaults**: Different optimal models for each app
866
- - **Cache directories**: Configurable cache locations for models and data
867
- - **Logging control**: Package-wide logging levels and debug mode
868
- - **API key management**: Centralized API key storage
869
- - **Interactive setup**: `abstractcore --configure` for guided configuration
870
-
871
- **Complete guide**: [Centralized Configuration](docs/centralized-config.md)
872
-
873
- ### Environment Variables
874
-
875
- AbstractCore supports environment variables for provider base URLs, enabling remote servers, Docker deployments, and non-standard ports:
876
-
877
- ```bash
878
- # Ollama on remote server
879
- export OLLAMA_BASE_URL="http://192.168.1.100:11434"
880
- # Alternative: OLLAMA_HOST is also supported
881
- export OLLAMA_HOST="http://192.168.1.100:11434"
882
-
883
- # LMStudio on non-standard port
884
- export LMSTUDIO_BASE_URL="http://localhost:1235/v1"
885
-
886
- # OpenAI-compatible proxy
887
- export OPENAI_BASE_URL="https://api.portkey.ai/v1"
888
-
889
- # Anthropic proxy
890
- export ANTHROPIC_BASE_URL="https://api.portkey.ai/v1"
891
- ```
892
-
893
- **Priority**: Programmatic `base_url` parameter > Runtime configuration > Environment variable > Default value
894
-
895
- **Provider discovery**: `get_all_providers_with_models()` automatically respects these environment variables when checking provider availability.
896
-
897
- ### Programmatic Configuration
898
-
899
- Configure provider settings at runtime without environment variables:
900
-
901
- ```python
902
- from abstractcore.config import configure_provider, get_provider_config, clear_provider_config
903
- from abstractcore import create_llm
904
-
905
- # Set provider base URL programmatically
906
- configure_provider('ollama', base_url='http://192.168.1.100:11434')
907
-
908
- # All future create_llm() calls automatically use the configured URL
909
- llm = create_llm('ollama', model='llama3:8b') # Uses http://192.168.1.100:11434
910
-
911
- # Query current configuration
912
- config = get_provider_config('ollama')
913
- print(config) # {'base_url': 'http://192.168.1.100:11434'}
914
-
915
- # Clear configuration (revert to env var / default)
916
- configure_provider('ollama', base_url=None)
917
- # Or clear all providers
918
- clear_provider_config()
919
- ```
920
-
921
- **Use Cases**:
922
- - **Web UI Settings**: Configure providers through settings pages
923
- - **Docker Startup**: Read from custom env vars and configure programmatically
924
- - **Testing**: Set mock server URLs for integration tests
925
- - **Multi-tenant**: Configure different base URLs per tenant
926
-
927
- **Priority System**:
928
- 1. Constructor parameter (highest): `create_llm("ollama", base_url="...")`
929
- 2. Runtime configuration: `configure_provider('ollama', base_url="...")`
930
- 3. Environment variable: `OLLAMA_BASE_URL`
931
- 4. Default value (lowest): `http://localhost:11434`
932
-
933
- ## Documentation
934
-
935
- **📚 Complete Documentation:** [docs/](docs/) - Full documentation index and navigation guide
936
-
937
- ### Getting Started
938
- - **[Prerequisites & Setup](docs/prerequisites.md)** - Install and configure providers (OpenAI, Anthropic, Ollama, etc.)
939
- - **[Getting Started Guide](docs/getting-started.md)** - 5-minute quick start with core concepts
940
- - **[Troubleshooting](docs/troubleshooting.md)** - Common issues and solutions
941
-
942
- ### Core Library (Python)
943
- - **[Python API Reference](docs/api-reference.md)** - Complete Python API documentation
944
- - **[Media Handling System](docs/media-handling-system.md)** - Images, PDFs, and document processing across all providers
945
- - **[Session Management](docs/session.md)** - Persistent conversations, serialization, and analytics
946
- - **[Embeddings Guide](docs/embeddings.md)** - Semantic search, RAG, and vector embeddings
947
- - **[Code Examples](examples/)** - Working examples for all features
948
- - **[Capabilities](docs/capabilities.md)** - What AbstractCore can and cannot do
949
-
950
- ### Server (Optional HTTP REST API)
951
- - **[Server Documentation](docs/server.md)** - Complete server setup, API reference, and deployment
952
-
953
- ### Architecture & Advanced
954
- - **[Architecture](docs/architecture.md)** - System design and architecture overview
955
- - **[Tool Calling](docs/tool-calling.md)** - Universal tool system and format conversion
956
-
957
- ## Use Cases
958
-
959
- ### 1. Provider Flexibility
960
-
961
- ```python
962
- # Same code works with any provider
963
- providers = ["openai", "anthropic", "ollama"]
964
-
965
- for provider in providers:
966
- llm = create_llm(provider, model="gpt-4o-mini") # Auto-selects appropriate model
967
- response = llm.generate("Hello!")
968
- ```
969
-
970
- ### 2. Vision Analysis Across Providers
971
-
972
- ```python
973
- # Same image analysis works with any vision model
974
- image_files = ["product_photo.jpg", "user_feedback.png"]
975
- prompt = "Analyze these product images and suggest improvements"
976
-
977
- # OpenAI GPT-4o
978
- openai_llm = create_llm("openai", model="gpt-4o")
979
- openai_analysis = openai_llm.generate(prompt, media=image_files)
980
-
981
- # Anthropic Claude
982
- claude_llm = create_llm("anthropic", model="claude-3.5-sonnet")
983
- claude_analysis = claude_llm.generate(prompt, media=image_files)
984
-
985
- # Local model (free)
986
- local_llm = create_llm("ollama", model="qwen3-vl:8b")
987
- local_analysis = local_llm.generate(prompt, media=image_files)
988
- ```
989
-
990
- ### 3. Document Processing Pipeline
991
-
992
- ```python
993
- # Universal document analysis
994
- documents = ["contract.pdf", "financial_data.xlsx", "presentation.ppt"]
995
- analysis_prompt = "Extract key information and identify potential risks"
996
-
997
- # Works with any provider
998
- llm = create_llm("anthropic", model="claude-3.5-sonnet")
999
- response = llm.generate(analysis_prompt, media=documents)
1000
-
1001
- # Automatic format handling:
1002
- # - PDF: Text extraction with PyMuPDF4LLM
1003
- # - Excel: Table parsing with pandas
1004
- # - PowerPoint: Slide content extraction with unstructured
1005
- ```
1006
-
1007
- ### 4. Local Development, Cloud Production
1008
-
1009
- ```python
1010
- # Development (free, local)
1011
- llm_dev = create_llm("ollama", model="qwen3:4b-instruct-2507-q4_K_M")
1012
-
1013
- # Production (high quality, cloud)
1014
- llm_prod = create_llm("openai", model="gpt-4o-mini")
1015
- ```
1016
-
1017
- ### 5. Embeddings & RAG
1018
-
1019
- ```python
1020
- from abstractcore.embeddings import EmbeddingManager
1021
-
1022
- # Create embeddings for semantic search
1023
- embedder = EmbeddingManager()
1024
- docs_embeddings = embedder.embed_batch([
1025
- "Python is great for data science",
1026
- "JavaScript powers the web",
1027
- "Rust ensures memory safety"
1028
- ])
1029
-
1030
- # Find most similar document
1031
- query_embedding = embedder.embed("Tell me about web development")
1032
- similarity = embedder.compute_similarity(query, docs[0])
1033
- ```
1034
-
1035
- [Learn more about Embeddings](docs/embeddings.md)
1036
-
1037
- ### 6. Structured Output
1038
-
1039
- ```python
1040
- from pydantic import BaseModel
1041
-
1042
- class MovieReview(BaseModel):
1043
- title: str
1044
- rating: int # 1-5
1045
- summary: str
1046
-
1047
- llm = create_llm("openai", model="gpt-4o-mini")
1048
- review = llm.generate(
1049
- "Review the movie Inception",
1050
- response_model=MovieReview
1051
- )
1052
- print(f"{review.title}: {review.rating}/5")
1053
- ```
1054
-
1055
- [Learn more about Structured Output](docs/structured-output.md)
1056
-
1057
- ### 7. Universal API Server
1058
-
1059
- ```bash
1060
- # Start server once
1061
- uvicorn abstractcore.server.app:app --port 8000
1062
-
1063
- # Use with any OpenAI client
1064
- curl -X POST http://localhost:8000/v1/chat/completions \
1065
- -H "Content-Type: application/json" \
1066
- -d '{
1067
- "model": "ollama/qwen3-coder:30b",
1068
- "messages": [{"role": "user", "content": "Write a Python function"}]
1069
- }'
1070
- ```
1071
-
1072
- ## Why AbstractCore?
1073
-
1074
- - **Offline-First Philosophy**: Designed for open source LLMs with complete offline operation. No internet required after initial model download
1075
- - **Unified Interface**: One API for all LLM providers
1076
- - **Multimodal Support**: Upload images, PDFs, and documents across all providers
1077
- - **Vision Models**: Seamless integration with GPT-4o, Claude Vision, qwen3-vl, and more
1078
- - **Production Ready**: Robust error handling, retries, timeouts
1079
- - **Type Safe**: Full Pydantic integration for structured outputs
1080
- - **Local & Cloud**: Run models locally or use cloud APIs
1081
- - **Tool Calling**: Consistent function calling across providers
1082
- - **Streaming**: Real-time responses for interactive applications
1083
- - **Embeddings**: Built-in vector embeddings for RAG
1084
- - **Server Mode**: Optional OpenAI-compatible API server
1085
- - **Well Documented**: Comprehensive guides and examples
1086
-
1087
- ## Installation Options
1088
-
1089
- ```bash
1090
- # Minimal core
1091
- pip install abstractcore
1092
-
1093
- # With media handling (images, PDFs, documents)
1094
- pip install abstractcore[media]
1095
-
1096
- # With specific providers
1097
- pip install abstractcore[openai]
1098
- pip install abstractcore[anthropic]
1099
- pip install abstractcore[ollama]
1100
- pip install abstractcore[lmstudio]
1101
- pip install abstractcore[huggingface]
1102
- pip install abstractcore[mlx] # macOS/Apple Silicon only
1103
- pip install abstractcore[vllm] # NVIDIA CUDA only (Linux)
1104
-
1105
- # With server support
1106
- pip install abstractcore[server]
1107
-
1108
- # With embeddings
1109
- pip install abstractcore[embeddings]
1110
-
1111
- # With compression (Glyph visual-text compression)
1112
- pip install abstractcore[compression]
1113
-
1114
- # Everything (recommended for Apple Silicon)
1115
- pip install abstractcore[all]
1116
-
1117
- # Cross-platform (all except MLX/vLLM - for Linux/Windows/Intel Mac)
1118
- pip install abstractcore[all-non-mlx]
1119
-
1120
- # Provider groups
1121
- pip install abstractcore[all-providers] # All providers (includes MLX, excludes vLLM)
1122
- pip install abstractcore[all-providers-non-mlx] # All providers except MLX (excludes vLLM)
1123
- pip install abstractcore[local-providers] # Ollama, LMStudio, MLX
1124
- pip install abstractcore[local-providers-non-mlx] # Ollama, LMStudio only
1125
- pip install abstractcore[api-providers] # OpenAI, Anthropic
1126
- pip install abstractcore[gpu-providers] # vLLM (NVIDIA CUDA only)
1127
- ```
1128
-
1129
- **Hardware-Specific Notes:**
1130
- - **MLX**: Requires Apple Silicon (M1/M2/M3/M4). Will not work on Intel Macs or other platforms.
1131
- - **vLLM**: Requires NVIDIA GPUs with CUDA support. Will not work on Apple Silicon, AMD GPUs, or Intel integrated graphics.
1132
- - **All other providers** (OpenAI, Anthropic, Ollama, LMStudio, HuggingFace): Work on any hardware.
1133
-
1134
- **Media processing extras:**
1135
- ```bash
1136
- # For PDF processing
1137
- pip install pymupdf4llm
1138
-
1139
- # For Office documents (DOCX, XLSX, PPT)
1140
- pip install unstructured
1141
-
1142
- # For image optimization
1143
- pip install pillow
1144
-
1145
- # For data processing (CSV, Excel)
1146
- pip install pandas
1147
- ```
1148
-
1149
- ## Testing Status
1150
-
1151
- All tests passing as of October 12th, 2025.
1152
-
1153
- **Test Environment:**
1154
- - Hardware: MacBook Pro (14-inch, Nov 2024)
1155
- - Chip: Apple M4 Max
1156
- - Memory: 128 GB
1157
- - Python: 3.12.2
1158
-
1159
- ## Quick Links
1160
-
1161
- - **[📚 Documentation Index](docs/)** - Complete documentation navigation guide
1162
- - **[🔍 Interaction Tracing](docs/interaction-tracing.md)** - LLM observability and debugging ⭐ NEW
1163
- - **[Getting Started](docs/getting-started.md)** - 5-minute quick start
1164
- - **[⚙️ Prerequisites](docs/prerequisites.md)** - Provider setup (OpenAI, Anthropic, Ollama, etc.)
1165
- - **[📖 Python API](docs/api-reference.md)** - Complete Python API reference
1166
- - **[🌐 Server Guide](docs/server.md)** - HTTP API server setup
1167
- - **[🔧 Troubleshooting](docs/troubleshooting.md)** - Fix common issues
1168
- - **[💻 Examples](examples/)** - Working code examples
1169
- - **[🐛 Issues](https://github.com/lpalbou/AbstractCore/issues)** - Report bugs
1170
- - **[💬 Discussions](https://github.com/lpalbou/AbstractCore/discussions)** - Get help
1171
-
1172
- ## Contact
1173
- **Maintainer:** Laurent-Philippe Albou
1174
- 📧 Email: [contact@abstractcore.ai](mailto:contact@abstractcore.ai)
1175
-
1176
- ## Contributing
1177
-
1178
- We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
1179
-
1180
- ## License
1181
-
1182
- MIT License - see [LICENSE](LICENSE) file for details.
1183
-
1184
- ---
1185
-
1186
- **AbstractCore** - One interface, all LLM providers. Focus on building, not managing API differences.
1187
-
1188
- ---
1189
-
1190
- > **Migration Note**: This project was previously known as "AbstractLLM" and has been completely rebranded to "AbstractCore" as of version 2.4.0. See [CHANGELOG.md](CHANGELOG.md) for migration details.