ox-ai-workers 0.9.3.1 → 0.9.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.cursor/rules/010-project-structure.mdc +89 -0
- data/.cursor/rules/998-clean-code.mdc +52 -0
- data/.cursor/rules/999-mdc-format.mdc +132 -0
- data/CHANGELOG.md +16 -0
- data/README.md +193 -1
- data/config/locales/en.oxaiworkers.tool.yml +4 -4
- data/config/locales/ru.oxaiworkers.tool.yml +3 -3
- data/lib/ox-ai-workers.rb +9 -2
- data/lib/oxaiworkers/assistant/module_base.rb +8 -0
- data/lib/oxaiworkers/assistant/painter.rb +6 -3
- data/lib/oxaiworkers/iterator.rb +52 -6
- data/lib/oxaiworkers/models/deepseek_max.rb +1 -3
- data/lib/oxaiworkers/models/images_base.rb +11 -0
- data/lib/oxaiworkers/models/{module_base.rb → llm_base.rb} +1 -1
- data/lib/oxaiworkers/models/openai_dalle3.rb +47 -0
- data/lib/oxaiworkers/models/openai_gpt_image.rb +37 -0
- data/lib/oxaiworkers/models/openai_max.rb +1 -3
- data/lib/oxaiworkers/models/stability_images.rb +32 -0
- data/lib/oxaiworkers/module_request.rb +17 -4
- data/lib/oxaiworkers/tool/pixels.rb +23 -42
- data/lib/oxaiworkers/version.rb +1 -1
- metadata +9 -3
- data/.cursorrules +0 -155
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: '09fdb6954c507cc0d70433141a70a4974b42feb625719c525d2b7c093d9e1ce5'
|
4
|
+
data.tar.gz: e93b4e8040e831c73d8dd0379466f70e48eafd07513d7e78fee3f19985ef4331
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: c7b1f5a861cc8b4536fd22d6dbe479f0886ebb260317d585bf80ba19c0b1e5dbb6d9c5998ab1e25f367a112a5a7e3dce6bca1a48d2edecb9a6db8af3aa19b1bb
|
7
|
+
data.tar.gz: d7e64a09aa0351a62fdb74c52a25c99db79a4fd7c2765001cd715e8ac32deb6cd2b81aef5e385e57c66aa4b9fb3ef616145eb7953e5eefadc3321ae9678e697f
|
@@ -0,0 +1,89 @@
|
|
1
|
+
---
|
2
|
+
description:
|
3
|
+
globs:
|
4
|
+
alwaysApply: true
|
5
|
+
---
|
6
|
+
# Overview
|
7
|
+
|
8
|
+
OxAiWorkers is a Ruby gem that implements a finite state machine (FSM, using the `state_machine` gem) to solve tasks using generative intelligence. This approach enhances the final result by utilizing internal monologue and external tools.
|
9
|
+
|
10
|
+
## Core Components
|
11
|
+
|
12
|
+
- `Request` and `DelayedRequest` - classes for executing API requests (immediate and delayed)
|
13
|
+
- `ModuleRequest` - base class for all API requests with parsing and response handling ([module_request.rb](mdc:lib/oxaiworkers/module_request.rb))
|
14
|
+
- `Iterator` - main class for iterative task execution with tools ([iterator.rb](mdc:lib/oxaiworkers/iterator.rb))
|
15
|
+
- `Assistant::ModuleBase` - high-level wrappers over Iterator (Sysop, Coder, Localizer, etc.) ([module_base.rb](mdc:lib/oxaiworkers/assistant/module_base.rb))
|
16
|
+
- `Tool` - tools that can be used during task execution (Eval, FileSystem, Database, Pixels, Pipeline)
|
17
|
+
- `ToolDefinition` - module for declaring functions and methods for tools ([tool_definition.rb](mdc:lib/oxaiworkers/tool_definition.rb))
|
18
|
+
- `StateTools` - base class for managing states and transitions ([state_tools.rb](mdc:lib/oxaiworkers/state_tools.rb))
|
19
|
+
- `ContextualLogger` - logging system with contextual information support
|
20
|
+
|
21
|
+
## Code Conventions
|
22
|
+
|
23
|
+
- Use `snake_case` for method and variable names
|
24
|
+
- All code comments, CHANGELOG, README, and other documentation must be written in English
|
25
|
+
|
26
|
+
## Interaction Patterns
|
27
|
+
|
28
|
+
- The system uses internal monologue (inner_monologue) for planning actions
|
29
|
+
- External voice (outer_voice) is used for communication with the user
|
30
|
+
- Execution flow management through finite state machine
|
31
|
+
- Implementation of callback mechanisms for flexible event handling
|
32
|
+
|
33
|
+
## Tools Architecture
|
34
|
+
|
35
|
+
- Each tool should be a self-contained module
|
36
|
+
- Tools are registered through the `define_function` interface
|
37
|
+
- All tools should handle their own errors and return readable messages
|
38
|
+
- Handle errors at the tool level, preventing them from interrupting the main execution flow
|
39
|
+
|
40
|
+
## Finite State Machine Implementation
|
41
|
+
|
42
|
+
- Core FSM based on `state_machine` gem with states: idle → prepared → requested → analyzed → finished → idle
|
43
|
+
- State transitions managed by events: prepare, request, analyze, complete, iterate, end ([state_tools.rb](mdc:lib/oxaiworkers/state_tools.rb), [iterator.rb](mdc:lib/oxaiworkers/iterator.rb))
|
44
|
+
- `StateTools` - base class for FSM implementation with event hooks and transition callbacks ([state_tools.rb](mdc:lib/oxaiworkers/state_tools.rb))
|
45
|
+
- `StateBatch` - FSM extension for batch request processing with additional states
|
46
|
+
- Automatic error recovery and retry mechanisms for failed API requests
|
47
|
+
|
48
|
+
## Iterator Lifecycle
|
49
|
+
|
50
|
+
- 3 core functions: inner_monologue, outer_voice, finish_it ([iterator.rb](mdc:lib/oxaiworkers/iterator.rb))
|
51
|
+
- Configurable message queue for stateful conversation history
|
52
|
+
- Callback system for processing each state transition
|
53
|
+
- Context and milestone management for optimizing token usage
|
54
|
+
- Support for custom steps and instruction templating
|
55
|
+
|
56
|
+
## Assistants Details
|
57
|
+
|
58
|
+
- `ModuleBase` - shared functionality for all assistant types ([module_base.rb](mdc:lib/oxaiworkers/assistant/module_base.rb))
|
59
|
+
- `Sysop` - system administration and shell command execution ([sysop.rb](mdc:lib/oxaiworkers/assistant/sysop.rb), [file_system.rb](mdc:lib/oxaiworkers/tool/file_system.rb))
|
60
|
+
- `Coder` - specialized for code generation and analysis ([coder.rb](mdc:lib/oxaiworkers/assistant/coder.rb), [eval.rb](mdc:lib/oxaiworkers/tool/eval.rb))
|
61
|
+
- `Localizer` - translation and localization support ([localizer.rb](mdc:lib/oxaiworkers/assistant/localizer.rb))
|
62
|
+
- `Orchestrator` - Coordinates multiple assistants to work together on complex tasks ([orchestrator.rb](mdc:lib/oxaiworkers/assistant/orchestrator.rb), [pipeline.rb](mdc:lib/oxaiworkers/tool/pipeline.rb))
|
63
|
+
- `Painter` - Image generation and manipulation ([painter.rb](mdc:lib/oxaiworkers/assistant/painter.rb), [pixels.rb](mdc:lib/oxaiworkers/tool/pixels.rb))
|
64
|
+
|
65
|
+
## Internationalization and Localization
|
66
|
+
|
67
|
+
- All user-facing strings MUST be properly localized using I18n (config/locales/*.yml)
|
68
|
+
- Use I18n.t for all text that will be shown to users or appears in assistant prompts
|
69
|
+
- Store translations in YAML files within the config/locales directory
|
70
|
+
- Follow the naming convention of language.namespace.key (e.g., en.oxaiworkers.assistant.role)
|
71
|
+
- Use named parameters (%{variable}) instead of positional parameters (%s) in translation strings
|
72
|
+
- Use the with_locale method to ensure proper locale context when processing localized text
|
73
|
+
- Implement locale-aware classes by including the OxAiWorkers::LoadI18n module
|
74
|
+
- Store the current locale on initialization and preserve it across method calls
|
75
|
+
- Support multiple languages simultaneously through careful locale management
|
76
|
+
- Default to English for developer-facing messages and logs
|
77
|
+
- Ensure that all assistant classes properly handle localization in their format_role methods
|
78
|
+
|
79
|
+
## LoadI18n Module Usage
|
80
|
+
|
81
|
+
- The `OxAiWorkers::LoadI18n` module provides two key methods for localization:
|
82
|
+
- `store_locale` - saves the current locale at initialization time
|
83
|
+
- `with_locale` - executes a block of code in the context of the saved locale
|
84
|
+
- Always include the `OxAiWorkers::LoadI18n` module in classes that need localization capabilities
|
85
|
+
- Call `store_locale` in the initialization methods of locale-aware classes
|
86
|
+
- Wrap all locale-dependent code in `with_locale` blocks
|
87
|
+
- NEVER redefine the `with_locale` method in classes that include LoadI18n
|
88
|
+
- All methods that produce user-visible text must use the locale context via `with_locale` blocks
|
89
|
+
- Regular method calls from classes including LoadI18n do not require additional locale handling
|
@@ -0,0 +1,52 @@
|
|
1
|
+
---
|
2
|
+
description: Guidelines for writing clean, maintainable, and human-readable code. Apply these rules when writing or reviewing code to ensure consistency and quality.
|
3
|
+
globs:
|
4
|
+
alwaysApply: false
|
5
|
+
---
|
6
|
+
# Clean Code Guidelines
|
7
|
+
|
8
|
+
## Constants Over Magic Numbers
|
9
|
+
- Replace hard-coded values with named constants
|
10
|
+
- Use descriptive constant names that explain the value's purpose
|
11
|
+
- Keep constants at the top of the file or in a dedicated constants file
|
12
|
+
|
13
|
+
## Meaningful Names
|
14
|
+
- Variables, functions, and classes should reveal their purpose
|
15
|
+
- Names should explain why something exists and how it's used
|
16
|
+
- Avoid abbreviations unless they're universally understood
|
17
|
+
|
18
|
+
## Smart Comments
|
19
|
+
- Don't comment on what the code does - make the code self-documenting
|
20
|
+
- Use comments to explain why something is done a certain way
|
21
|
+
- Document APIs, complex algorithms, and non-obvious side effects
|
22
|
+
|
23
|
+
## Single Responsibility
|
24
|
+
- Each function should do exactly one thing
|
25
|
+
- Functions should be small and focused
|
26
|
+
- If a function needs a comment to explain what it does, it should be split
|
27
|
+
|
28
|
+
## DRY (Don't Repeat Yourself)
|
29
|
+
- Extract repeated code into reusable functions
|
30
|
+
- Share common logic through proper abstraction
|
31
|
+
- Maintain single sources of truth
|
32
|
+
|
33
|
+
## Clean Structure
|
34
|
+
- Keep related code together
|
35
|
+
- Organize code in a logical hierarchy
|
36
|
+
- Use consistent file and folder naming conventions
|
37
|
+
|
38
|
+
## Encapsulation
|
39
|
+
- Hide implementation details
|
40
|
+
- Expose clear interfaces
|
41
|
+
- Move nested conditionals into well-named functions
|
42
|
+
|
43
|
+
## Code Quality Maintenance
|
44
|
+
- Refactor continuously
|
45
|
+
- Fix technical debt early
|
46
|
+
- Leave code cleaner than you found it
|
47
|
+
- Follow the "Fail fast" principle for early error detection
|
48
|
+
|
49
|
+
## Version Control
|
50
|
+
- Write clear commit messages
|
51
|
+
- Make small, focused commits
|
52
|
+
- Use meaningful branch names
|
@@ -0,0 +1,132 @@
|
|
1
|
+
---
|
2
|
+
description:
|
3
|
+
globs: *.mdc,**/*.mdc
|
4
|
+
alwaysApply: false
|
5
|
+
---
|
6
|
+
# MDC File Format Guide
|
7
|
+
|
8
|
+
MDC (Markdown Configuration) files are used by Cursor to provide context-specific instructions to AI assistants. This guide explains how to create and maintain these files properly.
|
9
|
+
|
10
|
+
## File Structure
|
11
|
+
|
12
|
+
Each MDC file consists of two main parts:
|
13
|
+
|
14
|
+
1. **Frontmatter** - Configuration metadata at the top of the file
|
15
|
+
2. **Markdown Content** - The actual instructions in Markdown format
|
16
|
+
|
17
|
+
### Frontmatter
|
18
|
+
|
19
|
+
The frontmatter must be the first thing in the file and must be enclosed between triple-dash lines (`---`). Configuration should be based on the intended behavior:
|
20
|
+
|
21
|
+
```
|
22
|
+
---
|
23
|
+
# Configure your rule based on desired behavior:
|
24
|
+
|
25
|
+
description: Brief description of what the rule does
|
26
|
+
globs: **/*.js, **/*.ts # Optional: Comma-separated list, not an array
|
27
|
+
alwaysApply: false # Set to true for global rules
|
28
|
+
---
|
29
|
+
```
|
30
|
+
|
31
|
+
> **Important**: Despite the appearance, the frontmatter is not strictly YAML formatted. The `globs` field is a comma-separated list and should NOT include brackets `[]` or quotes `"`.
|
32
|
+
|
33
|
+
#### Guidelines for Setting Fields
|
34
|
+
|
35
|
+
- **description**: Should be agent-friendly and clearly describe when the rule is relevant. Format as `<topic>: <details>` for best results.
|
36
|
+
- **globs**:
|
37
|
+
- If a rule is only relevant in very specific situations, leave globs empty so it's loaded only when applicable to the user request.
|
38
|
+
- If the only glob would match all files (like `**/*`), leave it empty and set `alwaysApply: true` instead.
|
39
|
+
- Otherwise, be as specific as possible with glob patterns to ensure rules are only applied with relevant files.
|
40
|
+
- **alwaysApply**: Use sparingly for truly global guidelines.
|
41
|
+
|
42
|
+
#### Glob Pattern Examples
|
43
|
+
|
44
|
+
- **/*.js - All JavaScript files
|
45
|
+
- src/**/*.jsx - All JSX files in the src directory
|
46
|
+
- **/components/**/*.vue - All Vue files in any components directory
|
47
|
+
|
48
|
+
### Markdown Content
|
49
|
+
|
50
|
+
After the frontmatter, the rest of the file should be valid Markdown:
|
51
|
+
|
52
|
+
```markdown
|
53
|
+
# Title of Your Rule
|
54
|
+
|
55
|
+
## Section 1
|
56
|
+
- Guidelines and information
|
57
|
+
- Code examples
|
58
|
+
|
59
|
+
## Section 2
|
60
|
+
More detailed information...
|
61
|
+
```
|
62
|
+
|
63
|
+
## Special Features
|
64
|
+
|
65
|
+
### File References
|
66
|
+
|
67
|
+
You can reference other files from within an MDC file using the markdown link syntax:
|
68
|
+
|
69
|
+
```
|
70
|
+
[rule-name.mdc](mdc:location/of/the/rule.mdc)
|
71
|
+
```
|
72
|
+
|
73
|
+
When this rule is activated, the referenced file will also be included in the context.
|
74
|
+
|
75
|
+
### Code Blocks
|
76
|
+
|
77
|
+
Use fenced code blocks for examples:
|
78
|
+
|
79
|
+
````markdown
|
80
|
+
```javascript
|
81
|
+
// Example code
|
82
|
+
function example() {
|
83
|
+
return "This is an example";
|
84
|
+
}
|
85
|
+
```
|
86
|
+
````
|
87
|
+
|
88
|
+
## Best Practices
|
89
|
+
|
90
|
+
1. **Clear Organization**
|
91
|
+
- Use numbered prefixes (e.g., `01-workflow.mdc`) for sorting rules logically
|
92
|
+
- Place task-specific rules in the `tasks/` subdirectory
|
93
|
+
- Use descriptive filenames that indicate the rule's purpose
|
94
|
+
|
95
|
+
2. **Frontmatter Specificity**
|
96
|
+
- Be specific with glob patterns to ensure rules are only applied in relevant contexts
|
97
|
+
- Use `alwaysApply: true` for truly global guidelines
|
98
|
+
- Make descriptions clear and concise so AI knows when to apply the rule
|
99
|
+
|
100
|
+
3. **Content Structure**
|
101
|
+
- Start with a clear title (H1)
|
102
|
+
- Use hierarchical headings (H2, H3, etc.) to organize content
|
103
|
+
- Include examples where appropriate
|
104
|
+
- Keep instructions clear and actionable
|
105
|
+
|
106
|
+
4. **File Size Considerations**
|
107
|
+
- Keep files focused on a single topic or closely related topics
|
108
|
+
- Split very large rule sets into multiple files and link them with references
|
109
|
+
- Aim for under 300 lines per file when possible
|
110
|
+
|
111
|
+
## Usage in Cursor
|
112
|
+
|
113
|
+
When working with files in Cursor, rules are automatically applied when:
|
114
|
+
|
115
|
+
1. The file you're working on matches a rule's glob pattern
|
116
|
+
2. A rule has `alwaysApply: true` set in its frontmatter
|
117
|
+
3. The agent thinks the rule's description matches the user request
|
118
|
+
4. You explicitly reference a rule in a conversation with Cursor's AI
|
119
|
+
|
120
|
+
## Creating/Renaming/Removing Rules
|
121
|
+
|
122
|
+
- When a rule file is added/renamed/removed, update also the list under 010-workflow.mdc.
|
123
|
+
- When changs are made to multiple `mdc` files from a single request, review also [999-mdc-format]((mdc:.cursor/rules/999-mdc-format.mdc)) to consider whether to update it too.
|
124
|
+
|
125
|
+
## Updating Rules
|
126
|
+
|
127
|
+
When updating existing rules:
|
128
|
+
|
129
|
+
1. Maintain the frontmatter format
|
130
|
+
2. Keep the same glob patterns unless intentionally changing the rule's scope
|
131
|
+
3. Update the description if the purpose of the rule changes
|
132
|
+
4. Consider whether changes should propagate to related rules (e.g., CE versions)
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,19 @@
|
|
1
|
+
|
2
|
+
## [0.9.6] - 2025-05-10
|
3
|
+
|
4
|
+
- Added `add_file` for `Iterator` (only pdf for now)
|
5
|
+
- Added `add_image` for `Iterator`
|
6
|
+
- Added `add_file` and `add_image` for Assistants
|
7
|
+
- Added `call_stack` for `Iterator` and `ModuleRequest`
|
8
|
+
- Added `stop_double_calls` for `Iterator` and `ModuleRequest`
|
9
|
+
|
10
|
+
## [0.9.4] - 2025-05-08
|
11
|
+
|
12
|
+
- Added `stability_images` model
|
13
|
+
- Added `openai_gpt_image` model
|
14
|
+
- Added `openai_dalle3` model
|
15
|
+
- Added `access_token_stability` for configuration
|
16
|
+
|
1
17
|
## [0.9.3] - 2025-05-06
|
2
18
|
|
3
19
|
- Changed `max_tokens` to `max_completion_tokens` in `params`
|
data/README.md
CHANGED
@@ -154,14 +154,25 @@ steps << "Step 4. When the solution is ready, notify about it and wait for the u
|
|
154
154
|
# To retain the locale if you have assistants in different languages in your project.
|
155
155
|
store_locale # Optional
|
156
156
|
|
157
|
+
tool = MyTool.new
|
158
|
+
|
157
159
|
@iterator = OxAiWorkers::Iterator.new(
|
158
160
|
worker: init_worker(delayed: delayed, model: model),
|
159
161
|
role: 'You are a software agent inside my computer',
|
160
|
-
tools: [
|
162
|
+
tools: [tool],
|
161
163
|
locale: @locale || I18n.locale,
|
162
164
|
steps: steps,
|
163
165
|
# def_except: [:outer_voice], # It's except steps with that functions
|
164
166
|
# def_only: [:inner_monologue, :outer_voice], # Use it only with your steps
|
167
|
+
# Forced Function: Uses call_stack parameter to force the model to call functions in this exact order, one at a time
|
168
|
+
# call_stack: [
|
169
|
+
# OxAiWorkers::Iterator.full_function_name(:outer_voice),
|
170
|
+
# tool.full_function_name(:func1)
|
171
|
+
# ],
|
172
|
+
# Stop Double Calls: Uses stop_double_calls parameter to prevent the model from calling the same function twice in a row
|
173
|
+
# stop_double_calls: [
|
174
|
+
# tool.full_function_name(:func1)
|
175
|
+
# ],
|
165
176
|
on_inner_monologue: ->(text:) { puts "monologue: #{text}".colorize(:yellow) },
|
166
177
|
on_outer_voice: ->(text:) { puts "voice: #{text}".colorize(:green) }
|
167
178
|
)
|
@@ -385,6 +396,34 @@ class MyTool
|
|
385
396
|
end
|
386
397
|
```
|
387
398
|
|
399
|
+
### Working with Files and Images
|
400
|
+
|
401
|
+
You can easily add files and images to your assistants:
|
402
|
+
|
403
|
+
```ruby
|
404
|
+
# Add a PDF file
|
405
|
+
iterator.add_file(
|
406
|
+
pdf: File.read('document.pdf'),
|
407
|
+
filename: 'document.pdf',
|
408
|
+
text: 'Here is the document you requested'
|
409
|
+
)
|
410
|
+
|
411
|
+
# Add image from URL
|
412
|
+
iterator.add_image(
|
413
|
+
text: 'Here is the image',
|
414
|
+
url: 'https://example.com/image.jpg',
|
415
|
+
detail: 'auto' # 'auto', 'low', or 'high'
|
416
|
+
)
|
417
|
+
|
418
|
+
# Add image from binary data
|
419
|
+
image_data = File.read('local_image.jpg')
|
420
|
+
iterator.add_image(
|
421
|
+
text: 'Image from binary data',
|
422
|
+
binary: image_data,
|
423
|
+
mime_type: 'image/jpeg' # Defaults to 'image/jpeg'
|
424
|
+
)
|
425
|
+
```
|
426
|
+
|
388
427
|
### Handling State Transitions with Callbacks
|
389
428
|
|
390
429
|
You can track and respond to state transitions with callbacks:
|
@@ -438,6 +477,159 @@ OxAiWorkers provides several specialized assistant types:
|
|
438
477
|
localizer.task = "Translate my application's interface"
|
439
478
|
```
|
440
479
|
|
480
|
+
- **Painter**: Image generation and manipulation
|
481
|
+
|
482
|
+
```ruby
|
483
|
+
painter = OxAiWorkers::Assistant::Painter.new
|
484
|
+
# or Set working directory to save generated images as files
|
485
|
+
# painter = OxAiWorkers::Assistant::Painter.new(current_dir: Dir.pwd)
|
486
|
+
painter.task = "Create an image of a sunset over mountains"
|
487
|
+
```
|
488
|
+
|
489
|
+
- **Orchestrator**: Coordinates multiple assistants to work together on complex tasks
|
490
|
+
|
491
|
+
```ruby
|
492
|
+
orchestrator = OxAiWorkers::Assistant::Orchestrator.new(
|
493
|
+
workflow: 'Development team creates an application and tests it.'
|
494
|
+
)
|
495
|
+
orchestrator.add_assistant(OxAiWorkers::Assistant::Coder.new)
|
496
|
+
orchestrator.add_assistant(OxAiWorkers::Assistant::Sysop.new)
|
497
|
+
orchestrator.add_assistant(OxAiWorkers::Assistant::Localizer.new)
|
498
|
+
orchestrator.task = "Create a hello world application in C, save it to hello_world.c, compile, run, and verify it works."
|
499
|
+
```
|
500
|
+
|
501
|
+
All assistants support working with files and images:
|
502
|
+
|
503
|
+
```ruby
|
504
|
+
# Add files and images to any assistant
|
505
|
+
sysop.add_file(pdf: File.read('error_log.pdf'), filename: 'error_log.pdf', text: 'Error log file')
|
506
|
+
sysop.add_image(text: 'Screenshot of the error', url: 'https://example.com/screenshot.png')
|
507
|
+
```
|
508
|
+
|
509
|
+
See the [Working with Files and Images](#working-with-files-and-images) section for full details.
|
510
|
+
|
511
|
+
### Available Tools
|
512
|
+
|
513
|
+
OxAiWorkers provides several specialized tools to extend functionality:
|
514
|
+
|
515
|
+
- **Pixels**: Image generation and manipulation tool
|
516
|
+
|
517
|
+
```ruby
|
518
|
+
# Initialize with worker and optional parameters
|
519
|
+
pixels = OxAiWorkers::Tool::Pixels.new(
|
520
|
+
worker: worker, # Required: Request or DelayedRequest instance
|
521
|
+
current_dir: Dir.pwd, # Optional: Directory to save generated images
|
522
|
+
image_model: 'dall-e-3', # Optional: 'dall-e-3' or 'gpt-image-1'
|
523
|
+
only: [:generate_image] # Optional: Limit available functions
|
524
|
+
)
|
525
|
+
```
|
526
|
+
|
527
|
+
Provides functions for generating images with customizable parameters like size and quality, with ability to save generated images to disk.
|
528
|
+
|
529
|
+
- **Pipeline**: Assistant coordination and communication tool
|
530
|
+
|
531
|
+
```ruby
|
532
|
+
# Initialize with optional parameters
|
533
|
+
pipeline = OxAiWorkers::Tool::Pipeline.new(
|
534
|
+
on_message: ->(text:) { puts text } # Optional: Message handler callback
|
535
|
+
)
|
536
|
+
|
537
|
+
# Add assistants to the pipeline
|
538
|
+
pipeline.add_assistant(OxAiWorkers::Assistant::Coder.new)
|
539
|
+
pipeline.add_assistant(OxAiWorkers::Assistant::Sysop.new)
|
540
|
+
```
|
541
|
+
|
542
|
+
Enables communication between multiple assistants, maintaining message context and facilitating collaborative problem-solving.
|
543
|
+
|
544
|
+
- **Eval**: Code execution tool
|
545
|
+
|
546
|
+
```ruby
|
547
|
+
# Initialize with optional parameters
|
548
|
+
eval_tool = OxAiWorkers::Tool::Eval.new(
|
549
|
+
only: [:ruby, :sh], # Optional: Limit available functions
|
550
|
+
current_dir: Dir.pwd # Optional: Directory to execute commands in
|
551
|
+
)
|
552
|
+
```
|
553
|
+
|
554
|
+
Allows execution of Ruby code and shell commands, with directory context support.
|
555
|
+
|
556
|
+
- **FileSystem**: File operations tool
|
557
|
+
|
558
|
+
```ruby
|
559
|
+
# Initialize with optional parameters
|
560
|
+
file_system = OxAiWorkers::Tool::FileSystem.new(
|
561
|
+
current_dir: Dir.pwd, # Optional: Base directory for operations
|
562
|
+
only: [:list_directory, :read_file, :write_to_file] # Optional: Limit available functions
|
563
|
+
)
|
564
|
+
```
|
565
|
+
|
566
|
+
Provides functions for listing directory contents, reading from files, and writing to files with support for relative paths.
|
567
|
+
|
568
|
+
Additional tools like Database and Converter are available for specialized tasks and can be integrated using the same pattern.
|
569
|
+
|
570
|
+
### Function Control Mechanisms
|
571
|
+
|
572
|
+
OxAiWorkers provides two powerful mechanisms to control function execution behavior in iterators:
|
573
|
+
|
574
|
+
#### Call Stack
|
575
|
+
|
576
|
+
The `call_stack` parameter allows you to force the model to call specific functions in a predetermined order:
|
577
|
+
|
578
|
+
```ruby
|
579
|
+
iterator = OxAiWorkers::Iterator.new(
|
580
|
+
worker: worker,
|
581
|
+
tools: [my_tool],
|
582
|
+
call_stack: [
|
583
|
+
my_tool.full_function_name(:process_data),
|
584
|
+
OxAiWorkers::Iterator.full_function_name(:outer_voice),
|
585
|
+
]
|
586
|
+
)
|
587
|
+
```
|
588
|
+
|
589
|
+
This feature is particularly useful when:
|
590
|
+
|
591
|
+
- You need to ensure a specific sequence of operations
|
592
|
+
- Certain functions must be called before others
|
593
|
+
- You want to guide the model through a predefined workflow
|
594
|
+
- Complex operations require strict ordering of function calls
|
595
|
+
|
596
|
+
The `call_stack` is processed sequentially, with each function being removed from the stack after it's called.
|
597
|
+
|
598
|
+
#### Stop Double Calls
|
599
|
+
|
600
|
+
The `stop_double_calls` parameter prevents the model from calling the same function twice in consecutive operations:
|
601
|
+
|
602
|
+
```ruby
|
603
|
+
iterator = OxAiWorkers::Iterator.new(
|
604
|
+
worker: worker,
|
605
|
+
tools: [my_tool],
|
606
|
+
stop_double_calls: [
|
607
|
+
my_tool.full_function_name(:expensive_operation)
|
608
|
+
]
|
609
|
+
)
|
610
|
+
```
|
611
|
+
|
612
|
+
This feature is valuable for:
|
613
|
+
|
614
|
+
- Preventing redundant operations that could waste resources
|
615
|
+
- Avoiding duplicate processing of the same data
|
616
|
+
- Ensuring that certain operations are executed only once in sequence
|
617
|
+
- Protecting against potential infinite loops in function calls
|
618
|
+
|
619
|
+
When a function is called, its name is stored as the `last_call`. If the next function call matches both the `last_call` and is included in the `stop_double_calls` list, it will be excluded from the available tools for that request.
|
620
|
+
|
621
|
+
By default, `stop_double_calls` is applied to the `inner_monologue` and `outer_voice` functions to prevent reasoning loops and repetitive responses. This default behavior helps models avoid getting stuck in circular thinking patterns.
|
622
|
+
|
623
|
+
If you need to override this default behavior (for example, when consecutive monologue or voice calls are required for your specific use case), you can reset the stop_double_calls list **after** the iterator is created:
|
624
|
+
|
625
|
+
```ruby
|
626
|
+
# Clear the default stop_double_calls constraints
|
627
|
+
@iterator.stop_double_calls = []
|
628
|
+
|
629
|
+
# Or set your own custom constraints
|
630
|
+
@iterator.stop_double_calls = [my_tool.full_function_name(:specific_function)]
|
631
|
+
```
|
632
|
+
|
441
633
|
### Implementing Your Own Assistant
|
442
634
|
|
443
635
|
Create custom assistants by inheriting from existing ones or composing with the Iterator:
|
@@ -40,8 +40,8 @@ en:
|
|
40
40
|
assistant_info: "ID: **%{id}**, Title: %{title}\nRole: %{role}\nDescription: %{description}\nCapabilities: %{capabilities}"
|
41
41
|
pixels:
|
42
42
|
generate_image:
|
43
|
-
description: 'Image Generation Tool: Creates an image based on the provided text prompt
|
44
|
-
prompt: 'Text description of the image to generate'
|
45
|
-
size: 'Size of the generated image
|
46
|
-
quality: 'Quality of the generated image
|
43
|
+
description: 'Image Generation Tool: Creates an image based on the provided text prompt on **English language**.'
|
44
|
+
prompt: 'Text description of the image to generate on **English language**'
|
45
|
+
size: 'Size of the generated image'
|
46
|
+
quality: 'Quality of the generated image'
|
47
47
|
file_name: 'File name for saving the generated image in png format. For example: image.png'
|
@@ -40,12 +40,12 @@ ru:
|
|
40
40
|
assistant_info: "ID: **%{id}**, Название: %{title}\nРоль: %{role}\nОписание: %{description}\nВозможности: %{capabilities}"
|
41
41
|
pixels:
|
42
42
|
generate_image:
|
43
|
-
description: 'Инструмент генерации изображений: Создает изображение на основе предоставленного текстового
|
44
|
-
prompt: 'Текстовое описание изображения для генерации'
|
43
|
+
description: 'Инструмент генерации изображений: Создает изображение на основе предоставленного текстового запроса на **английском языке**.'
|
44
|
+
prompt: 'Текстовое описание изображения для генерации строго **на английском** языке'
|
45
45
|
size: 'Размер генерируемого изображения'
|
46
46
|
quality: 'Качество генерируемого изображения'
|
47
47
|
file_name: 'Имя файла для сохранения изображения в формате png. Например: image.png'
|
48
48
|
edit_image:
|
49
|
-
description: 'Инструмент редактирования изображений: Редактирует изображение на основе предоставленного текстового запроса.
|
49
|
+
description: 'Инструмент редактирования изображений: Редактирует изображение на основе предоставленного текстового запроса.'
|
50
50
|
input_image: 'URL или путь к изображению для редактирования'
|
51
51
|
prompt: 'Текстовое описание действий для редактирования изображения'
|
data/lib/ox-ai-workers.rb
CHANGED
@@ -36,12 +36,17 @@ require_relative 'oxaiworkers/assistant/localizer'
|
|
36
36
|
require_relative 'oxaiworkers/assistant/orchestrator'
|
37
37
|
require_relative 'oxaiworkers/assistant/painter'
|
38
38
|
|
39
|
-
require_relative 'oxaiworkers/models/
|
39
|
+
require_relative 'oxaiworkers/models/llm_base'
|
40
40
|
require_relative 'oxaiworkers/models/openai_max'
|
41
41
|
require_relative 'oxaiworkers/models/openai_mini'
|
42
42
|
require_relative 'oxaiworkers/models/openai_nano'
|
43
43
|
require_relative 'oxaiworkers/models/deepseek_max'
|
44
44
|
|
45
|
+
require_relative 'oxaiworkers/models/images_base'
|
46
|
+
require_relative 'oxaiworkers/models/stability_images'
|
47
|
+
require_relative 'oxaiworkers/models/openai_gpt_image'
|
48
|
+
require_relative 'oxaiworkers/models/openai_dalle3'
|
49
|
+
|
45
50
|
require_relative 'oxaiworkers/engine' if defined?(Rails)
|
46
51
|
|
47
52
|
module OxAiWorkers
|
@@ -52,7 +57,8 @@ module OxAiWorkers
|
|
52
57
|
class ConfigurationError < Error; end
|
53
58
|
|
54
59
|
class Configuration
|
55
|
-
attr_accessor :max_tokens, :temperature, :wait_for_complete, :access_token_deepseek, :access_token_openai
|
60
|
+
attr_accessor :max_tokens, :temperature, :wait_for_complete, :access_token_deepseek, :access_token_openai,
|
61
|
+
:access_token_stability
|
56
62
|
|
57
63
|
def initialize
|
58
64
|
@max_tokens = DEFAULT_MAX_TOKEN
|
@@ -61,6 +67,7 @@ module OxAiWorkers
|
|
61
67
|
|
62
68
|
@access_token_deepseek = nil
|
63
69
|
@access_token_openai = nil
|
70
|
+
@access_token_stability = nil
|
64
71
|
|
65
72
|
[Array, NilClass, String, Symbol, Hash].each do |c|
|
66
73
|
c.send(:include, OxAiWorkers::PresentCompat) unless c.method_defined?(:present?)
|
@@ -34,6 +34,14 @@ module OxAiWorkers
|
|
34
34
|
@iterator.clear_context
|
35
35
|
@iterator.add_context context
|
36
36
|
end
|
37
|
+
|
38
|
+
def add_file(pdf:, filename:, text:, role: :user)
|
39
|
+
@iterator.add_file(pdf:, filename:, text:, role:)
|
40
|
+
end
|
41
|
+
|
42
|
+
def add_image(text:, url: nil, binary: nil, role: :user, detail: 'auto', mime_type: 'image/jpeg')
|
43
|
+
@iterator.add_image(text:, url:, binary:, role:, detail:, mime_type:)
|
44
|
+
end
|
37
45
|
end
|
38
46
|
end
|
39
47
|
end
|
@@ -5,7 +5,7 @@ module OxAiWorkers
|
|
5
5
|
class Painter
|
6
6
|
include OxAiWorkers::Assistant::ModuleBase
|
7
7
|
|
8
|
-
def initialize(current_dir: nil, delayed: false, model: nil)
|
8
|
+
def initialize(current_dir: nil, delayed: false, model: nil, image_model: nil)
|
9
9
|
store_locale
|
10
10
|
@current_dir = current_dir
|
11
11
|
|
@@ -17,10 +17,13 @@ module OxAiWorkers
|
|
17
17
|
@title = I18n.t('oxaiworkers.assistant.painter.title')
|
18
18
|
end
|
19
19
|
|
20
|
+
worker = init_worker(delayed:, model:)
|
21
|
+
image_model ||= OxAiWorkers::Models::OpenaiDalle3.new
|
22
|
+
|
20
23
|
@iterator = Iterator.new(
|
21
|
-
worker
|
24
|
+
worker:,
|
22
25
|
role: @role,
|
23
|
-
tools: [Tool::Pixels.new(worker:
|
26
|
+
tools: [Tool::Pixels.new(worker: image_model, current_dir:)],
|
24
27
|
locale: @locale,
|
25
28
|
on_inner_monologue: ->(text:) { puts "monologue: #{text}".colorize(:yellow) },
|
26
29
|
on_outer_voice: ->(text:) { puts "voice: #{text}".colorize(:green) }
|
data/lib/oxaiworkers/iterator.rb
CHANGED
@@ -8,10 +8,12 @@ module OxAiWorkers
|
|
8
8
|
include OxAiWorkers::LoadI18n
|
9
9
|
|
10
10
|
attr_accessor :worker, :role, :messages, :context, :tools, :queue, :monologue, :tasks,
|
11
|
-
:on_inner_monologue, :on_outer_voice, :on_finish, :def_except, :def_only
|
11
|
+
:on_inner_monologue, :on_outer_voice, :on_finish, :def_except, :def_only,
|
12
|
+
:call_stack, :stop_double_calls
|
12
13
|
|
13
14
|
def initialize(worker:, role: nil, tools: [], on_inner_monologue: nil, on_outer_voice: nil,
|
14
|
-
on_finish: nil, steps: nil, def_except: [], def_only: nil, locale: nil
|
15
|
+
on_finish: nil, steps: nil, def_except: [], def_only: nil, locale: nil,
|
16
|
+
call_stack: nil, stop_double_calls: [])
|
15
17
|
|
16
18
|
@locale = locale || I18n.locale
|
17
19
|
|
@@ -41,6 +43,18 @@ module OxAiWorkers
|
|
41
43
|
@on_outer_voice = on_outer_voice
|
42
44
|
@on_finish = on_finish
|
43
45
|
|
46
|
+
if call_stack&.any?
|
47
|
+
if available_defs.include?(:inner_monologue) && !call_stack.include?(OxAiWorkers::Iterator.full_function_name(:inner_monologue))
|
48
|
+
# Add inner_monologue first
|
49
|
+
@call_stack = [OxAiWorkers::Iterator.full_function_name(:inner_monologue)] + call_stack
|
50
|
+
end
|
51
|
+
# Add finish_it last
|
52
|
+
@call_stack = call_stack + [OxAiWorkers::Iterator.full_function_name(:finish_it)]
|
53
|
+
end
|
54
|
+
|
55
|
+
@stop_double_calls = [OxAiWorkers::Iterator.full_function_name(:inner_monologue),
|
56
|
+
OxAiWorkers::Iterator.full_function_name(:outer_voice)] + stop_double_calls
|
57
|
+
|
44
58
|
cleanup
|
45
59
|
|
46
60
|
super()
|
@@ -99,17 +113,20 @@ module OxAiWorkers
|
|
99
113
|
end
|
100
114
|
|
101
115
|
def rebuild_worker
|
116
|
+
@worker.last_call = nil
|
117
|
+
@worker.call_stack = @call_stack.dup
|
118
|
+
@worker.stop_double_calls = @stop_double_calls
|
102
119
|
@worker.messages = []
|
103
|
-
@worker.append(role: :system, content: @role) if @role.present?
|
120
|
+
@worker.append(role: :system, content: "<role>\n#{@role}\n</role>") if @role.present?
|
104
121
|
|
105
|
-
@
|
106
|
-
@worker.append(role: :
|
122
|
+
@worker.append(role: :system, content: "<instructions>\n#{valid_monologue.join("\n")}\n</instructions>")
|
123
|
+
@tasks.each { |task| @worker.append(role: :user, content: "<task>\n#{task}\n</task>") }
|
107
124
|
@worker.append(messages: @context) if @context.present?
|
108
125
|
@tools.each do |tool|
|
109
126
|
@worker.append(role: :user, content: tool.context) if tool.respond_to?(:context) && tool.context.present?
|
110
127
|
end
|
111
128
|
@worker.append(messages: @messages)
|
112
|
-
@tasks.each { |task| @worker.append(role: :user, content: task) }
|
129
|
+
@tasks.each { |task| @worker.append(role: :user, content: "<task>\n#{task}\n</task>") }
|
113
130
|
@worker.tools = function_schemas.to_openai_format(only: available_defs)
|
114
131
|
return unless @tools.present?
|
115
132
|
|
@@ -235,6 +252,35 @@ module OxAiWorkers
|
|
235
252
|
add_raw_context({ role:, content: text })
|
236
253
|
end
|
237
254
|
|
255
|
+
def add_file(pdf:, filename:, text:, role: :user)
|
256
|
+
content = []
|
257
|
+
content << { type: 'text', text: } if text.present?
|
258
|
+
content << {
|
259
|
+
type: 'file',
|
260
|
+
file: {
|
261
|
+
filename:,
|
262
|
+
file_data: Base64.strict_encode64(pdf)
|
263
|
+
}
|
264
|
+
}
|
265
|
+
|
266
|
+
add_raw_context({ role:, content: })
|
267
|
+
end
|
268
|
+
|
269
|
+
def add_image(text:, url: nil, binary: nil, role: :user, detail: 'auto', mime_type: 'image/jpeg')
|
270
|
+
content = []
|
271
|
+
content << { type: 'text', text: } if text.present?
|
272
|
+
|
273
|
+
image_url = if binary.present?
|
274
|
+
"data:#{mime_type};base64,#{Base64.strict_encode64(binary)}"
|
275
|
+
else
|
276
|
+
url
|
277
|
+
end
|
278
|
+
|
279
|
+
content << { type: 'image_url', image_url: { url: image_url, detail: } }
|
280
|
+
|
281
|
+
add_raw_context({ role:, content: })
|
282
|
+
end
|
283
|
+
|
238
284
|
def add_raw_context(c)
|
239
285
|
@context << c
|
240
286
|
end
|
@@ -2,9 +2,7 @@
|
|
2
2
|
|
3
3
|
module OxAiWorkers
|
4
4
|
module Models
|
5
|
-
class DeepseekMax
|
6
|
-
include OxAiWorkers::Models::ModuleBase
|
7
|
-
|
5
|
+
class DeepseekMax < LLMBase
|
8
6
|
def initialize(uri_base: nil, api_key: nil, model: nil, max_tokens: nil, temperature: nil, frequency_penalty: nil)
|
9
7
|
@model = model || 'deepseek-chat'
|
10
8
|
@uri_base = uri_base || 'https://api.deepseek.com/'
|
@@ -2,7 +2,7 @@
|
|
2
2
|
|
3
3
|
module OxAiWorkers
|
4
4
|
module Models
|
5
|
-
|
5
|
+
class LLMBase
|
6
6
|
attr_accessor :uri_base, :api_key, :model, :max_tokens, :temperature, :frequency_penalty
|
7
7
|
|
8
8
|
def initialize(uri_base:, api_key:, model:, max_tokens: nil, temperature: nil, frequency_penalty: nil)
|
@@ -0,0 +1,47 @@
|
|
1
|
+
module OxAiWorkers
|
2
|
+
module Models
|
3
|
+
class OpenaiDalle3 < ImagesBase
|
4
|
+
def initialize(api_key: nil, options: {})
|
5
|
+
@sizes = %w[1024x1024 1024x1792 1792x1024]
|
6
|
+
@qualities = %w[standard hd]
|
7
|
+
api_key ||= OxAiWorkers.configuration.access_token_openai
|
8
|
+
|
9
|
+
@client = OpenAI::Client.new(access_token: api_key)
|
10
|
+
super(options:)
|
11
|
+
end
|
12
|
+
|
13
|
+
def generate(prompt:, size: nil, quality: nil)
|
14
|
+
puts "OpenaiDalle3: #{prompt}"
|
15
|
+
|
16
|
+
size ||= @sizes.first
|
17
|
+
quality ||= @qualities.first
|
18
|
+
|
19
|
+
options = {
|
20
|
+
prompt:,
|
21
|
+
model: 'dall-e-3',
|
22
|
+
size:,
|
23
|
+
quality:
|
24
|
+
}
|
25
|
+
options.merge!(@options)
|
26
|
+
|
27
|
+
response = @client.images.generate(
|
28
|
+
parameters: options
|
29
|
+
)
|
30
|
+
|
31
|
+
url = response.dig('data', 0, 'url')
|
32
|
+
b64_json = response.dig('data', 0, 'b64_json')
|
33
|
+
revised_prompt = response.dig('data', 0, 'revised_prompt')
|
34
|
+
|
35
|
+
@result = "revised_prompt: #{revised_prompt}\n\n"
|
36
|
+
if url.present?
|
37
|
+
@result += "url: #{url}\n\n"
|
38
|
+
URI.open(url).read
|
39
|
+
elsif b64_json.present?
|
40
|
+
Base64.decode64(b64_json)
|
41
|
+
else
|
42
|
+
nil
|
43
|
+
end
|
44
|
+
end
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
@@ -0,0 +1,37 @@
|
|
1
|
+
module OxAiWorkers
|
2
|
+
module Models
|
3
|
+
class OpenaiGptImage < ImagesBase
|
4
|
+
def initialize(api_key: nil, options: {})
|
5
|
+
@sizes = %w[auto 1024x1024 1536x1024 1024x1536]
|
6
|
+
@qualities = %w[auto low medium high]
|
7
|
+
api_key ||= OxAiWorkers.configuration.access_token_openai
|
8
|
+
|
9
|
+
@client = OpenAI::Client.new(access_token: api_key)
|
10
|
+
super(options:)
|
11
|
+
end
|
12
|
+
|
13
|
+
def generate(prompt:, size: nil, quality: nil)
|
14
|
+
puts "OpenaiGptImage: #{prompt}"
|
15
|
+
|
16
|
+
size ||= @sizes.first
|
17
|
+
quality ||= @qualities.first
|
18
|
+
|
19
|
+
options = {
|
20
|
+
prompt:,
|
21
|
+
model: 'gpt-image-1',
|
22
|
+
size:,
|
23
|
+
quality:
|
24
|
+
}
|
25
|
+
options.merge!(@options)
|
26
|
+
|
27
|
+
response = @client.images.generate(
|
28
|
+
parameters: options
|
29
|
+
)
|
30
|
+
|
31
|
+
b64_json = response.dig('data', 0, 'b64_json')
|
32
|
+
|
33
|
+
Base64.decode64(b64_json)
|
34
|
+
end
|
35
|
+
end
|
36
|
+
end
|
37
|
+
end
|
@@ -2,9 +2,7 @@
|
|
2
2
|
|
3
3
|
module OxAiWorkers
|
4
4
|
module Models
|
5
|
-
class OpenaiMax
|
6
|
-
include OxAiWorkers::Models::ModuleBase
|
7
|
-
|
5
|
+
class OpenaiMax < LLMBase
|
8
6
|
def initialize(uri_base: nil, api_key: nil, model: nil, max_tokens: nil, temperature: nil, frequency_penalty: nil)
|
9
7
|
@model = model || 'gpt-4.1'
|
10
8
|
@api_key = api_key || OxAiWorkers.configuration.access_token_openai
|
@@ -0,0 +1,32 @@
|
|
1
|
+
module OxAiWorkers
|
2
|
+
module Models
|
3
|
+
class StabilityImages < ImagesBase
|
4
|
+
def initialize(api_key: nil, timeout: 300, options: {})
|
5
|
+
@sizes = %w[auto]
|
6
|
+
@qualities = %w[auto]
|
7
|
+
api_key ||= OxAiWorkers.configuration.access_token_stability
|
8
|
+
|
9
|
+
@client = StabilitySDK::Client.new(api_key:, timeout:)
|
10
|
+
super(options:)
|
11
|
+
end
|
12
|
+
|
13
|
+
def generate(prompt:, size: nil, quality: nil)
|
14
|
+
puts "StabilityImages: #{prompt}"
|
15
|
+
options = {
|
16
|
+
engine_id: 'stable-diffusion-xl-1024-v1-0'
|
17
|
+
}
|
18
|
+
options.merge!(@options)
|
19
|
+
binary = nil
|
20
|
+
@client.generate(prompt, options) do |answer|
|
21
|
+
answer.artifacts.each do |artifact|
|
22
|
+
next unless artifact.type == :ARTIFACT_IMAGE
|
23
|
+
|
24
|
+
binary = artifact.binary
|
25
|
+
end
|
26
|
+
end
|
27
|
+
|
28
|
+
binary
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
32
|
+
end
|
@@ -3,15 +3,17 @@
|
|
3
3
|
module OxAiWorkers
|
4
4
|
class ModuleRequest
|
5
5
|
attr_accessor :result, :client, :messages, :model, :custom_id, :tools, :errors,
|
6
|
-
:tool_calls_raw, :tool_calls, :is_truncated, :finish_reason,
|
6
|
+
:tool_calls_raw, :tool_calls, :is_truncated, :finish_reason,
|
7
|
+
:on_stream_proc, :call_stack, :last_call, :stop_double_calls
|
7
8
|
|
8
|
-
def initialize_requests(model:, on_stream: nil)
|
9
|
+
def initialize_requests(model:, on_stream: nil, call_stack: nil)
|
9
10
|
@custom_id = SecureRandom.uuid
|
10
11
|
@model = model
|
11
12
|
@client = nil
|
12
13
|
@is_truncated = false
|
13
14
|
@finish_reason = nil
|
14
15
|
@on_stream_proc = on_stream
|
16
|
+
@call_stack = call_stack
|
15
17
|
|
16
18
|
if @model.api_key.nil?
|
17
19
|
error_text = "#{@model.model} access token missing!"
|
@@ -34,6 +36,7 @@ module OxAiWorkers
|
|
34
36
|
@tool_calls_raw = nil
|
35
37
|
@is_truncated = false
|
36
38
|
@finish_reason = nil
|
39
|
+
@last_call = nil
|
37
40
|
end
|
38
41
|
|
39
42
|
def append(role: nil, content: nil, messages: nil)
|
@@ -50,8 +53,17 @@ module OxAiWorkers
|
|
50
53
|
frequency_penalty: @model.frequency_penalty
|
51
54
|
}
|
52
55
|
if @tools.present?
|
53
|
-
parameters[:tools] = @tools
|
54
|
-
|
56
|
+
parameters[:tools] = @tools.reject do |f|
|
57
|
+
tool_name = f[:function][:name]
|
58
|
+
tool_name == @last_call && @stop_double_calls.include?(tool_name)
|
59
|
+
end
|
60
|
+
if @call_stack&.any?
|
61
|
+
func1 = @call_stack.first
|
62
|
+
@call_stack = @call_stack.drop(1)
|
63
|
+
parameters[:tool_choice] = { type: 'function', function: { name: func1 } }
|
64
|
+
else
|
65
|
+
parameters[:tool_choice] = 'required'
|
66
|
+
end
|
55
67
|
end
|
56
68
|
if @on_stream_proc.present?
|
57
69
|
parameters[:stream] = @on_stream_proc
|
@@ -134,6 +146,7 @@ module OxAiWorkers
|
|
134
146
|
name: function['name'].split('__').last,
|
135
147
|
args: args
|
136
148
|
}
|
149
|
+
@last_call = function['name']
|
137
150
|
end
|
138
151
|
end
|
139
152
|
end
|
@@ -8,22 +8,9 @@ module OxAiWorkers
|
|
8
8
|
include OxAiWorkers::DependencyHelper
|
9
9
|
include OxAiWorkers::LoadI18n
|
10
10
|
|
11
|
-
attr_accessor :worker, :
|
12
|
-
|
13
|
-
|
14
|
-
'dall-e-3' => {
|
15
|
-
'model' => 'dall-e-3',
|
16
|
-
'size' => %w[1024x1024 1024x1792 1792x1024],
|
17
|
-
'quality' => %w[standard hd]
|
18
|
-
},
|
19
|
-
'gpt-image-1' => {
|
20
|
-
'model' => 'gpt-image-1',
|
21
|
-
'size' => %w[1024x1024 1024x1792 1792x1024],
|
22
|
-
'quality' => %w[auto low medium high]
|
23
|
-
}
|
24
|
-
}
|
25
|
-
|
26
|
-
def initialize(worker:, current_dir: nil, only: nil, image_model: 'dall-e-3', mask: nil)
|
11
|
+
attr_accessor :worker, :current_dir
|
12
|
+
|
13
|
+
def initialize(worker:, current_dir: nil, only: nil)
|
27
14
|
store_locale
|
28
15
|
|
29
16
|
init_white_list_with only
|
@@ -31,15 +18,19 @@ module OxAiWorkers
|
|
31
18
|
define_function :generate_image, description: I18n.t('oxaiworkers.tool.pixels.generate_image.description') do
|
32
19
|
property :prompt, type: 'string', description: I18n.t('oxaiworkers.tool.pixels.generate_image.prompt'),
|
33
20
|
required: true
|
34
|
-
|
35
|
-
|
21
|
+
if worker.sizes.length > 1
|
22
|
+
property :size, type: 'string', description: I18n.t('oxaiworkers.tool.pixels.generate_image.size'),
|
23
|
+
enum: worker.sizes
|
24
|
+
end
|
36
25
|
if current_dir.present?
|
37
26
|
property :file_name, type: 'string',
|
38
27
|
description: I18n.t('oxaiworkers.tool.pixels.generate_image.file_name'),
|
39
28
|
required: true
|
40
29
|
end
|
41
|
-
|
42
|
-
|
30
|
+
if worker.qualities.length > 1
|
31
|
+
property :quality, type: 'string', description: I18n.t('oxaiworkers.tool.pixels.generate_image.quality'),
|
32
|
+
enum: worker.qualities
|
33
|
+
end
|
43
34
|
end
|
44
35
|
|
45
36
|
# define_function :edit_image, description: I18n.t('oxaiworkers.tool.pixels.edit_image.description') do
|
@@ -56,32 +47,18 @@ module OxAiWorkers
|
|
56
47
|
|
57
48
|
@worker = worker
|
58
49
|
@current_dir = current_dir
|
59
|
-
@image_model = MODELS[image_model]
|
60
|
-
@mask = mask
|
61
50
|
end
|
62
51
|
|
63
52
|
def generate_image(prompt:, file_name: nil, size: nil, quality: nil)
|
64
|
-
|
65
|
-
|
66
|
-
size ||= @image_model['size'].first
|
67
|
-
quality ||= @image_model['quality'].first
|
68
|
-
|
69
|
-
response = @worker.client.images.generate(
|
70
|
-
parameters: {
|
71
|
-
prompt:,
|
72
|
-
model: @image_model['model'],
|
73
|
-
size:,
|
74
|
-
quality:
|
75
|
-
}
|
76
|
-
)
|
53
|
+
binary = @worker.generate(prompt:, size:, quality:)
|
77
54
|
|
78
|
-
@url = response.dig('data', 0, 'url')
|
79
|
-
revised_prompt = response.dig('data', 0, 'revised_prompt')
|
80
55
|
if file_name.present?
|
81
|
-
path = save_generated_image(file_name:)
|
82
|
-
"
|
56
|
+
path = save_generated_image(file_name:, binary:)
|
57
|
+
"Successfully generated image. file_name: #{path}\n\n#{@worker.result}"
|
58
|
+
elsif @worker.result.present?
|
59
|
+
@worker.result
|
83
60
|
else
|
84
|
-
|
61
|
+
'file_name not set for OxAiWorkers::Tool::Pixels. Please set file name first.'
|
85
62
|
end
|
86
63
|
end
|
87
64
|
|
@@ -109,20 +86,24 @@ module OxAiWorkers
|
|
109
86
|
end
|
110
87
|
end
|
111
88
|
|
112
|
-
def save_generated_image(file_name:)
|
89
|
+
def save_generated_image(file_name:, binary:)
|
113
90
|
unless @current_dir.present?
|
114
91
|
return 'Current directory not set for OxAiWorkers::Tool::Pixels. Please set current directory first.'
|
115
92
|
end
|
116
93
|
|
94
|
+
return 'File name not set for OxAiWorkers::Tool::Pixels. Please set file name first.' unless file_name.present?
|
95
|
+
|
117
96
|
# Ensure current_dir exists
|
118
97
|
FileUtils.mkdir_p(@current_dir) unless Dir.exist?(@current_dir)
|
119
98
|
|
120
99
|
path = File.join(@current_dir, file_name)
|
121
100
|
|
122
101
|
File.open(path, 'wb') do |file|
|
123
|
-
file.write(
|
102
|
+
file.write(binary)
|
124
103
|
end
|
125
104
|
|
105
|
+
puts "Successfully saved image. file_name: #{path}"
|
106
|
+
|
126
107
|
file_name
|
127
108
|
end
|
128
109
|
end
|
data/lib/oxaiworkers/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: ox-ai-workers
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.9.
|
4
|
+
version: 0.9.6
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Denis Smolev
|
@@ -123,7 +123,9 @@ executables:
|
|
123
123
|
extensions: []
|
124
124
|
extra_rdoc_files: []
|
125
125
|
files:
|
126
|
-
- ".
|
126
|
+
- ".cursor/rules/010-project-structure.mdc"
|
127
|
+
- ".cursor/rules/998-clean-code.mdc"
|
128
|
+
- ".cursor/rules/999-mdc-format.mdc"
|
127
129
|
- ".ruby-version"
|
128
130
|
- CHANGELOG.md
|
129
131
|
- CODE_OF_CONDUCT.md
|
@@ -155,10 +157,14 @@ files:
|
|
155
157
|
- lib/oxaiworkers/iterator.rb
|
156
158
|
- lib/oxaiworkers/load_i18n.rb
|
157
159
|
- lib/oxaiworkers/models/deepseek_max.rb
|
158
|
-
- lib/oxaiworkers/models/
|
160
|
+
- lib/oxaiworkers/models/images_base.rb
|
161
|
+
- lib/oxaiworkers/models/llm_base.rb
|
162
|
+
- lib/oxaiworkers/models/openai_dalle3.rb
|
163
|
+
- lib/oxaiworkers/models/openai_gpt_image.rb
|
159
164
|
- lib/oxaiworkers/models/openai_max.rb
|
160
165
|
- lib/oxaiworkers/models/openai_mini.rb
|
161
166
|
- lib/oxaiworkers/models/openai_nano.rb
|
167
|
+
- lib/oxaiworkers/models/stability_images.rb
|
162
168
|
- lib/oxaiworkers/module_request.rb
|
163
169
|
- lib/oxaiworkers/present_compat.rb
|
164
170
|
- lib/oxaiworkers/request.rb
|
data/.cursorrules
DELETED
@@ -1,155 +0,0 @@
|
|
1
|
-
# Overview
|
2
|
-
|
3
|
-
OxAiWorkers is a Ruby gem that implements a finite state machine (using the `state_machine` gem) to solve tasks using generative intelligence (with the `ruby-openai` gem). This approach enhances the final result by utilizing internal monologue and external tools.
|
4
|
-
|
5
|
-
## Architecture Principles
|
6
|
-
|
7
|
-
- The library is built on the finite state machine (FSM) pattern using the 'state_machine' gem
|
8
|
-
- Integration with generative models is implemented using the 'ruby-openai' gem
|
9
|
-
- DRY (Don't Repeat Yourself) principle is applied throughout all components
|
10
|
-
- Modular structure with clear separation of responsibilities between classes
|
11
|
-
- Encapsulation of states and transitions in separate classes
|
12
|
-
- Implementation of the "Composition" pattern for flexible tool integration
|
13
|
-
|
14
|
-
## Core Components
|
15
|
-
|
16
|
-
- `Request` and `DelayedRequest` - classes for executing API requests (immediate and delayed)
|
17
|
-
- `Iterator` - main class for iterative task execution with tools
|
18
|
-
- `Assistant` - high-level wrappers over Iterator (Sysop, Coder, Localizer, etc.)
|
19
|
-
- `Tool` - tools that can be used during task execution (Eval, FileSystem, Database)
|
20
|
-
- `ToolDefinition` - module for declaring functions and methods for tools
|
21
|
-
- `StateTools` - base class for managing states and transitions
|
22
|
-
- `ContextualLogger` - logging system with contextual information support
|
23
|
-
|
24
|
-
## Code Conventions
|
25
|
-
|
26
|
-
- Use `snake_case` for method and variable names
|
27
|
-
- Functions for generative models should also be in `snake_case` (inner_monologue, outer_voice, etc.)
|
28
|
-
- All public methods must have documentation with usage examples
|
29
|
-
- Tests are mandatory for all new functions
|
30
|
-
- All code comments, CHANGELOG, README, and other documentation must be written in English
|
31
|
-
- Use YARD-style documentation for all public methods
|
32
|
-
- Maintain a unified code formatting style (Rubocop is recommended)
|
33
|
-
- Follow the "Fail fast" principle for early error detection
|
34
|
-
|
35
|
-
## Interaction Patterns
|
36
|
-
|
37
|
-
- The system uses internal monologue (inner_monologue) for planning actions
|
38
|
-
- External voice (outer_voice) is used for communication with the user
|
39
|
-
- Execution flow management through finite state machine
|
40
|
-
- Implementation of callback mechanisms for flexible event handling
|
41
|
-
- Isolation of error handling functions at the tool level
|
42
|
-
|
43
|
-
## Integration
|
44
|
-
|
45
|
-
- CLI interface through `oxaiworkers init` and `oxaiworkers run` commands
|
46
|
-
- Rails support via ActiveRecord for storing delayed requests
|
47
|
-
- Configuration through the `OxAiWorkers.configure` block
|
48
|
-
- Multilingual support via standard I18n
|
49
|
-
- Integration with external APIs through request client templates
|
50
|
-
- Delayed execution mechanism via DelayedRequest
|
51
|
-
- Support for various language models (OpenAI, Anthropic, Gemini)
|
52
|
-
|
53
|
-
## Best Practices
|
54
|
-
|
55
|
-
- Use callbacks to handle various states (on_inner_monologue, on_outer_voice)
|
56
|
-
- Handle errors at the tool level, preventing them from interrupting the main execution flow
|
57
|
-
- When creating new assistants, inherit from the base Assistant class
|
58
|
-
- Use the white_list mechanism to restrict available functions
|
59
|
-
- Separate language model requests from result processing logic
|
60
|
-
- Practice dependency injection to improve code testability
|
61
|
-
- Use localization mechanisms for multilingual support
|
62
|
-
|
63
|
-
## Tools Architecture
|
64
|
-
|
65
|
-
- Each tool should be a self-contained module
|
66
|
-
- Tools are registered through the `define_function` interface
|
67
|
-
- All tools should handle their own errors and return readable messages
|
68
|
-
- Use parameter validation at the function definition level
|
69
|
-
- Maintain a unified format for return values
|
70
|
-
|
71
|
-
## Performance and Scaling
|
72
|
-
|
73
|
-
- Cache API request results when possible
|
74
|
-
- Use asynchronous processing for long operations
|
75
|
-
- Apply backoff strategies for repeated requests
|
76
|
-
- Break large tasks into atomic operations
|
77
|
-
- Provide monitoring and profiling mechanisms
|
78
|
-
|
79
|
-
## Finite State Machine Implementation
|
80
|
-
|
81
|
-
- Core FSM based on `state_machine` gem with states: idle → prepared → requested → analyzed → finished → idle
|
82
|
-
- State transitions managed by events: prepare, request, analyze, complete, iterate, end
|
83
|
-
- `StateTools` - base class for FSM implementation with event hooks and transition callbacks
|
84
|
-
- `StateBatch` - FSM extension for batch request processing with additional states
|
85
|
-
- Automatic error recovery and retry mechanisms for failed API requests
|
86
|
-
|
87
|
-
## Request Processing
|
88
|
-
|
89
|
-
- `ModuleRequest` - base class for all API requests with parsing and response handling
|
90
|
-
- Support for streaming responses with callback processing
|
91
|
-
- Built-in token usage tracking and truncation detection
|
92
|
-
- Error handling with automatic retries for server errors
|
93
|
-
|
94
|
-
## Iterator Lifecycle
|
95
|
-
|
96
|
-
- 3 core functions: inner_monologue, outer_voice, finish_it
|
97
|
-
- Configurable message queue for stateful conversation history
|
98
|
-
- Callback system for processing each state transition
|
99
|
-
- Context and milestone management for optimizing token usage
|
100
|
-
- Support for custom steps and instruction templating
|
101
|
-
|
102
|
-
## Additional Tools
|
103
|
-
|
104
|
-
- `Converter` - tools for data format conversion and transformation
|
105
|
-
- Support for custom tool development through inheritance and composition
|
106
|
-
- Automatic function name resolution and parameter validation
|
107
|
-
|
108
|
-
## Assistants Details
|
109
|
-
|
110
|
-
- `ModuleBase` - shared functionality for all assistant types
|
111
|
-
- `Sysop` - system administration and shell command execution
|
112
|
-
- `Coder` - specialized for code generation and analysis
|
113
|
-
- `Localizer` - translation and localization support
|
114
|
-
|
115
|
-
## Development Guidelines
|
116
|
-
|
117
|
-
- Use dependency injection for testability
|
118
|
-
- Follow the FSM pattern for all stateful operations
|
119
|
-
- Implement proper error boundaries at the tool level
|
120
|
-
- Use monologue for complex reasoning and planning
|
121
|
-
- Apply callbacks for event-driven architecture
|
122
|
-
- Utilize templates in the CLI for rapid prototyping
|
123
|
-
- Extend the base classes rather than modifying them
|
124
|
-
|
125
|
-
## Internationalization and Localization
|
126
|
-
|
127
|
-
- All code comments, variable names, and documentation MUST be written in English
|
128
|
-
- All user-facing strings MUST be properly localized using I18n
|
129
|
-
- Use I18n.t for all text that will be shown to users or appears in assistant prompts
|
130
|
-
- Store translations in YAML files within the config/locales directory
|
131
|
-
- Follow the naming convention of language.namespace.key (e.g., en.oxaiworkers.assistant.role)
|
132
|
-
- Use named parameters (%{variable}) instead of positional parameters (%s) in translation strings
|
133
|
-
- Use the with_locale method to ensure proper locale context when processing localized text
|
134
|
-
- Implement locale-aware classes by including the OxAiWorkers::LoadI18n module
|
135
|
-
- Store the current locale on initialization and preserve it across method calls
|
136
|
-
- Support multiple languages simultaneously through careful locale management
|
137
|
-
- Default to English for developer-facing messages and logs
|
138
|
-
- Ensure that all assistant classes properly handle localization in their format_role methods
|
139
|
-
|
140
|
-
## LoadI18n Module Usage
|
141
|
-
|
142
|
-
- The `OxAiWorkers::LoadI18n` module provides two key methods for localization:
|
143
|
-
- `store_locale` - saves the current locale at initialization time
|
144
|
-
- `with_locale` - executes a block of code in the context of the saved locale
|
145
|
-
- Always include the `OxAiWorkers::LoadI18n` module in classes that need localization capabilities
|
146
|
-
- Call `store_locale` in the initialization methods of locale-aware classes
|
147
|
-
- Wrap all locale-dependent code in `with_locale` blocks
|
148
|
-
- NEVER redefine the `with_locale` method in classes that include LoadI18n
|
149
|
-
- All methods that produce user-visible text must use the locale context via `with_locale` blocks
|
150
|
-
- Regular method calls from classes including LoadI18n do not require additional locale handling
|
151
|
-
|
152
|
-
## Multi-Language Support
|
153
|
-
|
154
|
-
- Use the store_locale and with_locale methods for consistent localization context
|
155
|
-
- All error messages should be localized and retrieved via I18n.t
|