decision_agent 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +234 -919
- data/bin/decision_agent +5 -5
- data/lib/decision_agent/agent.rb +19 -26
- data/lib/decision_agent/audit/null_adapter.rb +1 -2
- data/lib/decision_agent/decision.rb +3 -1
- data/lib/decision_agent/dsl/condition_evaluator.rb +4 -3
- data/lib/decision_agent/dsl/rule_parser.rb +4 -6
- data/lib/decision_agent/dsl/schema_validator.rb +27 -31
- data/lib/decision_agent/errors.rb +21 -6
- data/lib/decision_agent/evaluation.rb +3 -1
- data/lib/decision_agent/evaluation_validator.rb +78 -0
- data/lib/decision_agent/evaluators/json_rule_evaluator.rb +26 -0
- data/lib/decision_agent/evaluators/static_evaluator.rb +2 -6
- data/lib/decision_agent/monitoring/alert_manager.rb +282 -0
- data/lib/decision_agent/monitoring/dashboard/public/dashboard.css +381 -0
- data/lib/decision_agent/monitoring/dashboard/public/dashboard.js +471 -0
- data/lib/decision_agent/monitoring/dashboard/public/index.html +161 -0
- data/lib/decision_agent/monitoring/dashboard_server.rb +340 -0
- data/lib/decision_agent/monitoring/metrics_collector.rb +278 -0
- data/lib/decision_agent/monitoring/monitored_agent.rb +71 -0
- data/lib/decision_agent/monitoring/prometheus_exporter.rb +247 -0
- data/lib/decision_agent/replay/replay.rb +12 -22
- data/lib/decision_agent/scoring/base.rb +1 -1
- data/lib/decision_agent/scoring/consensus.rb +5 -5
- data/lib/decision_agent/scoring/weighted_average.rb +1 -1
- data/lib/decision_agent/version.rb +1 -1
- data/lib/decision_agent/versioning/activerecord_adapter.rb +141 -0
- data/lib/decision_agent/versioning/adapter.rb +100 -0
- data/lib/decision_agent/versioning/file_storage_adapter.rb +290 -0
- data/lib/decision_agent/versioning/version_manager.rb +127 -0
- data/lib/decision_agent/web/public/app.js +318 -0
- data/lib/decision_agent/web/public/index.html +56 -1
- data/lib/decision_agent/web/public/styles.css +219 -0
- data/lib/decision_agent/web/server.rb +169 -9
- data/lib/decision_agent.rb +11 -0
- data/lib/generators/decision_agent/install/install_generator.rb +40 -0
- data/lib/generators/decision_agent/install/templates/README +47 -0
- data/lib/generators/decision_agent/install/templates/migration.rb +37 -0
- data/lib/generators/decision_agent/install/templates/rule.rb +30 -0
- data/lib/generators/decision_agent/install/templates/rule_version.rb +66 -0
- data/spec/activerecord_thread_safety_spec.rb +553 -0
- data/spec/agent_spec.rb +13 -13
- data/spec/api_contract_spec.rb +16 -16
- data/spec/audit_adapters_spec.rb +3 -3
- data/spec/comprehensive_edge_cases_spec.rb +86 -86
- data/spec/dsl_validation_spec.rb +83 -83
- data/spec/edge_cases_spec.rb +23 -23
- data/spec/examples/feedback_aware_evaluator_spec.rb +7 -7
- data/spec/examples.txt +548 -0
- data/spec/issue_verification_spec.rb +685 -0
- data/spec/json_rule_evaluator_spec.rb +15 -15
- data/spec/monitoring/alert_manager_spec.rb +378 -0
- data/spec/monitoring/metrics_collector_spec.rb +281 -0
- data/spec/monitoring/monitored_agent_spec.rb +222 -0
- data/spec/monitoring/prometheus_exporter_spec.rb +242 -0
- data/spec/replay_edge_cases_spec.rb +58 -58
- data/spec/replay_spec.rb +11 -11
- data/spec/rfc8785_canonicalization_spec.rb +215 -0
- data/spec/scoring_spec.rb +1 -1
- data/spec/spec_helper.rb +9 -0
- data/spec/thread_safety_spec.rb +482 -0
- data/spec/thread_safety_spec.rb.broken +878 -0
- data/spec/versioning_spec.rb +777 -0
- data/spec/web_ui_rack_spec.rb +135 -0
- metadata +84 -11
data/README.md
CHANGED
|
@@ -7,1054 +7,369 @@
|
|
|
7
7
|
|
|
8
8
|
A production-grade, deterministic, explainable, and auditable decision engine for Ruby.
|
|
9
9
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
Enterprise applications need to make complex decisions based on business rules, but existing solutions fall short:
|
|
13
|
-
|
|
14
|
-
- **Traditional rule engines**: Often lack conflict resolution, confidence scoring, and audit replay capabilities
|
|
15
|
-
- **Framework-coupled solutions**: Tightly bound to specific frameworks (Rails, etc.), limiting portability
|
|
16
|
-
- **AI-first frameworks**: Non-deterministic, expensive, opaque, and unsuitable for regulated domains
|
|
10
|
+
**Built for regulated domains. Deterministic by design. AI-optional.**
|
|
17
11
|
|
|
18
|
-
|
|
12
|
+
## Why DecisionAgent?
|
|
19
13
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
6. **AI-optional** - Rules first, AI enhancement optional
|
|
14
|
+
- ✅ **Deterministic** - Same input always produces same output
|
|
15
|
+
- ✅ **Explainable** - Every decision includes human-readable reasoning
|
|
16
|
+
- ✅ **Auditable** - Reproduce any historical decision exactly
|
|
17
|
+
- ✅ **Framework-agnostic** - Pure Ruby, works anywhere
|
|
18
|
+
- ✅ **Production-ready** - Comprehensive testing, error handling, and versioning
|
|
26
19
|
|
|
27
20
|
## Installation
|
|
28
21
|
|
|
29
|
-
Add to your Gemfile:
|
|
30
|
-
|
|
31
|
-
```ruby
|
|
32
|
-
gem 'decision_agent'
|
|
33
|
-
```
|
|
34
|
-
|
|
35
|
-
Or install directly:
|
|
36
|
-
|
|
37
22
|
```bash
|
|
38
23
|
gem install decision_agent
|
|
39
24
|
```
|
|
40
25
|
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
```bash
|
|
46
|
-
decision_agent web
|
|
26
|
+
Or add to your Gemfile:
|
|
27
|
+
```ruby
|
|
28
|
+
gem 'decision_agent'
|
|
47
29
|
```
|
|
48
30
|
|
|
49
|
-
Then open [http://localhost:4567](http://localhost:4567) in your browser.
|
|
50
|
-
|
|
51
|
-
The Web UI provides:
|
|
52
|
-
- 📝 **Visual rule creation** - Build rules using forms and dropdowns
|
|
53
|
-
- 🔍 **Live validation** - Instant feedback on rule correctness
|
|
54
|
-
- 📤 **Export/Import** - Download or upload rules as JSON
|
|
55
|
-
- 📚 **Example templates** - Pre-built rule sets to get started
|
|
56
|
-
- ✨ **No coding required** - Perfect for business analysts and domain experts
|
|
57
|
-
|
|
58
|
-
See [WEB_UI.md](WEB_UI.md) for detailed documentation.
|
|
59
|
-
|
|
60
|
-
<img width="1602" height="770" alt="Screenshot 2025-12-19 at 3 06 07 PM" src="https://github.com/user-attachments/assets/6ee6859c-f9f2-4f93-8bff-923986ccb1bc" />
|
|
61
|
-
|
|
62
|
-
|
|
63
31
|
## Quick Start
|
|
64
32
|
|
|
65
33
|
```ruby
|
|
66
34
|
require 'decision_agent'
|
|
67
35
|
|
|
68
|
-
# Define
|
|
69
|
-
evaluator = DecisionAgent::Evaluators::
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
36
|
+
# Define evaluator with business rules
|
|
37
|
+
evaluator = DecisionAgent::Evaluators::JsonRuleEvaluator.new(
|
|
38
|
+
rules_json: {
|
|
39
|
+
version: "1.0",
|
|
40
|
+
ruleset: "approval_rules",
|
|
41
|
+
rules: [{
|
|
42
|
+
id: "high_value",
|
|
43
|
+
if: { field: "amount", op: "gt", value: 1000 },
|
|
44
|
+
then: { decision: "approve", weight: 0.9, reason: "High value transaction" }
|
|
45
|
+
}]
|
|
46
|
+
}
|
|
73
47
|
)
|
|
74
48
|
|
|
75
|
-
# Create agent
|
|
76
|
-
agent = DecisionAgent::Agent.new(
|
|
77
|
-
evaluators: [evaluator],
|
|
78
|
-
scoring_strategy: DecisionAgent::Scoring::WeightedAverage.new,
|
|
79
|
-
audit_adapter: DecisionAgent::Audit::LoggerAdapter.new
|
|
80
|
-
)
|
|
49
|
+
# Create decision agent
|
|
50
|
+
agent = DecisionAgent::Agent.new(evaluators: [evaluator])
|
|
81
51
|
|
|
82
52
|
# Make decision
|
|
83
|
-
result = agent.decide(
|
|
84
|
-
context: { user: "alice", priority: "high" }
|
|
85
|
-
)
|
|
86
|
-
|
|
87
|
-
puts result.decision # => "approve"
|
|
88
|
-
puts result.confidence # => 0.8
|
|
89
|
-
puts result.explanations # => ["Decision: approve (confidence: 0.8)", ...]
|
|
90
|
-
puts result.audit_payload # => Full audit trail
|
|
91
|
-
```
|
|
92
|
-
|
|
93
|
-
## Core Concepts
|
|
94
|
-
|
|
95
|
-
### 1. Agent
|
|
96
|
-
|
|
97
|
-
The orchestrator that coordinates evaluators, resolves conflicts, and produces decisions.
|
|
98
|
-
|
|
99
|
-
```ruby
|
|
100
|
-
agent = DecisionAgent::Agent.new(
|
|
101
|
-
evaluators: [eval1, eval2, eval3],
|
|
102
|
-
scoring_strategy: DecisionAgent::Scoring::WeightedAverage.new,
|
|
103
|
-
audit_adapter: DecisionAgent::Audit::NullAdapter.new
|
|
104
|
-
)
|
|
105
|
-
```
|
|
106
|
-
|
|
107
|
-
### 2. Evaluators
|
|
108
|
-
|
|
109
|
-
Pluggable components that evaluate context and return decisions.
|
|
110
|
-
|
|
111
|
-
#### StaticEvaluator
|
|
112
|
-
|
|
113
|
-
Always returns the same decision:
|
|
53
|
+
result = agent.decide(context: { amount: 1500 })
|
|
114
54
|
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
weight: 0.7,
|
|
119
|
-
reason: "Static approval rule"
|
|
120
|
-
)
|
|
55
|
+
puts result.decision # => "approve"
|
|
56
|
+
puts result.confidence # => 0.9
|
|
57
|
+
puts result.explanations # => ["High value transaction"]
|
|
121
58
|
```
|
|
122
59
|
|
|
123
|
-
|
|
60
|
+
## Web UI - Visual Rule Builder
|
|
124
61
|
|
|
125
|
-
|
|
62
|
+
The DecisionAgent Web UI provides a visual interface for building and testing rules.
|
|
126
63
|
|
|
127
|
-
|
|
128
|
-
rules = {
|
|
129
|
-
version: "1.0",
|
|
130
|
-
ruleset: "issue_triage",
|
|
131
|
-
rules: [
|
|
132
|
-
{
|
|
133
|
-
id: "high_priority_rule",
|
|
134
|
-
if: {
|
|
135
|
-
all: [
|
|
136
|
-
{ field: "priority", op: "eq", value: "high" },
|
|
137
|
-
{ field: "hours_inactive", op: "gte", value: 4 }
|
|
138
|
-
]
|
|
139
|
-
},
|
|
140
|
-
then: {
|
|
141
|
-
decision: "escalate",
|
|
142
|
-
weight: 0.9,
|
|
143
|
-
reason: "High priority issue inactive too long"
|
|
144
|
-
}
|
|
145
|
-
}
|
|
146
|
-
]
|
|
147
|
-
}
|
|
64
|
+
### Standalone Usage
|
|
148
65
|
|
|
149
|
-
|
|
150
|
-
rules_json: rules
|
|
151
|
-
)
|
|
152
|
-
```
|
|
153
|
-
|
|
154
|
-
### 3. Context
|
|
66
|
+
Launch the visual rule builder:
|
|
155
67
|
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
```ruby
|
|
159
|
-
context = DecisionAgent::Context.new({
|
|
160
|
-
user: "alice",
|
|
161
|
-
priority: "high",
|
|
162
|
-
hours_inactive: 5
|
|
163
|
-
})
|
|
68
|
+
```bash
|
|
69
|
+
decision_agent web
|
|
164
70
|
```
|
|
165
71
|
|
|
166
|
-
|
|
72
|
+
Open [http://localhost:4567](http://localhost:4567) in your browser.
|
|
167
73
|
|
|
168
|
-
|
|
74
|
+
### Mount in Rails
|
|
169
75
|
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
Sums weights for each decision, selects winner:
|
|
76
|
+
Add to your `config/routes.rb`:
|
|
173
77
|
|
|
174
78
|
```ruby
|
|
175
|
-
|
|
176
|
-
```
|
|
177
|
-
|
|
178
|
-
#### MaxWeight
|
|
179
|
-
|
|
180
|
-
Selects decision with highest individual weight:
|
|
181
|
-
|
|
182
|
-
```ruby
|
|
183
|
-
DecisionAgent::Scoring::MaxWeight.new
|
|
184
|
-
```
|
|
185
|
-
|
|
186
|
-
#### Consensus
|
|
187
|
-
|
|
188
|
-
Requires minimum agreement threshold:
|
|
79
|
+
require 'decision_agent/web/server'
|
|
189
80
|
|
|
190
|
-
|
|
191
|
-
DecisionAgent
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
#### Threshold
|
|
195
|
-
|
|
196
|
-
Requires minimum weight to accept decision:
|
|
197
|
-
|
|
198
|
-
```ruby
|
|
199
|
-
DecisionAgent::Scoring::Threshold.new(
|
|
200
|
-
threshold: 0.8,
|
|
201
|
-
fallback_decision: "manual_review"
|
|
202
|
-
)
|
|
81
|
+
Rails.application.routes.draw do
|
|
82
|
+
# Mount DecisionAgent Web UI
|
|
83
|
+
mount DecisionAgent::Web::Server, at: '/decision_agent'
|
|
84
|
+
end
|
|
203
85
|
```
|
|
204
86
|
|
|
205
|
-
|
|
87
|
+
Then visit `http://localhost:3000/decision_agent` in your browser.
|
|
206
88
|
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
#### NullAdapter
|
|
210
|
-
|
|
211
|
-
No-op (default):
|
|
89
|
+
**With Authentication:**
|
|
212
90
|
|
|
213
91
|
```ruby
|
|
214
|
-
|
|
92
|
+
authenticate :user, ->(user) { user.admin? } do
|
|
93
|
+
mount DecisionAgent::Web::Server, at: '/decision_agent'
|
|
94
|
+
end
|
|
215
95
|
```
|
|
216
96
|
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
Logs to any Ruby logger:
|
|
97
|
+
### Mount in Rack/Sinatra Apps
|
|
220
98
|
|
|
221
99
|
```ruby
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
level: Logger::INFO
|
|
225
|
-
)
|
|
226
|
-
```
|
|
227
|
-
|
|
228
|
-
## JSON Rule DSL
|
|
229
|
-
|
|
230
|
-
### Supported Operators
|
|
100
|
+
# config.ru
|
|
101
|
+
require 'decision_agent/web/server'
|
|
231
102
|
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
| `neq` | Not equal | `{ field: "status", op: "neq", value: "closed" }` |
|
|
236
|
-
| `gt` | Greater than | `{ field: "score", op: "gt", value: 80 }` |
|
|
237
|
-
| `gte` | Greater than or equal | `{ field: "hours", op: "gte", value: 4 }` |
|
|
238
|
-
| `lt` | Less than | `{ field: "temp", op: "lt", value: 32 }` |
|
|
239
|
-
| `lte` | Less than or equal | `{ field: "temp", op: "lte", value: 32 }` |
|
|
240
|
-
| `in` | Array membership | `{ field: "status", op: "in", value: ["open", "pending"] }` |
|
|
241
|
-
| `present` | Field exists and not empty | `{ field: "assignee", op: "present" }` |
|
|
242
|
-
| `blank` | Field missing, nil, or empty | `{ field: "description", op: "blank" }` |
|
|
243
|
-
|
|
244
|
-
### Condition Combinators
|
|
245
|
-
|
|
246
|
-
#### all
|
|
247
|
-
|
|
248
|
-
All sub-conditions must be true:
|
|
249
|
-
|
|
250
|
-
```json
|
|
251
|
-
{
|
|
252
|
-
"all": [
|
|
253
|
-
{ "field": "priority", "op": "eq", "value": "high" },
|
|
254
|
-
{ "field": "hours", "op": "gte", "value": 4 }
|
|
255
|
-
]
|
|
256
|
-
}
|
|
103
|
+
map '/decision_agent' do
|
|
104
|
+
run DecisionAgent::Web::Server
|
|
105
|
+
end
|
|
257
106
|
```
|
|
258
107
|
|
|
259
|
-
|
|
108
|
+
<img width="1622" height="820" alt="Screenshot" src="https://github.com/user-attachments/assets/687e9ff6-669a-40f9-be27-085c614392d4" />
|
|
260
109
|
|
|
261
|
-
|
|
110
|
+
See [Web UI Rails Integration Guide](wiki/WEB_UI_RAILS_INTEGRATION.md) for detailed setup instructions.
|
|
262
111
|
|
|
263
|
-
|
|
264
|
-
{
|
|
265
|
-
"any": [
|
|
266
|
-
{ "field": "escalated", "op": "eq", "value": true },
|
|
267
|
-
{ "field": "complaints", "op": "gte", "value": 3 }
|
|
268
|
-
]
|
|
269
|
-
}
|
|
270
|
-
```
|
|
112
|
+
## Monitoring & Analytics
|
|
271
113
|
|
|
272
|
-
|
|
114
|
+
Real-time monitoring, metrics, and alerting for production environments.
|
|
273
115
|
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
```json
|
|
277
|
-
{
|
|
278
|
-
"field": "user.role",
|
|
279
|
-
"op": "eq",
|
|
280
|
-
"value": "admin"
|
|
281
|
-
}
|
|
282
|
-
```
|
|
116
|
+
### Quick Start
|
|
283
117
|
|
|
284
118
|
```ruby
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
})
|
|
288
|
-
```
|
|
119
|
+
require 'decision_agent/monitoring/metrics_collector'
|
|
120
|
+
require 'decision_agent/monitoring/dashboard_server'
|
|
289
121
|
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
```json
|
|
293
|
-
{
|
|
294
|
-
"version": "1.0",
|
|
295
|
-
"ruleset": "redmine_triage",
|
|
296
|
-
"rules": [
|
|
297
|
-
{
|
|
298
|
-
"id": "critical_escalation",
|
|
299
|
-
"if": {
|
|
300
|
-
"all": [
|
|
301
|
-
{ "field": "priority", "op": "eq", "value": "critical" },
|
|
302
|
-
{
|
|
303
|
-
"any": [
|
|
304
|
-
{ "field": "hours_inactive", "op": "gte", "value": 2 },
|
|
305
|
-
{ "field": "customer_escalated", "op": "eq", "value": true }
|
|
306
|
-
]
|
|
307
|
-
}
|
|
308
|
-
]
|
|
309
|
-
},
|
|
310
|
-
"then": {
|
|
311
|
-
"decision": "escalate_immediately",
|
|
312
|
-
"weight": 1.0,
|
|
313
|
-
"reason": "Critical issue requires immediate attention"
|
|
314
|
-
}
|
|
315
|
-
},
|
|
316
|
-
{
|
|
317
|
-
"id": "auto_assign",
|
|
318
|
-
"if": {
|
|
319
|
-
"all": [
|
|
320
|
-
{ "field": "assignee", "op": "blank" },
|
|
321
|
-
{ "field": "priority", "op": "in", "value": ["high", "critical"] }
|
|
322
|
-
]
|
|
323
|
-
},
|
|
324
|
-
"then": {
|
|
325
|
-
"decision": "assign_to_team_lead",
|
|
326
|
-
"weight": 0.85,
|
|
327
|
-
"reason": "High priority issue needs assignment"
|
|
328
|
-
}
|
|
329
|
-
}
|
|
330
|
-
]
|
|
331
|
-
}
|
|
332
|
-
```
|
|
122
|
+
# Initialize metrics collection
|
|
123
|
+
collector = DecisionAgent::Monitoring::MetricsCollector.new(window_size: 3600)
|
|
333
124
|
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
### Strict Mode
|
|
339
|
-
|
|
340
|
-
Fails if replayed decision differs from original:
|
|
341
|
-
|
|
342
|
-
```ruby
|
|
343
|
-
original_result = agent.decide(context: { user: "alice" })
|
|
344
|
-
|
|
345
|
-
# Later, replay the exact decision
|
|
346
|
-
replayed_result = DecisionAgent::Replay.run(
|
|
347
|
-
original_result.audit_payload,
|
|
348
|
-
strict: true
|
|
125
|
+
# Start real-time dashboard
|
|
126
|
+
DecisionAgent::Monitoring::DashboardServer.start!(
|
|
127
|
+
port: 4568,
|
|
128
|
+
metrics_collector: collector
|
|
349
129
|
)
|
|
350
130
|
|
|
351
|
-
#
|
|
131
|
+
# Record decisions
|
|
132
|
+
agent = DecisionAgent::Agent.new(evaluators: [evaluator])
|
|
133
|
+
result = agent.decide(context: { amount: 1500 })
|
|
134
|
+
collector.record_decision(result, context, duration_ms: 25.5)
|
|
352
135
|
```
|
|
353
136
|
|
|
354
|
-
|
|
137
|
+
Open [http://localhost:4568](http://localhost:4568) for the monitoring dashboard.
|
|
355
138
|
|
|
356
|
-
|
|
139
|
+
### Features
|
|
357
140
|
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
141
|
+
- **Real-time Dashboard** - Live metrics with WebSocket updates
|
|
142
|
+
- **Prometheus Export** - Industry-standard metrics format
|
|
143
|
+
- **Intelligent Alerting** - Anomaly detection with customizable rules
|
|
144
|
+
- **Grafana Integration** - Pre-built dashboards and alert rules
|
|
145
|
+
- **Custom KPIs** - Track business-specific metrics
|
|
146
|
+
- **Thread-Safe** - Production-ready performance
|
|
364
147
|
|
|
365
|
-
###
|
|
148
|
+
### Prometheus & Grafana
|
|
366
149
|
|
|
367
|
-
```
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
decision: "approve",
|
|
375
|
-
weight: 0.8,
|
|
376
|
-
reason: "Rule matched",
|
|
377
|
-
evaluator_name: "JsonRuleEvaluator",
|
|
378
|
-
metadata: { rule_id: "high_priority_rule" }
|
|
379
|
-
}
|
|
380
|
-
],
|
|
381
|
-
decision: "approve",
|
|
382
|
-
confidence: 0.8,
|
|
383
|
-
scoring_strategy: "DecisionAgent::Scoring::WeightedAverage",
|
|
384
|
-
agent_version: "0.1.0",
|
|
385
|
-
deterministic_hash: "a3f2b9c..."
|
|
386
|
-
}
|
|
150
|
+
```yaml
|
|
151
|
+
# prometheus.yml
|
|
152
|
+
scrape_configs:
|
|
153
|
+
- job_name: 'decision_agent'
|
|
154
|
+
static_configs:
|
|
155
|
+
- targets: ['localhost:4568']
|
|
156
|
+
metrics_path: '/metrics'
|
|
387
157
|
```
|
|
388
158
|
|
|
389
|
-
|
|
159
|
+
Import the pre-built Grafana dashboard from [grafana/decision_agent_dashboard.json](grafana/decision_agent_dashboard.json).
|
|
390
160
|
|
|
391
|
-
###
|
|
161
|
+
### Alert Management
|
|
392
162
|
|
|
393
163
|
```ruby
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
)
|
|
397
|
-
|
|
398
|
-
ml_evaluator = DecisionAgent::Evaluators::StaticEvaluator.new(
|
|
399
|
-
decision: "review_manually",
|
|
400
|
-
weight: 0.6,
|
|
401
|
-
reason: "ML model suggests manual review"
|
|
402
|
-
)
|
|
403
|
-
|
|
404
|
-
agent = DecisionAgent::Agent.new(
|
|
405
|
-
evaluators: [rule_evaluator, ml_evaluator],
|
|
406
|
-
scoring_strategy: DecisionAgent::Scoring::Consensus.new(minimum_agreement: 0.7)
|
|
164
|
+
alert_manager = DecisionAgent::Monitoring::AlertManager.new(
|
|
165
|
+
metrics_collector: collector
|
|
407
166
|
)
|
|
408
167
|
|
|
409
|
-
|
|
410
|
-
|
|
168
|
+
# Add alert rules
|
|
169
|
+
alert_manager.add_rule(
|
|
170
|
+
name: 'High Error Rate',
|
|
171
|
+
condition: AlertManager.high_error_rate(threshold: 0.1),
|
|
172
|
+
severity: :critical
|
|
411
173
|
)
|
|
412
174
|
|
|
413
|
-
#
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
### Custom Evaluator
|
|
418
|
-
|
|
419
|
-
```ruby
|
|
420
|
-
class CustomBusinessLogicEvaluator < DecisionAgent::Evaluators::Base
|
|
421
|
-
def evaluate(context, feedback: {})
|
|
422
|
-
# Your custom logic here
|
|
423
|
-
if context[:revenue] > 100_000 && context[:customer_tier] == "enterprise"
|
|
424
|
-
DecisionAgent::Evaluation.new(
|
|
425
|
-
decision: "approve_immediately",
|
|
426
|
-
weight: 0.95,
|
|
427
|
-
reason: "High-value enterprise customer",
|
|
428
|
-
evaluator_name: "EnterpriseCustomerEvaluator",
|
|
429
|
-
metadata: { tier: "enterprise" }
|
|
430
|
-
)
|
|
431
|
-
else
|
|
432
|
-
nil # No decision
|
|
433
|
-
end
|
|
434
|
-
end
|
|
175
|
+
# Register alert handlers
|
|
176
|
+
alert_manager.add_handler do |alert|
|
|
177
|
+
SlackNotifier.notify("🚨 #{alert[:message]}")
|
|
435
178
|
end
|
|
436
|
-
```
|
|
437
|
-
|
|
438
|
-
### Custom Scoring Strategy
|
|
439
179
|
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
def score(evaluations)
|
|
443
|
-
# If any evaluator says "reject", veto everything
|
|
444
|
-
if evaluations.any? { |e| e.decision == "reject" }
|
|
445
|
-
return { decision: "reject", confidence: 1.0 }
|
|
446
|
-
end
|
|
447
|
-
|
|
448
|
-
# Otherwise, use max weight
|
|
449
|
-
max_eval = evaluations.max_by(&:weight)
|
|
450
|
-
{
|
|
451
|
-
decision: max_eval.decision,
|
|
452
|
-
confidence: normalize_confidence(max_eval.weight)
|
|
453
|
-
}
|
|
454
|
-
end
|
|
455
|
-
end
|
|
456
|
-
|
|
457
|
-
agent = DecisionAgent::Agent.new(
|
|
458
|
-
evaluators: [...],
|
|
459
|
-
scoring_strategy: VetoScoring.new
|
|
460
|
-
)
|
|
180
|
+
# Start monitoring
|
|
181
|
+
alert_manager.start_monitoring(interval: 60)
|
|
461
182
|
```
|
|
462
183
|
|
|
463
|
-
|
|
184
|
+
See [Monitoring & Analytics Guide](wiki/MONITORING_AND_ANALYTICS.md) for complete documentation.
|
|
464
185
|
|
|
465
|
-
```ruby
|
|
466
|
-
class DatabaseAuditAdapter < DecisionAgent::Audit::Adapter
|
|
467
|
-
def record(decision, context)
|
|
468
|
-
AuditLog.create!(
|
|
469
|
-
decision: decision.decision,
|
|
470
|
-
confidence: decision.confidence,
|
|
471
|
-
context_json: context.to_h.to_json,
|
|
472
|
-
audit_payload: decision.audit_payload.to_json,
|
|
473
|
-
deterministic_hash: decision.audit_payload[:deterministic_hash]
|
|
474
|
-
)
|
|
475
|
-
end
|
|
476
|
-
end
|
|
477
|
-
```
|
|
478
|
-
|
|
479
|
-
### Feedback Loop
|
|
480
|
-
|
|
481
|
-
The `feedback` parameter allows you to pass additional context about past decisions, manual overrides, or external signals that can influence decision-making in custom evaluators.
|
|
482
|
-
|
|
483
|
-
#### Built-in Evaluators and Feedback
|
|
484
|
-
|
|
485
|
-
**Built-in evaluators** (`JsonRuleEvaluator`, `StaticEvaluator`) **ignore feedback** to maintain determinism. This is intentional - the same context should always produce the same decision for auditability and replay purposes.
|
|
486
|
-
|
|
487
|
-
```ruby
|
|
488
|
-
# Feedback is accepted but not used by built-in evaluators
|
|
489
|
-
result = agent.decide(
|
|
490
|
-
context: { issue_id: 123 },
|
|
491
|
-
feedback: { source: "automated", past_accuracy: 0.95 }
|
|
492
|
-
)
|
|
493
186
|
|
|
494
|
-
|
|
495
|
-
puts result.audit_payload[:feedback] # => { source: "automated", past_accuracy: 0.95 }
|
|
496
|
-
```
|
|
187
|
+
## Key Features
|
|
497
188
|
|
|
498
|
-
|
|
189
|
+
### Decision Making
|
|
190
|
+
- **Multiple Evaluators** - Combine rule-based, ML, and custom logic
|
|
191
|
+
- **Conflict Resolution** - Weighted average, consensus, threshold, max weight
|
|
192
|
+
- **Rich Context** - Nested data, dot notation, flexible operators
|
|
499
193
|
|
|
500
|
-
|
|
194
|
+
### Auditability
|
|
195
|
+
- **Complete Audit Trails** - Every decision fully logged
|
|
196
|
+
- **Deterministic Replay** - Reproduce historical decisions exactly
|
|
197
|
+
- **Compliance Ready** - HIPAA, SOX, regulatory compliance support
|
|
501
198
|
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
if feedback[:override]
|
|
508
|
-
return Evaluation.new(
|
|
509
|
-
decision: feedback[:override],
|
|
510
|
-
weight: 0.9,
|
|
511
|
-
reason: feedback[:reason] || "Manual override",
|
|
512
|
-
evaluator_name: evaluator_name
|
|
513
|
-
)
|
|
514
|
-
end
|
|
515
|
-
|
|
516
|
-
# Or adjust confidence based on past accuracy
|
|
517
|
-
adjusted_weight = base_weight * feedback[:past_accuracy].to_f
|
|
518
|
-
|
|
519
|
-
Evaluation.new(
|
|
520
|
-
decision: base_decision,
|
|
521
|
-
weight: adjusted_weight,
|
|
522
|
-
reason: "Adjusted by past performance",
|
|
523
|
-
evaluator_name: evaluator_name
|
|
524
|
-
)
|
|
525
|
-
end
|
|
526
|
-
end
|
|
527
|
-
```
|
|
199
|
+
### Flexibility
|
|
200
|
+
- **Pluggable Architecture** - Custom evaluators, scoring, audit adapters
|
|
201
|
+
- **Framework Agnostic** - Works with Rails, Sinatra, or standalone
|
|
202
|
+
- **JSON Rule DSL** - Non-technical users can write rules
|
|
203
|
+
- **Visual Rule Builder** - Web UI for rule management
|
|
528
204
|
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
|
|
535
|
-
|
|
536
|
-
|
|
537
|
-
```
|
|
538
|
-
|
|
539
|
-
2. **Historical Performance**: Adjust confidence based on past accuracy
|
|
540
|
-
```ruby
|
|
541
|
-
agent.decide(
|
|
542
|
-
context: { transaction: tx },
|
|
543
|
-
feedback: { past_accuracy: 0.87 } # This evaluator was 87% accurate historically
|
|
544
|
-
)
|
|
545
|
-
```
|
|
546
|
-
|
|
547
|
-
3. **Source Attribution**: Weight decisions differently based on origin
|
|
548
|
-
```ruby
|
|
549
|
-
agent.decide(
|
|
550
|
-
context: { issue: issue },
|
|
551
|
-
feedback: { source: "expert_review" } # Higher confidence for expert reviews
|
|
552
|
-
)
|
|
553
|
-
```
|
|
554
|
-
|
|
555
|
-
4. **Learning Signals**: Collect data for offline model training
|
|
556
|
-
```ruby
|
|
557
|
-
# Initial decision
|
|
558
|
-
result = agent.decide(context: { user: user })
|
|
559
|
-
|
|
560
|
-
# Later: user provides feedback
|
|
561
|
-
user_feedback = {
|
|
562
|
-
correct: false,
|
|
563
|
-
actual_decision: "escalate",
|
|
564
|
-
user_id: "manager_bob",
|
|
565
|
-
timestamp: Time.now.utc.iso8601
|
|
566
|
-
}
|
|
567
|
-
|
|
568
|
-
# Log for analysis and future rule adjustments
|
|
569
|
-
# (DecisionAgent doesn't auto-update rules - this is for your ML/analysis pipeline)
|
|
570
|
-
FeedbackLog.create(
|
|
571
|
-
decision_hash: result.audit_payload[:deterministic_hash],
|
|
572
|
-
predicted: result.decision,
|
|
573
|
-
actual: user_feedback[:actual_decision],
|
|
574
|
-
feedback: user_feedback
|
|
575
|
-
)
|
|
576
|
-
```
|
|
577
|
-
|
|
578
|
-
#### Example: Complete Feedback-Aware System
|
|
579
|
-
|
|
580
|
-
See [examples/feedback_aware_evaluator.rb](examples/feedback_aware_evaluator.rb) for a complete example that demonstrates:
|
|
581
|
-
- Manual overrides with high confidence
|
|
582
|
-
- Past accuracy-based weight adjustment
|
|
583
|
-
- Source-based confidence boosting
|
|
584
|
-
- Comprehensive metadata tracking
|
|
585
|
-
|
|
586
|
-
**Key Principle**: Use feedback for **human oversight** and **continuous improvement**, but keep the core decision logic deterministic and auditable.
|
|
587
|
-
|
|
588
|
-
## Integration Examples
|
|
589
|
-
|
|
590
|
-
### Rails Integration
|
|
205
|
+
### Monitoring & Observability
|
|
206
|
+
- **Real-time Metrics** - Live dashboard with WebSocket updates (<1 second latency)
|
|
207
|
+
- **Prometheus Export** - Industry-standard metrics format at `/metrics` endpoint
|
|
208
|
+
- **Intelligent Alerting** - Anomaly detection with customizable rules and severity levels
|
|
209
|
+
- **Grafana Integration** - Pre-built dashboards and alert configurations in `grafana/` directory
|
|
210
|
+
- **Custom KPIs** - Track business-specific metrics with thread-safe operations
|
|
211
|
+
- **MonitoredAgent** - Drop-in replacement that auto-records all metrics
|
|
212
|
+
- **AlertManager** - Built-in anomaly detection (error rates, latency spikes, low confidence)
|
|
591
213
|
|
|
592
|
-
|
|
593
|
-
|
|
594
|
-
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
result = agent.decide(
|
|
599
|
-
context: {
|
|
600
|
-
priority: issue.priority,
|
|
601
|
-
hours_inactive: (Time.now - issue.updated_at) / 3600,
|
|
602
|
-
assignee: issue.assignee&.login,
|
|
603
|
-
status: issue.status
|
|
604
|
-
}
|
|
605
|
-
)
|
|
606
|
-
|
|
607
|
-
result
|
|
608
|
-
end
|
|
609
|
-
|
|
610
|
-
private
|
|
611
|
-
|
|
612
|
-
def self.build_agent
|
|
613
|
-
rules = JSON.parse(File.read(Rails.root.join("config/rules/issue_triage.json")))
|
|
614
|
-
|
|
615
|
-
DecisionAgent::Agent.new(
|
|
616
|
-
evaluators: [
|
|
617
|
-
DecisionAgent::Evaluators::JsonRuleEvaluator.new(rules_json: rules)
|
|
618
|
-
],
|
|
619
|
-
scoring_strategy: DecisionAgent::Scoring::WeightedAverage.new,
|
|
620
|
-
audit_adapter: DecisionAgent::Audit::LoggerAdapter.new(logger: Rails.logger)
|
|
621
|
-
)
|
|
622
|
-
end
|
|
623
|
-
end
|
|
624
|
-
```
|
|
214
|
+
### Production Ready
|
|
215
|
+
- **Comprehensive Testing** - 90%+ code coverage
|
|
216
|
+
- **Error Handling** - Clear, actionable error messages
|
|
217
|
+
- **Versioning** - Full rule version control and rollback
|
|
218
|
+
- **Performance** - Fast, zero external dependencies
|
|
219
|
+
- **Thread-Safe** - Safe for multi-threaded servers and background jobs
|
|
625
220
|
|
|
626
|
-
|
|
221
|
+
## Examples
|
|
627
222
|
|
|
628
223
|
```ruby
|
|
629
|
-
#
|
|
630
|
-
module RedmineSmartTriage
|
|
631
|
-
class DecisionEngine
|
|
632
|
-
def self.evaluate_issue(issue)
|
|
633
|
-
agent = build_agent
|
|
634
|
-
|
|
635
|
-
context = {
|
|
636
|
-
"priority" => issue.priority.name.downcase,
|
|
637
|
-
"status" => issue.status.name.downcase,
|
|
638
|
-
"hours_inactive" => hours_since_update(issue),
|
|
639
|
-
"assignee" => issue.assigned_to&.login,
|
|
640
|
-
"tracker" => issue.tracker.name.downcase
|
|
641
|
-
}
|
|
642
|
-
|
|
643
|
-
agent.decide(context: context)
|
|
644
|
-
end
|
|
645
|
-
|
|
646
|
-
private
|
|
647
|
-
|
|
648
|
-
def self.build_agent
|
|
649
|
-
rules_path = File.join(File.dirname(__FILE__), "../config/triage_rules.json")
|
|
650
|
-
rules = JSON.parse(File.read(rules_path))
|
|
651
|
-
|
|
652
|
-
DecisionAgent::Agent.new(
|
|
653
|
-
evaluators: [
|
|
654
|
-
DecisionAgent::Evaluators::JsonRuleEvaluator.new(rules_json: rules)
|
|
655
|
-
],
|
|
656
|
-
audit_adapter: RedmineAuditAdapter.new
|
|
657
|
-
)
|
|
658
|
-
end
|
|
659
|
-
|
|
660
|
-
def self.hours_since_update(issue)
|
|
661
|
-
((Time.now - issue.updated_on) / 3600).round
|
|
662
|
-
end
|
|
663
|
-
end
|
|
664
|
-
|
|
665
|
-
class RedmineAuditAdapter < DecisionAgent::Audit::Adapter
|
|
666
|
-
def record(decision, context)
|
|
667
|
-
# Store in Redmine custom field or separate table
|
|
668
|
-
Rails.logger.info "[DecisionAgent] #{decision.decision} (confidence: #{decision.confidence})"
|
|
669
|
-
end
|
|
670
|
-
end
|
|
671
|
-
end
|
|
672
|
-
```
|
|
673
|
-
|
|
674
|
-
### Standalone Service
|
|
675
|
-
|
|
676
|
-
```ruby
|
|
677
|
-
#!/usr/bin/env ruby
|
|
678
|
-
require 'decision_agent'
|
|
679
|
-
require 'json'
|
|
680
|
-
|
|
681
|
-
# Load rules
|
|
682
|
-
rules = JSON.parse(File.read("config/rules.json"))
|
|
683
|
-
|
|
684
|
-
# Build agent
|
|
224
|
+
# Multiple evaluators with conflict resolution
|
|
685
225
|
agent = DecisionAgent::Agent.new(
|
|
686
|
-
evaluators: [
|
|
687
|
-
|
|
688
|
-
],
|
|
689
|
-
scoring_strategy: DecisionAgent::Scoring::Threshold.new(
|
|
690
|
-
threshold: 0.75,
|
|
691
|
-
fallback_decision: "manual_review"
|
|
692
|
-
),
|
|
226
|
+
evaluators: [rule_evaluator, ml_evaluator],
|
|
227
|
+
scoring_strategy: DecisionAgent::Scoring::Consensus.new(minimum_agreement: 0.7),
|
|
693
228
|
audit_adapter: DecisionAgent::Audit::LoggerAdapter.new
|
|
694
229
|
)
|
|
695
230
|
|
|
696
|
-
#
|
|
697
|
-
|
|
698
|
-
|
|
699
|
-
|
|
700
|
-
|
|
701
|
-
|
|
702
|
-
|
|
703
|
-
|
|
704
|
-
|
|
705
|
-
|
|
706
|
-
|
|
231
|
+
# Complex rules with nested conditions
|
|
232
|
+
rules = {
|
|
233
|
+
version: "1.0",
|
|
234
|
+
ruleset: "fraud_detection",
|
|
235
|
+
rules: [{
|
|
236
|
+
id: "suspicious_activity",
|
|
237
|
+
if: {
|
|
238
|
+
all: [
|
|
239
|
+
{ field: "amount", op: "gt", value: 10000 },
|
|
240
|
+
{ any: [
|
|
241
|
+
{ field: "user.country", op: "in", value: ["XX", "YY"] },
|
|
242
|
+
{ field: "velocity", op: "gt", value: 5 }
|
|
243
|
+
]}
|
|
244
|
+
]
|
|
245
|
+
},
|
|
246
|
+
then: { decision: "flag_for_review", weight: 0.95, reason: "Suspicious patterns detected" }
|
|
247
|
+
}]
|
|
707
248
|
}
|
|
708
|
-
|
|
709
|
-
puts JSON.pretty_generate(output)
|
|
710
|
-
```
|
|
711
|
-
|
|
712
|
-
## Design Philosophy
|
|
713
|
-
|
|
714
|
-
### Why Deterministic > AI
|
|
715
|
-
|
|
716
|
-
1. **Regulatory Compliance**: Healthcare (HIPAA), finance (SOX), and government require auditable, explainable decisions
|
|
717
|
-
2. **Cost**: Rules are free to evaluate; LLM calls cost money and add latency
|
|
718
|
-
3. **Reliability**: Same input must produce same output for testing and legal defensibility
|
|
719
|
-
4. **Transparency**: Business rules are explicit and reviewable by domain experts
|
|
720
|
-
5. **AI Enhancement**: AI can suggest rule adjustments, but rules make final decisions
|
|
721
|
-
|
|
722
|
-
### When to Use DecisionAgent
|
|
723
|
-
|
|
724
|
-
- **Regulated domains**: Healthcare, finance, legal, government
|
|
725
|
-
- **Business rule engines**: Complex decision trees with multiple evaluators
|
|
726
|
-
- **Compliance requirements**: Need full audit trails and decision replay
|
|
727
|
-
- **Explainability required**: Humans must understand why decisions were made
|
|
728
|
-
- **Deterministic systems**: Same input must always produce same output
|
|
729
|
-
|
|
730
|
-
### When NOT to Use
|
|
731
|
-
|
|
732
|
-
- Simple if/else logic (just use Ruby)
|
|
733
|
-
- Purely AI-driven decisions with no rules
|
|
734
|
-
- Single-step validations (use standard validation libraries)
|
|
735
|
-
|
|
736
|
-
## Testing
|
|
737
|
-
|
|
738
|
-
```ruby
|
|
739
|
-
# spec/my_decision_spec.rb
|
|
740
|
-
RSpec.describe "My Decision Logic" do
|
|
741
|
-
it "escalates critical issues" do
|
|
742
|
-
rules = { ... }
|
|
743
|
-
evaluator = DecisionAgent::Evaluators::JsonRuleEvaluator.new(rules_json: rules)
|
|
744
|
-
agent = DecisionAgent::Agent.new(evaluators: [evaluator])
|
|
745
|
-
|
|
746
|
-
result = agent.decide(
|
|
747
|
-
context: { priority: "critical", hours_inactive: 3 }
|
|
748
|
-
)
|
|
749
|
-
|
|
750
|
-
expect(result.decision).to eq("escalate")
|
|
751
|
-
expect(result.confidence).to be > 0.8
|
|
752
|
-
end
|
|
753
|
-
end
|
|
754
|
-
```
|
|
755
|
-
|
|
756
|
-
## Error Handling
|
|
757
|
-
|
|
758
|
-
All errors are namespaced under `DecisionAgent`:
|
|
759
|
-
|
|
760
|
-
### NoEvaluationsError
|
|
761
|
-
|
|
762
|
-
Raised when no evaluator returns a decision (all returned `nil` or raised exceptions).
|
|
763
|
-
|
|
764
|
-
```ruby
|
|
765
|
-
begin
|
|
766
|
-
agent.decide(context: {})
|
|
767
|
-
rescue DecisionAgent::NoEvaluationsError => e
|
|
768
|
-
# No evaluator returned a decision
|
|
769
|
-
puts e.message # => "No evaluators returned a decision"
|
|
770
|
-
|
|
771
|
-
# Handle gracefully
|
|
772
|
-
fallback_decision = "manual_review"
|
|
773
|
-
end
|
|
774
|
-
```
|
|
775
|
-
|
|
776
|
-
### InvalidRuleDslError
|
|
777
|
-
|
|
778
|
-
Raised when JSON rule DSL is malformed or invalid.
|
|
779
|
-
|
|
780
|
-
```ruby
|
|
781
|
-
begin
|
|
782
|
-
rules = { invalid: "structure" }
|
|
783
|
-
evaluator = DecisionAgent::Evaluators::JsonRuleEvaluator.new(rules_json: rules)
|
|
784
|
-
rescue DecisionAgent::InvalidRuleDslError => e
|
|
785
|
-
# JSON rule DSL is malformed
|
|
786
|
-
puts e.message # => "Invalid rule DSL structure"
|
|
787
|
-
end
|
|
788
|
-
```
|
|
789
|
-
|
|
790
|
-
### ReplayMismatchError
|
|
791
|
-
|
|
792
|
-
Raised in strict replay mode when replayed decision differs from original.
|
|
793
|
-
|
|
794
|
-
```ruby
|
|
795
|
-
begin
|
|
796
|
-
replayed_result = DecisionAgent::Replay.run(audit_payload, strict: true)
|
|
797
|
-
rescue DecisionAgent::ReplayMismatchError => e
|
|
798
|
-
# Replay produced different result
|
|
799
|
-
puts "Expected: #{e.expected}" # => "approve"
|
|
800
|
-
puts "Actual: #{e.actual}" # => "reject"
|
|
801
|
-
puts "Differences: #{e.differences}" # => ["decision changed", "confidence changed"]
|
|
802
|
-
end
|
|
803
249
|
```
|
|
804
250
|
|
|
805
|
-
|
|
251
|
+
See [examples/](examples/) for complete working examples.
|
|
806
252
|
|
|
807
|
-
|
|
253
|
+
## Thread-Safety Guarantees
|
|
808
254
|
|
|
809
|
-
|
|
810
|
-
begin
|
|
811
|
-
decision = DecisionAgent::Decision.new(
|
|
812
|
-
decision: "approve",
|
|
813
|
-
confidence: 1.5, # Invalid!
|
|
814
|
-
explanations: [],
|
|
815
|
-
evaluations: [],
|
|
816
|
-
audit_payload: {}
|
|
817
|
-
)
|
|
818
|
-
rescue DecisionAgent::InvalidConfidenceError => e
|
|
819
|
-
puts e.message # => "Confidence must be between 0.0 and 1.0, got: 1.5"
|
|
820
|
-
end
|
|
821
|
-
```
|
|
255
|
+
DecisionAgent is designed to be **thread-safe and FAST** for use in multi-threaded environments:
|
|
822
256
|
|
|
823
|
-
###
|
|
257
|
+
### Performance
|
|
258
|
+
- **10,000+ decisions/second** throughput
|
|
259
|
+
- **~0.1ms average latency** per decision
|
|
260
|
+
- **Zero performance overhead** from thread-safety
|
|
261
|
+
- **Linear scalability** with thread count
|
|
824
262
|
|
|
825
|
-
|
|
263
|
+
### Safe Concurrent Usage
|
|
264
|
+
- **Agent instances** can be shared across threads safely
|
|
265
|
+
- **Evaluators** are immutable after initialization
|
|
266
|
+
- **Decisions and Evaluations** are deeply frozen
|
|
267
|
+
- **File storage** uses mutex-protected operations
|
|
826
268
|
|
|
269
|
+
### Best Practices
|
|
827
270
|
```ruby
|
|
828
|
-
|
|
829
|
-
|
|
830
|
-
decision: "approve",
|
|
831
|
-
weight: -0.5, # Invalid!
|
|
832
|
-
reason: "Test",
|
|
833
|
-
evaluator_name: "Test"
|
|
834
|
-
)
|
|
835
|
-
rescue DecisionAgent::InvalidWeightError => e
|
|
836
|
-
puts e.message # => "Weight must be between 0.0 and 1.0, got: -0.5"
|
|
837
|
-
end
|
|
838
|
-
```
|
|
839
|
-
|
|
840
|
-
### Configuration Errors
|
|
841
|
-
|
|
842
|
-
Raised during agent initialization when configuration is invalid.
|
|
271
|
+
# Safe: Reuse agent instance across threads
|
|
272
|
+
agent = DecisionAgent::Agent.new(evaluators: [evaluator])
|
|
843
273
|
|
|
844
|
-
|
|
845
|
-
|
|
846
|
-
# No evaluators provided
|
|
847
|
-
agent = DecisionAgent::Agent.new(evaluators: [])
|
|
848
|
-
rescue DecisionAgent::InvalidConfigurationError => e
|
|
849
|
-
puts e.message # => "At least one evaluator is required"
|
|
850
|
-
end
|
|
274
|
+
Thread.new { agent.decide(context: { user_id: 1 }) }
|
|
275
|
+
Thread.new { agent.decide(context: { user_id: 2 }) }
|
|
851
276
|
|
|
852
|
-
|
|
853
|
-
|
|
854
|
-
|
|
855
|
-
|
|
856
|
-
puts e.message # => "Evaluator must respond to #evaluate"
|
|
857
|
-
end
|
|
277
|
+
# Safe: Share evaluators across agent instances
|
|
278
|
+
evaluator = DecisionAgent::Evaluators::JsonRuleEvaluator.new(rules_json: rules)
|
|
279
|
+
agent1 = DecisionAgent::Agent.new(evaluators: [evaluator])
|
|
280
|
+
agent2 = DecisionAgent::Agent.new(evaluators: [evaluator])
|
|
858
281
|
```
|
|
859
282
|
|
|
860
|
-
|
|
861
|
-
|
|
862
|
-
|
|
283
|
+
### What's Frozen
|
|
284
|
+
All data structures are deeply frozen to prevent mutation:
|
|
285
|
+
- Decision objects (decision, confidence, explanations, evaluations)
|
|
286
|
+
- Evaluation objects (decision, weight, reason, metadata)
|
|
287
|
+
- Context data
|
|
288
|
+
- Rule definitions in evaluators
|
|
863
289
|
|
|
864
|
-
|
|
865
|
-
|
|
866
|
-
**Constructor:**
|
|
867
|
-
```ruby
|
|
868
|
-
DecisionAgent::Agent.new(
|
|
869
|
-
evaluators: [evaluator1, evaluator2],
|
|
870
|
-
scoring_strategy: DecisionAgent::Scoring::WeightedAverage.new, # Optional, defaults to WeightedAverage
|
|
871
|
-
audit_adapter: DecisionAgent::Audit::NullAdapter.new # Optional, defaults to NullAdapter
|
|
872
|
-
)
|
|
873
|
-
```
|
|
290
|
+
This ensures safe concurrent access without race conditions.
|
|
874
291
|
|
|
875
|
-
|
|
292
|
+
### RFC 8785 Canonical JSON
|
|
293
|
+
DecisionAgent uses **RFC 8785 (JSON Canonicalization Scheme)** for deterministic audit hashing:
|
|
876
294
|
|
|
877
|
-
-
|
|
878
|
-
|
|
879
|
-
|
|
880
|
-
|
|
295
|
+
- **Industry Standard** - Official IETF specification for canonical JSON
|
|
296
|
+
- **Cryptographically Sound** - Ensures deterministic hashing of decision payloads
|
|
297
|
+
- **Reproducible** - Same decision always produces same audit hash
|
|
298
|
+
- **Interoperable** - Compatible with other systems using RFC 8785
|
|
881
299
|
|
|
882
|
-
|
|
883
|
-
-
|
|
884
|
-
-
|
|
885
|
-
-
|
|
886
|
-
|
|
887
|
-
### Decision
|
|
888
|
-
|
|
889
|
-
Immutable result object representing a decision.
|
|
890
|
-
|
|
891
|
-
**Constructor:**
|
|
892
|
-
```ruby
|
|
893
|
-
DecisionAgent::Decision.new(
|
|
894
|
-
decision: "approve",
|
|
895
|
-
confidence: 0.85,
|
|
896
|
-
explanations: ["High priority rule matched"],
|
|
897
|
-
evaluations: [evaluation1, evaluation2],
|
|
898
|
-
audit_payload: {...}
|
|
899
|
-
)
|
|
900
|
-
```
|
|
901
|
-
|
|
902
|
-
**Attributes:**
|
|
903
|
-
- `#decision` → `String` - The final decision (frozen)
|
|
904
|
-
- `#confidence` → `Float` - Confidence score between 0.0 and 1.0
|
|
905
|
-
- `#explanations` → `Array<String>` - Human-readable explanations (frozen)
|
|
906
|
-
- `#evaluations` → `Array<Evaluation>` - All evaluations that contributed (frozen)
|
|
907
|
-
- `#audit_payload` → `Hash` - Complete audit trail for replay (frozen)
|
|
908
|
-
|
|
909
|
-
**Public Methods:**
|
|
910
|
-
|
|
911
|
-
- `#to_h` → `Hash` - Converts to hash representation
|
|
912
|
-
- `#==(other)` → `Boolean` - Equality comparison (compares decision, confidence, explanations, evaluations)
|
|
913
|
-
|
|
914
|
-
### Evaluation
|
|
915
|
-
|
|
916
|
-
Immutable result from a single evaluator.
|
|
917
|
-
|
|
918
|
-
**Constructor:**
|
|
919
|
-
```ruby
|
|
920
|
-
DecisionAgent::Evaluation.new(
|
|
921
|
-
decision: "approve",
|
|
922
|
-
weight: 0.8,
|
|
923
|
-
reason: "User meets criteria",
|
|
924
|
-
evaluator_name: "MyEvaluator",
|
|
925
|
-
metadata: { rule_id: "R1" } # Optional, defaults to {}
|
|
926
|
-
)
|
|
927
|
-
```
|
|
300
|
+
Every decision includes a deterministic SHA-256 hash in the audit payload, enabling:
|
|
301
|
+
- Tamper detection in audit logs
|
|
302
|
+
- Exact replay verification
|
|
303
|
+
- Regulatory compliance documentation
|
|
928
304
|
|
|
929
|
-
|
|
930
|
-
- `#decision` → `String` - The evaluator's decision (frozen)
|
|
931
|
-
- `#weight` → `Float` - Weight between 0.0 and 1.0
|
|
932
|
-
- `#reason` → `String` - Human-readable reason (frozen)
|
|
933
|
-
- `#evaluator_name` → `String` - Name of the evaluator (frozen)
|
|
934
|
-
- `#metadata` → `Hash` - Additional context (frozen)
|
|
305
|
+
Learn more: [RFC 8785 Specification](https://datatracker.ietf.org/doc/html/rfc8785)
|
|
935
306
|
|
|
936
|
-
|
|
937
|
-
|
|
938
|
-
|
|
939
|
-
|
|
940
|
-
|
|
941
|
-
### Context
|
|
942
|
-
|
|
943
|
-
Immutable wrapper for decision context data.
|
|
944
|
-
|
|
945
|
-
**Constructor:**
|
|
946
|
-
```ruby
|
|
947
|
-
DecisionAgent::Context.new(
|
|
948
|
-
user: "alice",
|
|
949
|
-
priority: "high",
|
|
950
|
-
nested: { role: "admin" }
|
|
951
|
-
)
|
|
307
|
+
### Performance Benchmark
|
|
308
|
+
Run the included benchmark to verify zero overhead:
|
|
309
|
+
```bash
|
|
310
|
+
ruby examples/thread_safe_performance.rb
|
|
952
311
|
```
|
|
953
312
|
|
|
954
|
-
|
|
955
|
-
|
|
956
|
-
- `#[]` → `Object` - Access context value by key (supports both string and symbol keys)
|
|
957
|
-
- `#to_h` → `Hash` - Returns underlying hash (frozen)
|
|
958
|
-
- `#==(other)` → `Boolean` - Equality comparison
|
|
959
|
-
|
|
960
|
-
### Evaluators::Base
|
|
961
|
-
|
|
962
|
-
Base class for custom evaluators.
|
|
963
|
-
|
|
964
|
-
**Public Methods:**
|
|
965
|
-
|
|
966
|
-
- `#evaluate(context, feedback: {})` → `Evaluation | nil`
|
|
967
|
-
- Must be implemented by subclasses
|
|
968
|
-
- Returns `Evaluation` if a decision is made, `nil` otherwise
|
|
969
|
-
- `context` is a `Context` object
|
|
970
|
-
- `feedback` is an optional hash
|
|
971
|
-
|
|
972
|
-
### Scoring::Base
|
|
973
|
-
|
|
974
|
-
Base class for custom scoring strategies.
|
|
975
|
-
|
|
976
|
-
**Public Methods:**
|
|
313
|
+
See [THREAD_SAFETY.md](wiki/THREAD_SAFETY.md) for detailed implementation guide and [PERFORMANCE_AND_THREAD_SAFETY.md](wiki/PERFORMANCE_AND_THREAD_SAFETY.md) for detailed performance analysis.
|
|
977
314
|
|
|
978
|
-
|
|
979
|
-
- Must be implemented by subclasses
|
|
980
|
-
- Takes array of `Evaluation` objects
|
|
981
|
-
- Returns hash with `:decision` and `:confidence` keys
|
|
982
|
-
- Confidence must be between 0.0 and 1.0
|
|
315
|
+
## When to Use DecisionAgent
|
|
983
316
|
|
|
984
|
-
**
|
|
985
|
-
-
|
|
986
|
-
-
|
|
317
|
+
✅ **Perfect for:**
|
|
318
|
+
- Regulated industries (healthcare, finance, legal)
|
|
319
|
+
- Complex business rule engines
|
|
320
|
+
- Audit trail requirements
|
|
321
|
+
- Explainable AI systems
|
|
322
|
+
- Multi-step decision workflows
|
|
987
323
|
|
|
988
|
-
|
|
324
|
+
❌ **Not suitable for:**
|
|
325
|
+
- Simple if/else logic (use plain Ruby)
|
|
326
|
+
- Pure AI/ML with no rules
|
|
327
|
+
- Single-step validations
|
|
989
328
|
|
|
990
|
-
|
|
329
|
+
## Documentation
|
|
991
330
|
|
|
992
|
-
**
|
|
331
|
+
**Getting Started**
|
|
332
|
+
- [Installation](#installation)
|
|
333
|
+
- [Quick Start](#quick-start)
|
|
334
|
+
- [Examples](examples/README.md)
|
|
993
335
|
|
|
994
|
-
|
|
995
|
-
|
|
996
|
-
|
|
997
|
-
|
|
998
|
-
|
|
336
|
+
**Core Features**
|
|
337
|
+
- [Versioning System](wiki/VERSIONING.md) - Version control for rules
|
|
338
|
+
- [Web UI](wiki/WEB_UI.md) - Visual rule builder
|
|
339
|
+
- [Web UI Setup](wiki/WEB_UI_SETUP.md) - Setup guide
|
|
340
|
+
- [Web UI Rails Integration](wiki/WEB_UI_RAILS_INTEGRATION.md) - Mount in Rails/Rack apps
|
|
341
|
+
- [Monitoring & Analytics](wiki/MONITORING_AND_ANALYTICS.md) - Real-time monitoring, metrics, and alerting
|
|
342
|
+
- [Monitoring Architecture](wiki/MONITORING_ARCHITECTURE.md) - System architecture and design
|
|
999
343
|
|
|
1000
|
-
|
|
344
|
+
**Performance & Thread-Safety**
|
|
345
|
+
- [Performance & Thread-Safety Summary](wiki/PERFORMANCE_AND_THREAD_SAFETY.md) - Benchmarks and production readiness
|
|
346
|
+
- [Thread-Safety Implementation](wiki/THREAD_SAFETY.md) - Technical implementation guide
|
|
1001
347
|
|
|
1002
|
-
|
|
348
|
+
**Reference**
|
|
349
|
+
- [API Contract](wiki/API_CONTRACT.md) - Full API reference
|
|
350
|
+
- [Changelog](wiki/CHANGELOG.md) - Version history
|
|
1003
351
|
|
|
1004
|
-
**
|
|
1005
|
-
|
|
1006
|
-
-
|
|
1007
|
-
- Replays a decision from audit payload
|
|
1008
|
-
- `strict: true` raises `ReplayMismatchError` on differences
|
|
1009
|
-
- `strict: false` logs differences but allows evolution
|
|
1010
|
-
|
|
1011
|
-
## Versioning
|
|
1012
|
-
|
|
1013
|
-
DecisionAgent follows [Semantic Versioning 2.0.0](https://semver.org/):
|
|
1014
|
-
|
|
1015
|
-
- **MAJOR** version for incompatible API changes
|
|
1016
|
-
- **MINOR** version for backwards-compatible functionality additions
|
|
1017
|
-
- **PATCH** version for backwards-compatible bug fixes
|
|
1018
|
-
|
|
1019
|
-
### Stability Guarantees
|
|
1020
|
-
|
|
1021
|
-
- **Public API**: All classes and methods documented in this README are stable
|
|
1022
|
-
- **Audit Payload Format**: The structure of `audit_payload` is stable and will remain replayable across versions
|
|
1023
|
-
- **Deterministic Hash**: The algorithm for computing `deterministic_hash` is frozen to ensure replay compatibility
|
|
1024
|
-
- **Breaking Changes**: Will only occur in major version bumps, with clear migration guides
|
|
1025
|
-
|
|
1026
|
-
### Deprecation Policy
|
|
1027
|
-
|
|
1028
|
-
- Deprecated features will be marked in documentation and emit warnings
|
|
1029
|
-
- Deprecated features will be maintained for at least one minor version before removal
|
|
1030
|
-
- Breaking changes will be documented in CHANGELOG.md with migration instructions
|
|
352
|
+
**More Resources**
|
|
353
|
+
- [Wiki Home](wiki/README.md) - Documentation index
|
|
354
|
+
- [GitHub Issues](https://github.com/samaswin87/decision_agent/issues) - Report bugs or request features
|
|
1031
355
|
|
|
1032
356
|
## Contributing
|
|
1033
357
|
|
|
1034
358
|
1. Fork the repository
|
|
1035
359
|
2. Create a feature branch
|
|
1036
360
|
3. Add tests (maintain 90%+ coverage)
|
|
1037
|
-
4.
|
|
1038
|
-
5. Submit a pull request
|
|
361
|
+
4. Submit a pull request
|
|
1039
362
|
|
|
1040
|
-
##
|
|
1041
|
-
|
|
1042
|
-
MIT License. See [LICENSE.txt](LICENSE.txt).
|
|
1043
|
-
|
|
1044
|
-
## Roadmap
|
|
363
|
+
## Support
|
|
1045
364
|
|
|
1046
|
-
- [
|
|
1047
|
-
- [
|
|
1048
|
-
- [
|
|
1049
|
-
- [ ] Prometheus metrics adapter
|
|
1050
|
-
- [ ] Additional scoring strategies (Bayesian, etc.)
|
|
1051
|
-
- [ ] AI evaluator adapter (optional, non-deterministic mode)
|
|
365
|
+
- **Issues**: [GitHub Issues](https://github.com/samaswin87/decision_agent/issues)
|
|
366
|
+
- **Documentation**: [Wiki](wiki/README.md)
|
|
367
|
+
- **Examples**: [examples/](examples/)
|
|
1052
368
|
|
|
1053
|
-
##
|
|
369
|
+
## License
|
|
1054
370
|
|
|
1055
|
-
-
|
|
1056
|
-
- Documentation: [https://github.com/samaswin87/decision_agent](https://github.com/samaswin87/decision_agent)
|
|
371
|
+
MIT License - see [LICENSE.txt](LICENSE.txt)
|
|
1057
372
|
|
|
1058
373
|
---
|
|
1059
374
|
|
|
1060
|
-
**
|
|
375
|
+
⭐ **Star this repo** if you find it useful!
|