shugocore 1.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- shugocore-1.2.0/LICENSE +21 -0
- shugocore-1.2.0/PKG-INFO +285 -0
- shugocore-1.2.0/README.md +264 -0
- shugocore-1.2.0/acceleration.py +328 -0
- shugocore-1.2.0/android_bridge.py +437 -0
- shugocore-1.2.0/android_node.py +457 -0
- shugocore-1.2.0/android_runtime.py +237 -0
- shugocore-1.2.0/audit.py +162 -0
- shugocore-1.2.0/autonomy.py +134 -0
- shugocore-1.2.0/decision_engine.py +759 -0
- shugocore-1.2.0/execution_layer.py +313 -0
- shugocore-1.2.0/fallbacks.py +188 -0
- shugocore-1.2.0/gazebo_simulation.py +238 -0
- shugocore-1.2.0/logging_manager.py +90 -0
- shugocore-1.2.0/memory_system.py +1028 -0
- shugocore-1.2.0/mobile_nodes.py +398 -0
- shugocore-1.2.0/model_backends.py +154 -0
- shugocore-1.2.0/model_manager.py +102 -0
- shugocore-1.2.0/moveit_planner.py +395 -0
- shugocore-1.2.0/policy.py +326 -0
- shugocore-1.2.0/pyproject.toml +33 -0
- shugocore-1.2.0/reinforcement_learning.py +55 -0
- shugocore-1.2.0/robotics_handler.py +417 -0
- shugocore-1.2.0/ros2_interface.py +416 -0
- shugocore-1.2.0/security.py +285 -0
- shugocore-1.2.0/setup.cfg +4 -0
- shugocore-1.2.0/shugocore.egg-info/PKG-INFO +285 -0
- shugocore-1.2.0/shugocore.egg-info/SOURCES.txt +48 -0
- shugocore-1.2.0/shugocore.egg-info/dependency_links.txt +1 -0
- shugocore-1.2.0/shugocore.egg-info/entry_points.txt +2 -0
- shugocore-1.2.0/shugocore.egg-info/requires.txt +13 -0
- shugocore-1.2.0/shugocore.egg-info/top_level.txt +28 -0
- shugocore-1.2.0/state_machine.py +215 -0
- shugocore-1.2.0/subconscious.py +171 -0
- shugocore-1.2.0/task_manager.py +80 -0
- shugocore-1.2.0/telemetry.py +159 -0
- shugocore-1.2.0/tests/test_acceleration.py +192 -0
- shugocore-1.2.0/tests/test_android.py +437 -0
- shugocore-1.2.0/tests/test_android_lifecycle_stress.py +415 -0
- shugocore-1.2.0/tests/test_mobile.py +301 -0
- shugocore-1.2.0/tests/test_model_execution_stress.py +245 -0
- shugocore-1.2.0/tests/test_platform_isolation.py +101 -0
- shugocore-1.2.0/tests/test_robotics.py +874 -0
- shugocore-1.2.0/tests/test_ros2_transport_stress.py +412 -0
- shugocore-1.2.0/tests/test_security.py +413 -0
- shugocore-1.2.0/tests/test_thermal_stress.py +249 -0
- shugocore-1.2.0/tests/test_v1.py +295 -0
- shugocore-1.2.0/token_budget.py +79 -0
- shugocore-1.2.0/vector_db.py +98 -0
- shugocore-1.2.0/version.py +10 -0
shugocore-1.2.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 SamurAI-Official
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
shugocore-1.2.0/PKG-INFO
ADDED
|
@@ -0,0 +1,285 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: shugocore
|
|
3
|
+
Version: 1.2.0
|
|
4
|
+
Summary: A continuous orchestration layer for synthetic functional agency: multi-model decisions, tool execution, four-tier memory, deterministic safety interlocks.
|
|
5
|
+
Author: SamurAI-Official
|
|
6
|
+
License: MIT
|
|
7
|
+
Keywords: agent,orchestration,memory,autonomy,safety
|
|
8
|
+
Requires-Python: >=3.9
|
|
9
|
+
Description-Content-Type: text/markdown
|
|
10
|
+
License-File: LICENSE
|
|
11
|
+
Requires-Dist: requests>=2.32.0
|
|
12
|
+
Provides-Extra: torch
|
|
13
|
+
Requires-Dist: torch; extra == "torch"
|
|
14
|
+
Provides-Extra: chromadb
|
|
15
|
+
Requires-Dist: chromadb; extra == "chromadb"
|
|
16
|
+
Provides-Extra: telemetry
|
|
17
|
+
Requires-Dist: opentelemetry-api>=1.20; extra == "telemetry"
|
|
18
|
+
Provides-Extra: termux
|
|
19
|
+
Requires-Dist: websocket-client>=1.6; extra == "termux"
|
|
20
|
+
Dynamic: license-file
|
|
21
|
+
|
|
22
|
+
# ShugoCore
|
|
23
|
+
|
|
24
|
+
> A continuous orchestration layer for synthetic functional agency.
|
|
25
|
+
|
|
26
|
+
ShugoCore coordinates models, tools, and a four-tier memory system so that an
|
|
27
|
+
artificial agent can *act* in an environment, register the consequences, and
|
|
28
|
+
adapt - indefinitely, without context degradation or unbounded state growth.
|
|
29
|
+
|
|
30
|
+
Where a chat model produces text, a functionally agentic system produces
|
|
31
|
+
*effects*: decisions that become tool calls and API interactions, outcomes
|
|
32
|
+
that become reward signals, and experience that consolidates into durable
|
|
33
|
+
knowledge. ShugoCore is the layer that makes that cycle safe, auditable, and
|
|
34
|
+
able to run continuously.
|
|
35
|
+
|
|
36
|
+
## Design principles
|
|
37
|
+
|
|
38
|
+
**Continuous.** Long-running agency fails when memory is naive: the context
|
|
39
|
+
window exhausts, or raw logs grow forever. ShugoCore's memory pipeline
|
|
40
|
+
consolidates, decays, and promotes continuously so the observe-act loop can
|
|
41
|
+
run for as long as the mission requires.
|
|
42
|
+
|
|
43
|
+
**Functional.** Decisions terminate in execution. Every action is gated by
|
|
44
|
+
deterministic world-model invariants *before* it touches the environment,
|
|
45
|
+
and every outcome is recorded as a structured episodic event that feeds
|
|
46
|
+
reinforcement learning.
|
|
47
|
+
|
|
48
|
+
**Accountable.** Agency without accountability is unsafe. Hard constraints
|
|
49
|
+
live in a read-only world model (Tier 3) that the agent's own execution
|
|
50
|
+
path cannot rewrite, and every decision and outcome is logged.
|
|
51
|
+
|
|
52
|
+
**Bounded.** Every subsystem is capacity- or decay-bounded: ring-buffered
|
|
53
|
+
episodes, salience-decayed facts, iteration-capped autonomous cycles, and a
|
|
54
|
+
decoupled maintenance worker that never blocks the primary loop.
|
|
55
|
+
|
|
56
|
+
## The orchestration loop
|
|
57
|
+
|
|
58
|
+
```
|
|
59
|
+
1. OBSERVE task arrives; reasoning tokens enter the Tier 0 scratchpad
|
|
60
|
+
2. GATE Tier 3 invariants check the action before anything runs
|
|
61
|
+
3. DECIDE models are selected and aggregated, enriched with Tier 2 context
|
|
62
|
+
4. EXECUTE the execution layer performs the tool / API interaction
|
|
63
|
+
5. EVALUATE reinforcement learning turns the outcome into a reward signal
|
|
64
|
+
6. RECORD the event lands in the Tier 1 episodic buffer
|
|
65
|
+
7. CONSOLIDATE a decoupled worker compresses episodes into Tier 2 facts,
|
|
66
|
+
decays stale salience and prunes forgotten knowledge
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
## System architecture
|
|
70
|
+
|
|
71
|
+
| Module | Responsibility |
|
|
72
|
+
|---|---|
|
|
73
|
+
| `decision_engine.py` | Orchestrates models, ethics, memory and execution; entry point |
|
|
74
|
+
| `model_manager.py` | Model registry, capability-based selection, performance tracking |
|
|
75
|
+
| `subconscious.py` | Model output generation (Ollama integration) and weight adaptation |
|
|
76
|
+
| `execution_layer.py` | Executes decisions against tools/APIs |
|
|
77
|
+
| `reinforcement_learning.py` | Reward signals and model weight updates from outcomes |
|
|
78
|
+
| `task_manager.py` | Queued task execution with callbacks |
|
|
79
|
+
| `autonomy.py` | Autonomous task generation / learning cycles |
|
|
80
|
+
| `vector_db.py` | Optional ChromaDB vector store (stub mode without it) |
|
|
81
|
+
| `logging_manager.py` | Structured logging of tasks and decisions |
|
|
82
|
+
| `memory_system.py` | Four-tier memory architecture (below) |
|
|
83
|
+
| `security.py` | Secrets, redaction, URL validation, rate limiting, circuit breaker |
|
|
84
|
+
| `policy.py` | Capability registry, approval broker, consent registry |
|
|
85
|
+
| `audit.py` | Tamper-evident hash-chained audit log |
|
|
86
|
+
| `model_backends.py` | Pluggable model adapters (Ollama HTTP, OpenAI-compatible, stub) |
|
|
87
|
+
|
|
88
|
+
## Memory architecture
|
|
89
|
+
|
|
90
|
+
```
|
|
91
|
+
[ Tier 0: Scratchpad / Working Memory ] <-- Unfiltered Token Stream (In-Context)
|
|
92
|
+
|
|
|
93
|
+
v (Consolidation Pipeline)
|
|
94
|
+
[ Tier 1: Episodic / Short-Term Memory ] <-- Event Logs, Recent Tool Execution (FIFO / Sliding)
|
|
95
|
+
|
|
|
96
|
+
v (Decay & Summarization Engine)
|
|
97
|
+
[ Tier 2: Semantic / Long-Term Memory ] <-- Entity Maps, Consolidated Facts (SQLite + Vectors)
|
|
98
|
+
|
|
|
99
|
+
v (Abstraction / Generalization)
|
|
100
|
+
[ Tier 3: Core Identity & World Model ] <-- System Invariants, Permanent Rules (Read-Only)
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
| Tier | Class | Purpose | Lifetime |
|
|
104
|
+
|---|---|---|---|
|
|
105
|
+
| 0 | `memory_system.Scratchpad` | Active context, step-by-step reasoning tokens, instantaneous sensory/API inputs | Milliseconds-minutes; flushed on task-step resolution |
|
|
106
|
+
| 1 | `memory_system.EpisodicMemory` | Exact sequence of recent actions, tool outputs, environmental responses | Hours-days; session-bounded JSON ring buffer |
|
|
107
|
+
| 2 | `memory_system.SemanticMemory` | Consolidated learnings, success/failure patterns, historical interactions | Semi-permanent; SQLite facts + embeddings |
|
|
108
|
+
| 3 | `memory_system.CoreIdentity` | Hard constraints, safety boundaries, fundamental environmental rules | Permanent; read-only during standard execution |
|
|
109
|
+
|
|
110
|
+
### Memory dynamics
|
|
111
|
+
|
|
112
|
+
- **Active consolidation (compression):** episodic events are drained and
|
|
113
|
+
summarized into compact semantic facts in Tier 2; raw logs are flushed.
|
|
114
|
+
- **Decay & pruning (forgetting):** Tier 2 salience decays exponentially
|
|
115
|
+
since last access; re-accessed memories are reinforced on retrieval and
|
|
116
|
+
low-salience memories are pruned.
|
|
117
|
+
- **Selective promotion:** critical failure modes and recurring patterns in
|
|
118
|
+
Tier 1 are promoted into Tier 2 as high-salience procedural insights.
|
|
119
|
+
- **Tier 2 -> Tier 3 review:** `MemoryManager.review_promotion_candidates()`
|
|
120
|
+
surfaces frequently re-accessed, high-salience facts; elevation into the
|
|
121
|
+
world model stays an explicit privileged step (`promote_to_core`).
|
|
122
|
+
|
|
123
|
+
### Isolation model
|
|
124
|
+
|
|
125
|
+
- Tier 0 / Tier 1 are created per `MemoryManager` (per-agent isolation -
|
|
126
|
+
no cross-task context contamination).
|
|
127
|
+
- Tier 2 / Tier 3 are shareable: pass the same `SemanticMemory` /
|
|
128
|
+
`CoreIdentity` instances into multiple `MemoryManager`s so planning
|
|
129
|
+
nodes see one consistent world model.
|
|
130
|
+
- Consolidation, decay and pruning run in a daemon worker thread and never
|
|
131
|
+
block the observation-action loop; use `consolidate_now()` for
|
|
132
|
+
deterministic, synchronous control.
|
|
133
|
+
|
|
134
|
+
## Safety model
|
|
135
|
+
|
|
136
|
+
Functional agency must be bounded by rules the agent cannot rewrite in the
|
|
137
|
+
course of acting. Enforcement is layered, so bypassing any single component
|
|
138
|
+
defeats nothing:
|
|
139
|
+
|
|
140
|
+
| Layer | Enforcement |
|
|
141
|
+
|---|---|
|
|
142
|
+
| Tier 3 world model | Immutable invariants (`no_harm`, `consent_required`, `no_manipulation`, `privacy`, `auditability`) evaluated before any model call or execution |
|
|
143
|
+
| `ConsentRegistry` | Side-effecting actions (`api_call`, `database_update`, `hardware_interaction`) require operator-issued grants - a `consent` flag written by the acting agent itself is never trusted |
|
|
144
|
+
| `ApprovalBroker` | Side effects additionally require human approval; fail-closed (no operator channel attached, or TTL expiry, means denied) |
|
|
145
|
+
| Policy verdict token | The engine binds an allow verdict to the canonical hash of the exact decision; the execution layer refuses missing, non-allow, or mismatched tokens |
|
|
146
|
+
| `CapabilityRegistry` | https-only egress, host allowlists, HTTP-method allowlists, SQL statement-type allowlists, empty-by-default hardware command allowlists |
|
|
147
|
+
| Egress controls | Mandatory timeouts, per-host rate limiting, circuit breakers, response size caps, redirects disabled |
|
|
148
|
+
| Hash-chained audit log | Every block, approval and execution is appended to a tamper-evident JSONL chain - verify with `python3 audit.py verify audit_chain.jsonl` |
|
|
149
|
+
| Secret hygiene | API keys resolved from environment variables at execution time, never carried in decision dicts; every log record passes a redaction filter |
|
|
150
|
+
| Honest execution | Unimplemented side-effecting actions return `not_implemented` - never simulated success - so the reinforcement signal cannot reward no-ops |
|
|
151
|
+
|
|
152
|
+
Key properties:
|
|
153
|
+
|
|
154
|
+
- **Single gated path.** Interactive tasks, autonomous cycles and the task
|
|
155
|
+
queue all execute through `DecisionEngine.execute_task` - the autonomous
|
|
156
|
+
loop cannot bypass the gate.
|
|
157
|
+
- **Read-only at runtime.** The world model changes only through the
|
|
158
|
+
privileged `promote_to_core()` path, which requires operator attribution
|
|
159
|
+
(`authorized_by=`) and appends to the Tier 3 ledger.
|
|
160
|
+
- **Fail-closed everywhere.** Missing verdict, missing consent, missing
|
|
161
|
+
approval channel, unknown host, unknown command - all refuse.
|
|
162
|
+
|
|
163
|
+
## Installation
|
|
164
|
+
|
|
165
|
+
Requires Python 3.9+.
|
|
166
|
+
|
|
167
|
+
```bash
|
|
168
|
+
git clone https://github.com/SamurAI-Official/ShugoCore.git
|
|
169
|
+
cd ShugoCore
|
|
170
|
+
pip install -r requirements.txt # core dependency: requests
|
|
171
|
+
python decision_engine.py # run the built-in demo
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
Optional extras:
|
|
175
|
+
|
|
176
|
+
- `torch` - enables CUDA/accelerated device selection (CPU-only mode without it)
|
|
177
|
+
- `chromadb` - enables persistent vector storage in `vector_db.py` (stub mode without it)
|
|
178
|
+
|
|
179
|
+
## Quickstart
|
|
180
|
+
|
|
181
|
+
```python
|
|
182
|
+
from decision_engine import DecisionEngine
|
|
183
|
+
|
|
184
|
+
models = [
|
|
185
|
+
{'id': 'gpt-4', 'type': 'text', 'weight': 0.5, 'backend': {'type': 'stub'}},
|
|
186
|
+
{'id': 'deepseek', 'type': 'text', 'weight': 0.3, 'backend': {'type': 'ollama'}},
|
|
187
|
+
{'id': 'llama', 'type': 'text', 'weight': 0.2, 'backend': {'type': 'ollama'}},
|
|
188
|
+
]
|
|
189
|
+
|
|
190
|
+
engine = DecisionEngine(
|
|
191
|
+
models=models,
|
|
192
|
+
vector_db_config={'type': 'chroma'}, # stub mode without chromadb
|
|
193
|
+
news_api_key=None, # or set SHUGOCORE_NEWS_API_KEY
|
|
194
|
+
memory_db_path='semantic_memory.db', # Tier 2 storage
|
|
195
|
+
audit_path='audit_chain.jsonl', # tamper-evident audit chain
|
|
196
|
+
)
|
|
197
|
+
|
|
198
|
+
# Tier 3 invariants gate every task before execution
|
|
199
|
+
result = engine.execute_task({'type': 'test', 'content': 'say hello'})
|
|
200
|
+
|
|
201
|
+
# Side-effecting actions need an operator consent grant AND an approval:
|
|
202
|
+
engine.consents.grant('api_call', granted_by='operator')
|
|
203
|
+
engine.approvals.attach_operator(lambda request: True) # operator channel
|
|
204
|
+
|
|
205
|
+
# Decisions carry long-term context retrieved from Tier 2
|
|
206
|
+
decision = engine.make_decision({'type': 'test', 'content': 'say hello'})
|
|
207
|
+
print(decision['memory_context'])
|
|
208
|
+
|
|
209
|
+
engine.shutdown() # flushes episodic memory into Tier 2, stops maintenance worker
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
### Autonomous operation
|
|
213
|
+
|
|
214
|
+
```python
|
|
215
|
+
# Generate, execute, learn, consolidate - with a hard iteration cap
|
|
216
|
+
tasks = [engine.autonomy.generate_task("test", "collect environment readings")]
|
|
217
|
+
engine.autonomy.autonomous_learning_cycle(tasks, max_iterations=10)
|
|
218
|
+
|
|
219
|
+
# Adapt to new environment data; observations persist in Tier 2
|
|
220
|
+
engine.autonomy.adapt_to_environment({"mode": "field", "temperature": 22})
|
|
221
|
+
|
|
222
|
+
# Review which Tier 2 facts proved durable enough to become permanent rules
|
|
223
|
+
candidates = engine.memory.review_promotion_candidates(min_salience=2.0,
|
|
224
|
+
min_access_count=3)
|
|
225
|
+
for fact in candidates:
|
|
226
|
+
print(fact["content"], fact["salience"], fact["access_count"])
|
|
227
|
+
# Promotion is an explicit, operator-attributed privileged step:
|
|
228
|
+
# engine.memory.promote_to_core("rule_key", "operator-approved rule",
|
|
229
|
+
# authorized_by="operator")
|
|
230
|
+
```
|
|
231
|
+
|
|
232
|
+
## Memory configuration
|
|
233
|
+
|
|
234
|
+
`MemoryManager` knobs (tuned when constructing `MemoryManager` directly;
|
|
235
|
+
`DecisionEngine` uses these defaults):
|
|
236
|
+
|
|
237
|
+
| Parameter | Default | Meaning |
|
|
238
|
+
|---|---|---|
|
|
239
|
+
| `consolidation_interval` | 10.0 s | Background worker tick |
|
|
240
|
+
| `consolidation_threshold` | 25 events | Episodic backlog that triggers consolidation |
|
|
241
|
+
| `failure_promotion_threshold` | 3 | Repeated failures promoted as procedural insights |
|
|
242
|
+
| `pattern_promotion_threshold` | 5 | Repeating events promoted as patterns |
|
|
243
|
+
| `decay_half_life_hours` | 72.0 | Salience half-life since last access |
|
|
244
|
+
| `prune_min_salience` | 0.05 | Deletion floor for decayed memories |
|
|
245
|
+
|
|
246
|
+
## Project structure
|
|
247
|
+
|
|
248
|
+
```
|
|
249
|
+
ShugoCore/
|
|
250
|
+
├── decision_engine.py # orchestration entry point; single gated path
|
|
251
|
+
├── autonomy.py # autonomous task generation and learning cycles
|
|
252
|
+
├── model_manager.py # model registry and performance tracking
|
|
253
|
+
├── subconscious.py # structured-decision prompts via backends
|
|
254
|
+
├── model_backends.py # Ollama HTTP / OpenAI-compatible / stub adapters
|
|
255
|
+
├── execution_layer.py # verdict-verified, allowlisted execution
|
|
256
|
+
├── policy.py # capability registry, approval broker, consent
|
|
257
|
+
├── security.py # secrets, redaction, rate limiting, breakers
|
|
258
|
+
├── audit.py # hash-chained audit log (+ verifier CLI)
|
|
259
|
+
├── reinforcement_learning.py # reward signals and weight updates
|
|
260
|
+
├── task_manager.py # bounded queued task execution
|
|
261
|
+
├── vector_db.py # optional ChromaDB integration
|
|
262
|
+
├── logging_manager.py # structured, redacted logging
|
|
263
|
+
├── memory_system.py # four-tier memory architecture
|
|
264
|
+
├── tests/ # security & integration regression tests
|
|
265
|
+
└── requirements.txt
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
Runtime artifacts (`semantic_memory.db`, logs) are local and gitignored.
|
|
269
|
+
|
|
270
|
+
## Roadmap
|
|
271
|
+
|
|
272
|
+
- Pluggable embedding backends for Tier 2 (current: dependency-free hashing vectors)
|
|
273
|
+
- PostgreSQL + pgvector storage option for shared multi-process deployments
|
|
274
|
+
- Entity/relation graphs alongside vector similarity in Tier 2
|
|
275
|
+
- Per-agent memory policies (isolation vs. sharing profiles)
|
|
276
|
+
- HMAC-signed audit chains and remote log shipping
|
|
277
|
+
- Human approval UI beyond the programmatic broker API
|
|
278
|
+
- Per-model backend pools with health-based routing
|
|
279
|
+
|
|
280
|
+
## Contributing
|
|
281
|
+
|
|
282
|
+
Issues and pull requests are welcome. Please keep changes consistent with
|
|
283
|
+
the architecture's invariants: Tier 0/1 stay per-agent, Tier 2/3 stay
|
|
284
|
+
shareable, and nothing in the standard execution path may mutate Tier 3.
|
|
285
|
+
|
|
@@ -0,0 +1,264 @@
|
|
|
1
|
+
# ShugoCore
|
|
2
|
+
|
|
3
|
+
> A continuous orchestration layer for synthetic functional agency.
|
|
4
|
+
|
|
5
|
+
ShugoCore coordinates models, tools, and a four-tier memory system so that an
|
|
6
|
+
artificial agent can *act* in an environment, register the consequences, and
|
|
7
|
+
adapt - indefinitely, without context degradation or unbounded state growth.
|
|
8
|
+
|
|
9
|
+
Where a chat model produces text, a functionally agentic system produces
|
|
10
|
+
*effects*: decisions that become tool calls and API interactions, outcomes
|
|
11
|
+
that become reward signals, and experience that consolidates into durable
|
|
12
|
+
knowledge. ShugoCore is the layer that makes that cycle safe, auditable, and
|
|
13
|
+
able to run continuously.
|
|
14
|
+
|
|
15
|
+
## Design principles
|
|
16
|
+
|
|
17
|
+
**Continuous.** Long-running agency fails when memory is naive: the context
|
|
18
|
+
window exhausts, or raw logs grow forever. ShugoCore's memory pipeline
|
|
19
|
+
consolidates, decays, and promotes continuously so the observe-act loop can
|
|
20
|
+
run for as long as the mission requires.
|
|
21
|
+
|
|
22
|
+
**Functional.** Decisions terminate in execution. Every action is gated by
|
|
23
|
+
deterministic world-model invariants *before* it touches the environment,
|
|
24
|
+
and every outcome is recorded as a structured episodic event that feeds
|
|
25
|
+
reinforcement learning.
|
|
26
|
+
|
|
27
|
+
**Accountable.** Agency without accountability is unsafe. Hard constraints
|
|
28
|
+
live in a read-only world model (Tier 3) that the agent's own execution
|
|
29
|
+
path cannot rewrite, and every decision and outcome is logged.
|
|
30
|
+
|
|
31
|
+
**Bounded.** Every subsystem is capacity- or decay-bounded: ring-buffered
|
|
32
|
+
episodes, salience-decayed facts, iteration-capped autonomous cycles, and a
|
|
33
|
+
decoupled maintenance worker that never blocks the primary loop.
|
|
34
|
+
|
|
35
|
+
## The orchestration loop
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
1. OBSERVE task arrives; reasoning tokens enter the Tier 0 scratchpad
|
|
39
|
+
2. GATE Tier 3 invariants check the action before anything runs
|
|
40
|
+
3. DECIDE models are selected and aggregated, enriched with Tier 2 context
|
|
41
|
+
4. EXECUTE the execution layer performs the tool / API interaction
|
|
42
|
+
5. EVALUATE reinforcement learning turns the outcome into a reward signal
|
|
43
|
+
6. RECORD the event lands in the Tier 1 episodic buffer
|
|
44
|
+
7. CONSOLIDATE a decoupled worker compresses episodes into Tier 2 facts,
|
|
45
|
+
decays stale salience and prunes forgotten knowledge
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
## System architecture
|
|
49
|
+
|
|
50
|
+
| Module | Responsibility |
|
|
51
|
+
|---|---|
|
|
52
|
+
| `decision_engine.py` | Orchestrates models, ethics, memory and execution; entry point |
|
|
53
|
+
| `model_manager.py` | Model registry, capability-based selection, performance tracking |
|
|
54
|
+
| `subconscious.py` | Model output generation (Ollama integration) and weight adaptation |
|
|
55
|
+
| `execution_layer.py` | Executes decisions against tools/APIs |
|
|
56
|
+
| `reinforcement_learning.py` | Reward signals and model weight updates from outcomes |
|
|
57
|
+
| `task_manager.py` | Queued task execution with callbacks |
|
|
58
|
+
| `autonomy.py` | Autonomous task generation / learning cycles |
|
|
59
|
+
| `vector_db.py` | Optional ChromaDB vector store (stub mode without it) |
|
|
60
|
+
| `logging_manager.py` | Structured logging of tasks and decisions |
|
|
61
|
+
| `memory_system.py` | Four-tier memory architecture (below) |
|
|
62
|
+
| `security.py` | Secrets, redaction, URL validation, rate limiting, circuit breaker |
|
|
63
|
+
| `policy.py` | Capability registry, approval broker, consent registry |
|
|
64
|
+
| `audit.py` | Tamper-evident hash-chained audit log |
|
|
65
|
+
| `model_backends.py` | Pluggable model adapters (Ollama HTTP, OpenAI-compatible, stub) |
|
|
66
|
+
|
|
67
|
+
## Memory architecture
|
|
68
|
+
|
|
69
|
+
```
|
|
70
|
+
[ Tier 0: Scratchpad / Working Memory ] <-- Unfiltered Token Stream (In-Context)
|
|
71
|
+
|
|
|
72
|
+
v (Consolidation Pipeline)
|
|
73
|
+
[ Tier 1: Episodic / Short-Term Memory ] <-- Event Logs, Recent Tool Execution (FIFO / Sliding)
|
|
74
|
+
|
|
|
75
|
+
v (Decay & Summarization Engine)
|
|
76
|
+
[ Tier 2: Semantic / Long-Term Memory ] <-- Entity Maps, Consolidated Facts (SQLite + Vectors)
|
|
77
|
+
|
|
|
78
|
+
v (Abstraction / Generalization)
|
|
79
|
+
[ Tier 3: Core Identity & World Model ] <-- System Invariants, Permanent Rules (Read-Only)
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
| Tier | Class | Purpose | Lifetime |
|
|
83
|
+
|---|---|---|---|
|
|
84
|
+
| 0 | `memory_system.Scratchpad` | Active context, step-by-step reasoning tokens, instantaneous sensory/API inputs | Milliseconds-minutes; flushed on task-step resolution |
|
|
85
|
+
| 1 | `memory_system.EpisodicMemory` | Exact sequence of recent actions, tool outputs, environmental responses | Hours-days; session-bounded JSON ring buffer |
|
|
86
|
+
| 2 | `memory_system.SemanticMemory` | Consolidated learnings, success/failure patterns, historical interactions | Semi-permanent; SQLite facts + embeddings |
|
|
87
|
+
| 3 | `memory_system.CoreIdentity` | Hard constraints, safety boundaries, fundamental environmental rules | Permanent; read-only during standard execution |
|
|
88
|
+
|
|
89
|
+
### Memory dynamics
|
|
90
|
+
|
|
91
|
+
- **Active consolidation (compression):** episodic events are drained and
|
|
92
|
+
summarized into compact semantic facts in Tier 2; raw logs are flushed.
|
|
93
|
+
- **Decay & pruning (forgetting):** Tier 2 salience decays exponentially
|
|
94
|
+
since last access; re-accessed memories are reinforced on retrieval and
|
|
95
|
+
low-salience memories are pruned.
|
|
96
|
+
- **Selective promotion:** critical failure modes and recurring patterns in
|
|
97
|
+
Tier 1 are promoted into Tier 2 as high-salience procedural insights.
|
|
98
|
+
- **Tier 2 -> Tier 3 review:** `MemoryManager.review_promotion_candidates()`
|
|
99
|
+
surfaces frequently re-accessed, high-salience facts; elevation into the
|
|
100
|
+
world model stays an explicit privileged step (`promote_to_core`).
|
|
101
|
+
|
|
102
|
+
### Isolation model
|
|
103
|
+
|
|
104
|
+
- Tier 0 / Tier 1 are created per `MemoryManager` (per-agent isolation -
|
|
105
|
+
no cross-task context contamination).
|
|
106
|
+
- Tier 2 / Tier 3 are shareable: pass the same `SemanticMemory` /
|
|
107
|
+
`CoreIdentity` instances into multiple `MemoryManager`s so planning
|
|
108
|
+
nodes see one consistent world model.
|
|
109
|
+
- Consolidation, decay and pruning run in a daemon worker thread and never
|
|
110
|
+
block the observation-action loop; use `consolidate_now()` for
|
|
111
|
+
deterministic, synchronous control.
|
|
112
|
+
|
|
113
|
+
## Safety model
|
|
114
|
+
|
|
115
|
+
Functional agency must be bounded by rules the agent cannot rewrite in the
|
|
116
|
+
course of acting. Enforcement is layered, so bypassing any single component
|
|
117
|
+
defeats nothing:
|
|
118
|
+
|
|
119
|
+
| Layer | Enforcement |
|
|
120
|
+
|---|---|
|
|
121
|
+
| Tier 3 world model | Immutable invariants (`no_harm`, `consent_required`, `no_manipulation`, `privacy`, `auditability`) evaluated before any model call or execution |
|
|
122
|
+
| `ConsentRegistry` | Side-effecting actions (`api_call`, `database_update`, `hardware_interaction`) require operator-issued grants - a `consent` flag written by the acting agent itself is never trusted |
|
|
123
|
+
| `ApprovalBroker` | Side effects additionally require human approval; fail-closed (no operator channel attached, or TTL expiry, means denied) |
|
|
124
|
+
| Policy verdict token | The engine binds an allow verdict to the canonical hash of the exact decision; the execution layer refuses missing, non-allow, or mismatched tokens |
|
|
125
|
+
| `CapabilityRegistry` | https-only egress, host allowlists, HTTP-method allowlists, SQL statement-type allowlists, empty-by-default hardware command allowlists |
|
|
126
|
+
| Egress controls | Mandatory timeouts, per-host rate limiting, circuit breakers, response size caps, redirects disabled |
|
|
127
|
+
| Hash-chained audit log | Every block, approval and execution is appended to a tamper-evident JSONL chain - verify with `python3 audit.py verify audit_chain.jsonl` |
|
|
128
|
+
| Secret hygiene | API keys resolved from environment variables at execution time, never carried in decision dicts; every log record passes a redaction filter |
|
|
129
|
+
| Honest execution | Unimplemented side-effecting actions return `not_implemented` - never simulated success - so the reinforcement signal cannot reward no-ops |
|
|
130
|
+
|
|
131
|
+
Key properties:
|
|
132
|
+
|
|
133
|
+
- **Single gated path.** Interactive tasks, autonomous cycles and the task
|
|
134
|
+
queue all execute through `DecisionEngine.execute_task` - the autonomous
|
|
135
|
+
loop cannot bypass the gate.
|
|
136
|
+
- **Read-only at runtime.** The world model changes only through the
|
|
137
|
+
privileged `promote_to_core()` path, which requires operator attribution
|
|
138
|
+
(`authorized_by=`) and appends to the Tier 3 ledger.
|
|
139
|
+
- **Fail-closed everywhere.** Missing verdict, missing consent, missing
|
|
140
|
+
approval channel, unknown host, unknown command - all refuse.
|
|
141
|
+
|
|
142
|
+
## Installation
|
|
143
|
+
|
|
144
|
+
Requires Python 3.9+.
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
git clone https://github.com/SamurAI-Official/ShugoCore.git
|
|
148
|
+
cd ShugoCore
|
|
149
|
+
pip install -r requirements.txt # core dependency: requests
|
|
150
|
+
python decision_engine.py # run the built-in demo
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
Optional extras:
|
|
154
|
+
|
|
155
|
+
- `torch` - enables CUDA/accelerated device selection (CPU-only mode without it)
|
|
156
|
+
- `chromadb` - enables persistent vector storage in `vector_db.py` (stub mode without it)
|
|
157
|
+
|
|
158
|
+
## Quickstart
|
|
159
|
+
|
|
160
|
+
```python
|
|
161
|
+
from decision_engine import DecisionEngine
|
|
162
|
+
|
|
163
|
+
models = [
|
|
164
|
+
{'id': 'gpt-4', 'type': 'text', 'weight': 0.5, 'backend': {'type': 'stub'}},
|
|
165
|
+
{'id': 'deepseek', 'type': 'text', 'weight': 0.3, 'backend': {'type': 'ollama'}},
|
|
166
|
+
{'id': 'llama', 'type': 'text', 'weight': 0.2, 'backend': {'type': 'ollama'}},
|
|
167
|
+
]
|
|
168
|
+
|
|
169
|
+
engine = DecisionEngine(
|
|
170
|
+
models=models,
|
|
171
|
+
vector_db_config={'type': 'chroma'}, # stub mode without chromadb
|
|
172
|
+
news_api_key=None, # or set SHUGOCORE_NEWS_API_KEY
|
|
173
|
+
memory_db_path='semantic_memory.db', # Tier 2 storage
|
|
174
|
+
audit_path='audit_chain.jsonl', # tamper-evident audit chain
|
|
175
|
+
)
|
|
176
|
+
|
|
177
|
+
# Tier 3 invariants gate every task before execution
|
|
178
|
+
result = engine.execute_task({'type': 'test', 'content': 'say hello'})
|
|
179
|
+
|
|
180
|
+
# Side-effecting actions need an operator consent grant AND an approval:
|
|
181
|
+
engine.consents.grant('api_call', granted_by='operator')
|
|
182
|
+
engine.approvals.attach_operator(lambda request: True) # operator channel
|
|
183
|
+
|
|
184
|
+
# Decisions carry long-term context retrieved from Tier 2
|
|
185
|
+
decision = engine.make_decision({'type': 'test', 'content': 'say hello'})
|
|
186
|
+
print(decision['memory_context'])
|
|
187
|
+
|
|
188
|
+
engine.shutdown() # flushes episodic memory into Tier 2, stops maintenance worker
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
### Autonomous operation
|
|
192
|
+
|
|
193
|
+
```python
|
|
194
|
+
# Generate, execute, learn, consolidate - with a hard iteration cap
|
|
195
|
+
tasks = [engine.autonomy.generate_task("test", "collect environment readings")]
|
|
196
|
+
engine.autonomy.autonomous_learning_cycle(tasks, max_iterations=10)
|
|
197
|
+
|
|
198
|
+
# Adapt to new environment data; observations persist in Tier 2
|
|
199
|
+
engine.autonomy.adapt_to_environment({"mode": "field", "temperature": 22})
|
|
200
|
+
|
|
201
|
+
# Review which Tier 2 facts proved durable enough to become permanent rules
|
|
202
|
+
candidates = engine.memory.review_promotion_candidates(min_salience=2.0,
|
|
203
|
+
min_access_count=3)
|
|
204
|
+
for fact in candidates:
|
|
205
|
+
print(fact["content"], fact["salience"], fact["access_count"])
|
|
206
|
+
# Promotion is an explicit, operator-attributed privileged step:
|
|
207
|
+
# engine.memory.promote_to_core("rule_key", "operator-approved rule",
|
|
208
|
+
# authorized_by="operator")
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
## Memory configuration
|
|
212
|
+
|
|
213
|
+
`MemoryManager` knobs (tuned when constructing `MemoryManager` directly;
|
|
214
|
+
`DecisionEngine` uses these defaults):
|
|
215
|
+
|
|
216
|
+
| Parameter | Default | Meaning |
|
|
217
|
+
|---|---|---|
|
|
218
|
+
| `consolidation_interval` | 10.0 s | Background worker tick |
|
|
219
|
+
| `consolidation_threshold` | 25 events | Episodic backlog that triggers consolidation |
|
|
220
|
+
| `failure_promotion_threshold` | 3 | Repeated failures promoted as procedural insights |
|
|
221
|
+
| `pattern_promotion_threshold` | 5 | Repeating events promoted as patterns |
|
|
222
|
+
| `decay_half_life_hours` | 72.0 | Salience half-life since last access |
|
|
223
|
+
| `prune_min_salience` | 0.05 | Deletion floor for decayed memories |
|
|
224
|
+
|
|
225
|
+
## Project structure
|
|
226
|
+
|
|
227
|
+
```
|
|
228
|
+
ShugoCore/
|
|
229
|
+
├── decision_engine.py # orchestration entry point; single gated path
|
|
230
|
+
├── autonomy.py # autonomous task generation and learning cycles
|
|
231
|
+
├── model_manager.py # model registry and performance tracking
|
|
232
|
+
├── subconscious.py # structured-decision prompts via backends
|
|
233
|
+
├── model_backends.py # Ollama HTTP / OpenAI-compatible / stub adapters
|
|
234
|
+
├── execution_layer.py # verdict-verified, allowlisted execution
|
|
235
|
+
├── policy.py # capability registry, approval broker, consent
|
|
236
|
+
├── security.py # secrets, redaction, rate limiting, breakers
|
|
237
|
+
├── audit.py # hash-chained audit log (+ verifier CLI)
|
|
238
|
+
├── reinforcement_learning.py # reward signals and weight updates
|
|
239
|
+
├── task_manager.py # bounded queued task execution
|
|
240
|
+
├── vector_db.py # optional ChromaDB integration
|
|
241
|
+
├── logging_manager.py # structured, redacted logging
|
|
242
|
+
├── memory_system.py # four-tier memory architecture
|
|
243
|
+
├── tests/ # security & integration regression tests
|
|
244
|
+
└── requirements.txt
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
Runtime artifacts (`semantic_memory.db`, logs) are local and gitignored.
|
|
248
|
+
|
|
249
|
+
## Roadmap
|
|
250
|
+
|
|
251
|
+
- Pluggable embedding backends for Tier 2 (current: dependency-free hashing vectors)
|
|
252
|
+
- PostgreSQL + pgvector storage option for shared multi-process deployments
|
|
253
|
+
- Entity/relation graphs alongside vector similarity in Tier 2
|
|
254
|
+
- Per-agent memory policies (isolation vs. sharing profiles)
|
|
255
|
+
- HMAC-signed audit chains and remote log shipping
|
|
256
|
+
- Human approval UI beyond the programmatic broker API
|
|
257
|
+
- Per-model backend pools with health-based routing
|
|
258
|
+
|
|
259
|
+
## Contributing
|
|
260
|
+
|
|
261
|
+
Issues and pull requests are welcome. Please keep changes consistent with
|
|
262
|
+
the architecture's invariants: Tier 0/1 stay per-agent, Tier 2/3 stay
|
|
263
|
+
shareable, and nothing in the standard execution path may mutate Tier 3.
|
|
264
|
+
|