@aws/agentcore 0.3.0-preview.6.0 → 0.3.0-preview.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +18 -5
- package/dist/cli/index.mjs +247 -247
- package/package.json +5 -5
- package/scripts/start-tui-harness.sh +90 -0
package/README.md
CHANGED
|
@@ -83,13 +83,24 @@ agentcore invoke
|
|
|
83
83
|
|
|
84
84
|
### Resource Management
|
|
85
85
|
|
|
86
|
-
| Command | Description
|
|
87
|
-
| -------- |
|
|
88
|
-
| `add` | Add agents, memory, identity, targets |
|
|
89
|
-
| `remove` | Remove resources from project
|
|
86
|
+
| Command | Description |
|
|
87
|
+
| -------- | ------------------------------------------------- |
|
|
88
|
+
| `add` | Add agents, memory, identity, evaluators, targets |
|
|
89
|
+
| `remove` | Remove resources from project |
|
|
90
90
|
|
|
91
91
|
> **Note**: Run `agentcore deploy` after `add` or `remove` to update resources in AWS.
|
|
92
92
|
|
|
93
|
+
### Evaluations
|
|
94
|
+
|
|
95
|
+
| Command | Description |
|
|
96
|
+
| -------------------- | --------------------------------------------- |
|
|
97
|
+
| `add evaluator` | Add a custom LLM-as-a-Judge evaluator |
|
|
98
|
+
| `add online-eval` | Add continuous evaluation for live traffic |
|
|
99
|
+
| `run evals` | Run on-demand evaluation against agent traces |
|
|
100
|
+
| `evals history` | View past eval run results |
|
|
101
|
+
| `pause online-eval` | Pause a deployed online eval config |
|
|
102
|
+
| `resume online-eval` | Resume a paused online eval config |
|
|
103
|
+
|
|
93
104
|
## Project Structure
|
|
94
105
|
|
|
95
106
|
```
|
|
@@ -116,7 +127,7 @@ my-project/
|
|
|
116
127
|
|
|
117
128
|
Projects use JSON schema files in the `agentcore/` directory:
|
|
118
129
|
|
|
119
|
-
- `agentcore.json` - Agent specifications, memory, identity,
|
|
130
|
+
- `agentcore.json` - Agent specifications, memory, identity, evaluators, online evals
|
|
120
131
|
- `deployed-state.json` - Runtime state in agentcore/.cli/ (auto-managed)
|
|
121
132
|
- `aws-targets.json` - Deployment targets (account, region)
|
|
122
133
|
|
|
@@ -125,11 +136,13 @@ Projects use JSON schema files in the `agentcore/` directory:
|
|
|
125
136
|
- **Runtime** - Managed execution environment for deployed agents
|
|
126
137
|
- **Memory** - Semantic, summarization, and user preference strategies
|
|
127
138
|
- **Identity** - Secure API key management via Secrets Manager
|
|
139
|
+
- **Evaluations** - LLM-as-a-Judge for on-demand and continuous agent quality monitoring
|
|
128
140
|
|
|
129
141
|
## Documentation
|
|
130
142
|
|
|
131
143
|
- [CLI Commands Reference](docs/commands.md) - Full command reference for scripting and CI/CD
|
|
132
144
|
- [Configuration](docs/configuration.md) - Schema reference for config files
|
|
145
|
+
- [Evaluations](docs/evals.md) - Evaluators, on-demand evals, and online monitoring
|
|
133
146
|
- [Frameworks](docs/frameworks.md) - Supported frameworks and model providers
|
|
134
147
|
- [Gateway](docs/gateway.md) - Gateway setup, targets, and authentication
|
|
135
148
|
- [Memory](docs/memory.md) - Memory strategies and sharing
|