@aws/agentcore 0.3.0-preview.6.0 → 0.3.0-preview.7.0

package/README.md

@@ -17,17 +17,21 @@ AgentCore with minimal configuration.
 
 ## Installation
 
-```bash
-npm install -g @aws/agentcore
-```
-
-> **Public Preview**: If you previously used the
-> [Bedrock AgentCore Starter Toolkit](https://github.com/aws/bedrock-agentcore-starter-toolkit), uninstall it before
-> using this CLI:
+> **Upgrading from the Bedrock AgentCore Starter Toolkit?** The old Python CLI conflicts with this package. If it is
+> still installed, `npm install` will fail with an error telling you which package manager has it. Uninstall it first
+> using whichever tool you originally used:
 >
 > ```bash
-> pip uninstall bedrock-agentcore-starter-toolkit
+> pip uninstall bedrock-agentcore-starter-toolkit    # if installed via pip
+> pipx uninstall bedrock-agentcore-starter-toolkit   # if installed via pipx
+> uv tool uninstall bedrock-agentcore-starter-toolkit # if installed via uv
 > ```
+>
+> If you need to bypass the check (for example, in CI), set `AGENTCORE_SKIP_CONFLICT_CHECK=1` before installing.
+
+```bash
+npm install -g @aws/agentcore
+```
 
 ## Quick Start
 
@@ -83,13 +87,24 @@ agentcore invoke
 
 ### Resource Management
 
-| Command  | Description                           |
-| -------- | ------------------------------------- |
-| `add`    | Add agents, memory, identity, targets |
-| `remove` | Remove resources from project         |
+| Command  | Description                                       |
+| -------- | ------------------------------------------------- |
+| `add`    | Add agents, memory, identity, evaluators, targets |
+| `remove` | Remove resources from project                     |
 
 > **Note**: Run `agentcore deploy` after `add` or `remove` to update resources in AWS.
 
+### Evaluations
+
+| Command              | Description                                   |
+| -------------------- | --------------------------------------------- |
+| `add evaluator`      | Add a custom LLM-as-a-Judge evaluator         |
+| `add online-eval`    | Add continuous evaluation for live traffic    |
+| `run evals`          | Run on-demand evaluation against agent traces |
+| `evals history`      | View past eval run results                    |
+| `pause online-eval`  | Pause a deployed online eval config           |
+| `resume online-eval` | Resume a paused online eval config            |
+
 ## Project Structure
 
 ```
@@ -116,7 +131,7 @@ my-project/
 
 Projects use JSON schema files in the `agentcore/` directory:
 
-- `agentcore.json` - Agent specifications, memory, identity, remote tools
+- `agentcore.json` - Agent specifications, memory, identity, evaluators, online evals
 - `deployed-state.json` - Runtime state in agentcore/.cli/ (auto-managed)
 - `aws-targets.json` - Deployment targets (account, region)
 
@@ -125,11 +140,13 @@ Projects use JSON schema files in the `agentcore/` directory:
 - **Runtime** - Managed execution environment for deployed agents
 - **Memory** - Semantic, summarization, and user preference strategies
 - **Identity** - Secure API key management via Secrets Manager
+- **Evaluations** - LLM-as-a-Judge for on-demand and continuous agent quality monitoring
 
 ## Documentation
 
 - [CLI Commands Reference](docs/commands.md) - Full command reference for scripting and CI/CD
 - [Configuration](docs/configuration.md) - Schema reference for config files
+- [Evaluations](docs/evals.md) - Evaluators, on-demand evals, and online monitoring
 - [Frameworks](docs/frameworks.md) - Supported frameworks and model providers
 - [Gateway](docs/gateway.md) - Gateway setup, targets, and authentication
 - [Memory](docs/memory.md) - Memory strategies and sharing

package/dist/assets/__tests__/__snapshots__/assets.snapshot.test.ts.snap

@@ -374,6 +374,7 @@ test('AgentCoreStack synthesizes with empty spec', () => {
       credentials: [],
       evaluators: [],
       onlineEvalConfigs: [],
+      policyEngines: [],
     },
   });
   const template = Template.fromStack(stack);

package/dist/assets/cdk/test/cdk.test.ts

@@ -13,6 +13,7 @@ test('AgentCoreStack synthesizes with empty spec', () => {
       credentials: [],
       evaluators: [],
       onlineEvalConfigs: [],
+      policyEngines: [],
     },
   });
   const template = Template.fromStack(stack);