@lmctl-ai/lmctl 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +36 -0
- package/README.md +36 -0
- package/bin/lmctl +4 -0
- package/dist/cli/index.js +2180 -0
- package/dist/cli/schema.sql +660 -0
- package/dist/cli/side_effect_classifier.json +19 -0
- package/dist/config/ai_test_templates/example-test.md.template +50 -0
- package/dist/config/ai_test_templates/index.md.template +27 -0
- package/dist/config/durable_memory_templates/index.md.template +29 -0
- package/dist/config/durable_memory_templates/skills_general.md.template +42 -0
- package/dist/config/durable_memory_templates/skills_lmdebug.md.template +31 -0
- package/dist/config/durable_memory_templates/skills_lmprobe.md.template +31 -0
- package/package.json +42 -0
- package/workflows/bugfix-extended-v2.compound.json +1018 -0
- package/workflows/bugfix-v2.compound.json +856 -0
- package/workflows/claim-check-spike-v2.compound.json +136 -0
- package/workflows/document-creation.compound.json +200 -0
- package/workflows/durable-memory-consolidation-v2.compound.json +185 -0
- package/workflows/example-v2.compound.json +200 -0
- package/workflows/image-qa.compound.json +128 -0
- package/workflows/index.jsonl +15 -0
- package/workflows/info-qa.compound.json +120 -0
- package/workflows/newspaper.compound.json +129 -0
- package/workflows/pr-fix.compound.json +183 -0
- package/workflows/pr-followup-v2.compound.json +969 -0
- package/workflows/provider-probe.compound.json +107 -0
- package/workflows/qa-suite.compound.json +186 -0
- package/workflows/spec-driven-task.compound.json +460 -0
- package/workflows/triage-v2.compound.json +721 -0
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: example-test
|
|
3
|
+
description: Replace with a one-line description of what this test verifies
|
|
4
|
+
type: smoke
|
|
5
|
+
tags: [example]
|
|
6
|
+
last_run_at: never
|
|
7
|
+
last_run_status: unknown
|
|
8
|
+
last_run_id: 0
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# Test: example
|
|
12
|
+
|
|
13
|
+
Replace this content with your real test case. The structure below is what the QA Tester agent expects.
|
|
14
|
+
|
|
15
|
+
## Setup
|
|
16
|
+
|
|
17
|
+
Describe what the system state must be before running this test. Examples:
|
|
18
|
+
- A project exists with a configured team
|
|
19
|
+
- An image fixture is present at a known path
|
|
20
|
+
- The serve daemon is running on $LMCTL_API_URL
|
|
21
|
+
|
|
22
|
+
Use shell-style commands (the Tester will execute them) or describe in prose if manual steps are needed.
|
|
23
|
+
|
|
24
|
+
## Action
|
|
25
|
+
|
|
26
|
+
What the test actually does. Be specific about commands, inputs, and expected intermediate observations.
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
# Example action
|
|
30
|
+
curl -X GET $LMCTL_API_URL/api/status \
|
|
31
|
+
-H "Authorization: Bearer $LMCTL_API_TOKEN"
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
## Expected
|
|
35
|
+
|
|
36
|
+
What "pass" looks like:
|
|
37
|
+
- HTTP 200 response
|
|
38
|
+
- JSON body contains key `jobs` with `total >= 0`
|
|
39
|
+
- Response time under 1 second
|
|
40
|
+
|
|
41
|
+
The Interpreter agent reads this section + the Action's actual observations to emit STANCE: pass / fail / inconclusive.
|
|
42
|
+
|
|
43
|
+
## Cleanup
|
|
44
|
+
|
|
45
|
+
Idempotent cleanup steps. Should be safe to run even if Setup or Action failed.
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
# Example cleanup
|
|
49
|
+
# (nothing to clean up for this read-only example)
|
|
50
|
+
```
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# AI Test Suite - index
|
|
2
|
+
|
|
3
|
+
This directory holds **AI-driven manual tests** for this project. Each chapter (`*.md`) describes one test case that an LLM agent (the QA Tester) executes; an Interpreter agent judges pass/fail.
|
|
4
|
+
|
|
5
|
+
**This is NOT for unit/integration tests** - those live in `tests/` and run via your test framework (e.g., `npm test`). `ai-test/` is for things that need LLM-quality judgment: vision quality, prompt-driven behavior, end-to-end scenarios that exercise the full stack.
|
|
6
|
+
|
|
7
|
+
## Conventions
|
|
8
|
+
|
|
9
|
+
- One test case per `.md` file
|
|
10
|
+
- YAML frontmatter (agent-writable: `last_run_*` fields)
|
|
11
|
+
- Markdown body sections: `## Setup`, `## Action`, `## Expected`, `## Cleanup`
|
|
12
|
+
- Issues found by qa-suite reference the chapter via `ai_test_path` (e.g. `ai-test/test-image-upload-basic.md`)
|
|
13
|
+
|
|
14
|
+
## Test cases
|
|
15
|
+
|
|
16
|
+
See individual chapters in this directory. Use `grep -l 'type: smoke' ai-test/*.md` (etc.) to filter.
|
|
17
|
+
|
|
18
|
+
## How to add a new test case
|
|
19
|
+
|
|
20
|
+
1. Copy a chapter as a template: `cp ai-test/example-test.md ai-test/test-my-new-case.md`
|
|
21
|
+
2. Edit frontmatter (name, description, type, tags)
|
|
22
|
+
3. Edit body (Setup, Action, Expected, Cleanup)
|
|
23
|
+
4. Commit alongside the code change that motivated the test
|
|
24
|
+
|
|
25
|
+
## Pointer to lmctl's own ai-test/ (canonical example)
|
|
26
|
+
|
|
27
|
+
See `<lmctl-repo>/ai-test/` for the test corpus used to verify lmctl itself.
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: index
|
|
3
|
+
description: Index of all durable-memory chapters for this project
|
|
4
|
+
type: project
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Durable-memory index
|
|
8
|
+
|
|
9
|
+
This directory contains durable knowledge accumulated by agents working on this project.
|
|
10
|
+
|
|
11
|
+
## How to use
|
|
12
|
+
|
|
13
|
+
ON EVERY NEW TASK:
|
|
14
|
+
1. Read this file first. Scan the chapter list below.
|
|
15
|
+
2. Pull only the chapters relevant to your task. Don't load everything at once.
|
|
16
|
+
3. If you learn something worth preserving (a convention, a decision, a gotcha), write it back:
|
|
17
|
+
create or update `durable-memory/<descriptive-name>.md` with YAML frontmatter
|
|
18
|
+
(name / description / type) and a body. The engine syncs the write automatically.
|
|
19
|
+
|
|
20
|
+
## Canonical example
|
|
21
|
+
|
|
22
|
+
If you're new to durable-memory, lmctl's own `durable-memory/` directory is the canonical reference.
|
|
23
|
+
Read its `index.md` + `architecture-foundations.md` + `agent-onboarding-pattern.md` to see the pattern in production use.
|
|
24
|
+
Path: `<lmctl-repo>/durable-memory/`
|
|
25
|
+
|
|
26
|
+
## Chapters
|
|
27
|
+
|
|
28
|
+
<!-- Agents: add a one-line entry here when you create a new chapter. -->
|
|
29
|
+
<!-- Format: - **<name>** (`<filename>.md`) — <one-sentence description> -->
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Skills — general agent discipline
|
|
3
|
+
description: Core agent habits for durable-memory, STANCE format, and bypass-test discipline
|
|
4
|
+
type: feedback
|
|
5
|
+
tags: [skills, discipline]
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# General agent discipline
|
|
9
|
+
|
|
10
|
+
## STANCE format
|
|
11
|
+
|
|
12
|
+
Always end a response or turn with a STANCE line so the engine can parse your outcome:
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
STANCE: <outcome_key>
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
Common outcomes: `completed`, `needs_more_clarity_from_lead`, `blocked_with_question`, `no_writeback`.
|
|
19
|
+
|
|
20
|
+
Use `STANCE: no_writeback` after a write-back turn only when there is genuinely nothing new to record.
|
|
21
|
+
|
|
22
|
+
## Durable-memory read-before-write
|
|
23
|
+
|
|
24
|
+
- Read `durable-memory/index.md` at the start of every new task.
|
|
25
|
+
- Pull only the chapters relevant to your task; don't load the full directory.
|
|
26
|
+
- Before creating a new chapter, check whether an existing one covers the topic.
|
|
27
|
+
|
|
28
|
+
## Write-back discipline
|
|
29
|
+
|
|
30
|
+
Write back when you learn something durable:
|
|
31
|
+
- A project convention that future agents must follow
|
|
32
|
+
- An architectural decision and its rationale
|
|
33
|
+
- A non-obvious constraint, integration gotcha, or environment quirk
|
|
34
|
+
|
|
35
|
+
Do NOT write back:
|
|
36
|
+
- Information derivable from reading the current code or git history
|
|
37
|
+
- Ephemeral task state or in-progress notes
|
|
38
|
+
- Anything already documented in CLAUDE.md or equivalent config files
|
|
39
|
+
|
|
40
|
+
## Bypass-test discipline
|
|
41
|
+
|
|
42
|
+
Tests that verify a behaviour must be "load-bearing": removing the code under test must make the test fail. If a test would still pass after the fix is removed, it is not a bypass test — rewrite it so it is.
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Skills — lmdebug
|
|
3
|
+
description: When to use lmdebug, common invocations, and project-specific gotchas
|
|
4
|
+
type: feedback
|
|
5
|
+
tags: [skills, lmdebug]
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# lmdebug skills
|
|
9
|
+
|
|
10
|
+
<!-- Agents: fill in concrete examples from your first walk-through of this project. -->
|
|
11
|
+
<!-- This stub gives structure; you supply the project-specific content. -->
|
|
12
|
+
|
|
13
|
+
## When to use lmdebug
|
|
14
|
+
|
|
15
|
+
[TODO: describe the scenarios in this project where lmdebug is the right tool —
|
|
16
|
+
e.g., inspecting a specific failure mode, tracing an unexpected code path, etc.]
|
|
17
|
+
|
|
18
|
+
## Common invocations
|
|
19
|
+
|
|
20
|
+
[TODO: list the lmdebug commands / flags most useful for this project, with a one-line
|
|
21
|
+
description of what each does and when to reach for it.]
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
# Example placeholder — replace with real commands:
|
|
25
|
+
# lmdebug attach --target <...> --breakpoint <...>
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Gotchas
|
|
29
|
+
|
|
30
|
+
[TODO: note any project-specific pitfalls — environment requirements, symbol resolution
|
|
31
|
+
issues, known interpreter quirks — discovered during real usage.]
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Skills — lmprobe
|
|
3
|
+
description: When to use lmprobe, common invocations, and project-specific gotchas
|
|
4
|
+
type: feedback
|
|
5
|
+
tags: [skills, lmprobe]
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# lmprobe skills
|
|
9
|
+
|
|
10
|
+
<!-- Agents: fill in concrete examples from your first walk-through of this project. -->
|
|
11
|
+
<!-- This stub gives structure; you supply the project-specific content. -->
|
|
12
|
+
|
|
13
|
+
## When to use lmprobe
|
|
14
|
+
|
|
15
|
+
[TODO: describe the scenarios in this project where lmprobe is the right tool —
|
|
16
|
+
e.g., reproducing a specific class of failure, validating a fix under load, etc.]
|
|
17
|
+
|
|
18
|
+
## Common invocations
|
|
19
|
+
|
|
20
|
+
[TODO: list the lmprobe commands / flags most useful for this project, with a one-line
|
|
21
|
+
description of what each does and when to reach for it.]
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
# Example placeholder — replace with real commands:
|
|
25
|
+
# lmprobe run --target <...> --scenario <...>
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Gotchas
|
|
29
|
+
|
|
30
|
+
[TODO: note any project-specific pitfalls — environment requirements, known flaky
|
|
31
|
+
scenarios, interpretation quirks — discovered during real usage.]
|
package/package.json
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@lmctl-ai/lmctl",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "lmctl — a durable multi-step agent workflow engine and CLI (public preview).",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"bin": {
|
|
7
|
+
"lmctl": "bin/lmctl"
|
|
8
|
+
},
|
|
9
|
+
"files": [
|
|
10
|
+
"dist/cli/index.js",
|
|
11
|
+
"dist/cli/schema.sql",
|
|
12
|
+
"dist/cli/side_effect_classifier.json",
|
|
13
|
+
"dist/config/",
|
|
14
|
+
"workflows/",
|
|
15
|
+
"bin/",
|
|
16
|
+
"package.json",
|
|
17
|
+
"LICENSE",
|
|
18
|
+
"README.md"
|
|
19
|
+
],
|
|
20
|
+
"engines": {
|
|
21
|
+
"node": ">=20"
|
|
22
|
+
},
|
|
23
|
+
"license": "SEE LICENSE IN LICENSE",
|
|
24
|
+
"publishConfig": {
|
|
25
|
+
"access": "public",
|
|
26
|
+
"tag": "staging"
|
|
27
|
+
},
|
|
28
|
+
"dependencies": {
|
|
29
|
+
"@aws-sdk/client-cognito-identity": "^3.1053.0",
|
|
30
|
+
"@aws-sdk/client-s3": "^3.1053.0",
|
|
31
|
+
"@aws-sdk/credential-provider-cognito-identity": "^3.972.36",
|
|
32
|
+
"@hono/node-server": "1.13.8",
|
|
33
|
+
"better-sqlite3": "11.5.0",
|
|
34
|
+
"docx": "^9.6.1",
|
|
35
|
+
"exceljs": "^4.4.0",
|
|
36
|
+
"hono": "4.6.20",
|
|
37
|
+
"jexl": "^2.3.0",
|
|
38
|
+
"marked": "^18.0.3",
|
|
39
|
+
"node-pty": "^1.1.0",
|
|
40
|
+
"zod": "4.4.3"
|
|
41
|
+
}
|
|
42
|
+
}
|