@thebackstoryis/engineering-with-ai 0.3.2 → 0.3.4-beta.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/Docs/README.md +6 -0
- package/Docs/context-management-and-token-efficiency.md +8 -0
- package/Docs/operations/jev-beta-experiment.md +80 -0
- package/Docs/operations/openai-decisions-beta.md +62 -0
- package/Docs/releases/0.3.4-beta.6.md +22 -0
- package/README.md +24 -1
- package/package.json +6 -4
- package/public/index.html +62 -0
- package/public/jev-credentials.js +43 -0
- package/public/jev.js +11 -0
- package/public/openai-credentials.js +43 -0
- package/public/styles.css +28 -4
- package/scripts/jev-experiment.mjs +15 -0
- package/src/cli.mjs +59 -14
- package/src/jev-credentials.mjs +104 -0
- package/src/jev.mjs +220 -0
- package/src/openai-credentials.mjs +104 -0
- package/src/openai-decisions.mjs +62 -0
- package/src/runtime/context-assembly.mjs +21 -3
- package/src/runtime/dashboard-server.mjs +56 -7
- package/src/runtime/jev-operations.mjs +160 -0
- package/src/runtime/mcp-server.mjs +26 -6
- package/tests/fixtures/jev-synthetic.json +8 -0
package/Docs/README.md
CHANGED
|
@@ -8,6 +8,8 @@ For feature delivery, **`ewai-deliver` coordinates the full fourteen-stage workf
|
|
|
8
8
|
|
|
9
9
|
If you want EWAI to act on an approved, exact pool of registered intents, read [governed autonomous intent delivery](autonomous-intent-delivery.md). It starts off, needs a named and bounded grant, and leaves human Build approval and Manual QA separate.
|
|
10
10
|
|
|
11
|
+
Read how [EWAI now supports Jev and the OpenAI Decisions API](releases/0.3.4-beta.6.md) in the latest beta.
|
|
12
|
+
|
|
11
13
|
## New to EWAI?
|
|
12
14
|
|
|
13
15
|
1. [Install EWAI](operations/installation-updating-and-entitlements.md) and open it in your project folder.
|
|
@@ -36,6 +38,8 @@ To try the beta communication guidance, read [concise answers and guided decisio
|
|
|
36
38
|
|
|
37
39
|
[Choose which views you need](operations/dashboard-configuration.md). Portfolio, Team Hub, policy tools and the other advanced views are optional and start hidden. Showing a view doesn't configure the service behind it; hiding one doesn't remove checks your project requires.
|
|
38
40
|
|
|
41
|
+
Try the optional [Jev decision-assistance beta](operations/jev-beta-experiment.md) to rank bounded options, compare LLM recommendations and recommend available personas. Start with shadow mode; overall coding savings remain unmeasured.
|
|
42
|
+
|
|
39
43
|
## Something isn't working?
|
|
40
44
|
|
|
41
45
|
Start with [troubleshooting and recovery](operations/troubleshooting-and-recovery.md). For a problem that needs support, [prepare a private error report](error-reporting-guide.md) and review it before sharing.
|
|
@@ -52,3 +56,5 @@ The [complete guide catalogue](guide-catalogue.md) separates everyday use, engin
|
|
|
52
56
|
- **An installed pack isn't automatically suitable.** Review shared guidance before adopting it. Executable adapters need explicit registration and run with your operating-system permissions; they aren't OS-sandboxed.
|
|
53
57
|
|
|
54
58
|
If you're changing EWAI itself, start in the [maintainer section](guide-catalogue.md#maintaining-ewai), not the application delivery instructions.
|
|
59
|
+
|
|
60
|
+
OpenAI Decisions joins the shared bounded engine in `0.3.4-beta.3`. New settings default off with Automatic provider preference; historical Jev policies retain their vendor consent. Private CLI/dashboard OpenAI key setup, independent text consent, typed conversion and provider-aware costs are described in the [OpenAI Decisions beta guide](operations/openai-decisions-beta.md). Live OpenAI access and completed-task savings remain unmeasured.
|
|
@@ -153,3 +153,11 @@ If you're changing EWAI itself, follow [maintainer context benchmarks](maintaine
|
|
|
153
153
|
- If a digest, source, phase or task has changed, prepare a new pack and review the delta.
|
|
154
154
|
|
|
155
155
|
Context efficiency is useful only when the resulting engineering work remains at least as reliable, testable, secure and accountable as the full-context baseline.
|
|
156
|
+
|
|
157
|
+
## Jev beta decision assistance
|
|
158
|
+
|
|
159
|
+
The local `0.3.4-beta.1` experiment adds an opt-in structured decision path before a larger model comparison. Jev can order bounded options, compare supported LLM recommendations and recommend installed personas from metadata. See [Jev setup and experiment limits](operations/jev-beta-experiment.md). It starts off and first enablement should use shadow. No incremental token or cost saving has been measured on equivalent completed coding tasks; the synthetic runner measures Jev's own usage and latency. Publication and human Manual QA remain separate.
|
|
160
|
+
|
|
161
|
+
OpenAI Decisions joins the shared bounded engine in experimental `0.3.4-beta.3`. New settings default off with Automatic provider preference; historical Jev policies retain their vendor consent. Private CLI/dashboard OpenAI key setup, independent text consent, typed conversion and provider-aware costs are described in the [OpenAI Decisions beta guide](operations/openai-decisions-beta.md). Live OpenAI access and completed-task savings remain unmeasured.
|
|
162
|
+
|
|
163
|
+
`0.3.4-beta.6` includes these integrations. Read the [release update](releases/0.3.4-beta.6.md).
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# Jev decision assistance beta
|
|
2
|
+
|
|
3
|
+
Jev evaluates bounded choice, score and yes/no questions. Use it to rank an existing set of options before asking a generative model to compare the whole set. It can also compare a structured recommendation returned by an LLM. This experiment does not replace coding models, validate arbitrary prose or establish that an answer is correct.
|
|
4
|
+
|
|
5
|
+
## Enablement and credentials
|
|
6
|
+
|
|
7
|
+
Jev is off by default. In Configuration, use **Jev API credentials** to check and save a TypeSafe key, then select **Shadow** in **Decision assistance**, choose uses, accept the text-processing disclosure and save. First enablement should use shadow. Select Active explicitly after reviewing measurements. Saving credentials does not enable inference.
|
|
8
|
+
|
|
9
|
+
CLI equivalents:
|
|
10
|
+
|
|
11
|
+
```sh
|
|
12
|
+
ewai jev credentials configure --project .
|
|
13
|
+
ewai jev configure --mode shadow --use-cases context,personas,skill,claim,failure,impact,tests --acknowledge-cloud-processing --yes --project .
|
|
14
|
+
ewai jev status --project . --json
|
|
15
|
+
ewai jev measurements --project . --json
|
|
16
|
+
ewai jev experiment --limit 6 --project . --json
|
|
17
|
+
ewai jev configure --mode off --yes --project .
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Credential configuration requires a genuinely interactive hidden terminal prompt. Never put a key in arguments or project JSON. The saved key is an owner-only **unencrypted** file at `~/.ewai/credentials/jev.json`, outside the project. `TYPESAFE_API_KEY` in the launching environment takes precedence. Credential status, check and remove commands are available; removing a saved key does not remove an environment override. Check uses the fixed metadata endpoint without inference and does not prove credit or decision quality.
|
|
21
|
+
|
|
22
|
+
CLI, dashboard and MCP share project-local settings at `.ewai-pipeline/jev.json`. Saves require explicit confirmation and the current revision. An off policy makes no inference calls. Shadow automatic journeys retain their local baseline; explicit decision tools return suggestions for comparison. Active mode permits the bounded automatic applications below. No recommendation grants approval, executes a suggested skill, installs a persona pack, changes provider roles or removes required reviews/tests.
|
|
23
|
+
|
|
24
|
+
## Use boundaries
|
|
25
|
+
|
|
26
|
+
| Use | Integration | Boundaries |
|
|
27
|
+
|---|---|---|
|
|
28
|
+
| Context | CLI/MCP/dashboard context preparation | Rank nonmandatory segment labels; reassemble locally, preserve mandatory evidence, fidelity and overflow failures. Source bodies stay local. |
|
|
29
|
+
| Personas | Context preparation and `jev personas` / `ewai_jev_personas` | Resolve core, project, personal and installed premium metadata locally; score names, categories, tags and capabilities for up to 256 unique available personas in batches of 16. Expose considered/available counts and incomplete coverage. Add eligible advice only after complete evaluation; retain baseline reviewers. |
|
|
30
|
+
| Skills and answer options | `jev decide`, `ewai_jev_decide`, focused companion status | Choice probabilities order up to 31 candidates (each combined label and description at most 255 characters) and select the recommended ID. Companion automatically reorders only equal-class eligible work, preserving human/blocked positions and handoffs. |
|
|
31
|
+
| Claim support | Explicit decision tool | Compare the full claim with supplied eligible evidence: supports, contradicts or insufficient. A supplied quotation must exist verbatim locally before calling. Advice is not factual verification or a gate pass. |
|
|
32
|
+
| Failure triage | Explicit decision tool | Classify a supplied sanitised diagnostic into auth, rate, dependency or unknown. Never upload automatic raw process capture or execute a remedy. |
|
|
33
|
+
| Impact | Dashboard impact preview | Preserve canonical route recommendations and personas. Show Jev specialist advice in the inferred-consequences cards and a supplemental API field; human changes still require a rationale. |
|
|
34
|
+
| Tests | Explicit decision tool | Rank supplemental tests; return mandatory test IDs unchanged. Required tests cannot be deselected. |
|
|
35
|
+
|
|
36
|
+
Persona recommendation without sending the catalogue to the coding LLM:
|
|
37
|
+
|
|
38
|
+
```sh
|
|
39
|
+
ewai jev personas --focus 'Review keyboard access to the settings form' --limit 4 --project . --json
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
The response contains safe IDs, names, tiers, scores and coverage. Tier describes provenance, not authority. Premium recommendations require an installed persona; this operation never purchases, downloads or updates a pack. Automatic persona advice can add to the local baseline up to its existing eight-persona cap.
|
|
43
|
+
|
|
44
|
+
## Rank options and review LLM recommendations
|
|
45
|
+
|
|
46
|
+
Write a bounded decision input to a project-local JSON file, then call `ewai jev decide --input decisions/options.json --project . --json`, or send the same fields directly to `ewai_jev_decide`. The dashboard exposes the same engine at `/api/jev/decisions` to matching-origin callers.
|
|
47
|
+
|
|
48
|
+
```json
|
|
49
|
+
{
|
|
50
|
+
"useCase": "skill",
|
|
51
|
+
"source": "public",
|
|
52
|
+
"task": "Recommend reusable model-to-model access in Laravel",
|
|
53
|
+
"candidates": [
|
|
54
|
+
{"id": "relationship", "label": "Native Eloquent relationship"},
|
|
55
|
+
{"id": "queries", "label": "Duplicate controller queries"}
|
|
56
|
+
]
|
|
57
|
+
}
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
The result supplies `orderedIds`, `recommendedId`, `decision`, probabilities and confidence. `none` means no suitable option; it is excluded from ordered candidate IDs and gives a null recommendation. Probabilities express the model's judgement, not an independently measured correctness rate.
|
|
61
|
+
|
|
62
|
+
For supported structured options already returned by an LLM, also supply `"origin": "llm"` and `"recommendedId": "queries"`. Jev evaluates the candidates without being shown the LLM's preferred ID, then returns `validation.status` as `agrees`, `disagrees` or `inconclusive`. A recommendation ID outside the supplied set or unsupported shape is rejected. Agreement is a semantic comparison, not proof of validity. Keep the existing recommendation on timeout, low confidence or other inconclusive outcomes; escalate unresolved decisions when needed.
|
|
63
|
+
|
|
64
|
+
Use these tools as preflight operations where bounded choices are sufficient. They do not intercept every host-model turn or guarantee that a host follows advice. Avoid asking a larger model to re-rank the same full set unless Jev falls back or the decision needs reasoning beyond the rubric.
|
|
65
|
+
|
|
66
|
+
## Data, budgets and measurements
|
|
67
|
+
|
|
68
|
+
Enablement discloses that focus text, selected labels and descriptions leave the machine for TypeSafe. Automatic persona ranking uses names, categories, tags and capabilities, excluding descriptions that might have been derived from persona bodies. Explicit checks can send evidence or sanitised diagnostics supplied by the caller. Automatic ranking excludes project source and persona bodies. Explicit sources must be `public`, `synthetic`, `cloud-approved` or `metadata-only`; unknown/imported/cloud-denied material is refused. Source assertions are the caller's responsibility. Common credential patterns and the configured key are rejected, but pattern matching is not a complete data-classification system. Do not label confidential text metadata-only to bypass its handling policy.
|
|
69
|
+
|
|
70
|
+
Requests use the fixed HTTPS TypeSafe endpoint and pinned `jev-1.13.0`, with no retries or redirects. Defaults: 100 calls, 100,000 reserved input tokens, three-second deadline and 0.65 minimum confidence. Limits apply across processes to a saved policy revision; a new explicitly saved revision starts another bounded budget. UTF-8 bytes plus headroom reserve input capacity conservatively. Reported usage determines measured cost; over-reservation or unknown usage blocks further calls in the current budget. A cache hit requires the same policy, credential identity and request; disabling or changing these invalidates application of in-flight advice.
|
|
71
|
+
|
|
72
|
+
Telemetry stores bounded IDs, outcome codes, numeric results, model, usage, timings and effects. It excludes prompts, candidate descriptions, evidence bodies, raw provider responses and credentials. Numeric cache entries are bounded separately. The history retains the latest 500 observation events; latency/call/fallback/cache figures cover that history, while recorded input and cost totals accumulate across revisions. Before dispatch, an unresolved reservation is persisted. Concurrent or interrupted unresolved calls block new inference in that policy revision and leave total cost unknown; it is never silently reported as zero. Measurements distinguish suggestions from application; downstream savings remain unknown.
|
|
73
|
+
|
|
74
|
+
The synthetic runner evaluates at most six labelled challenge cases using the already enabled policy and configured key. It never widens limits or enables Jev. Run comparisons on equivalent completed coding tasks before claiming lower overall token use, cost or elapsed time. Include Jev's own latency, cache hits, fallback frequency, reviewer quality and any large-model calls still needed. A local deterministic selector is usually faster than a network call; Jev's potential benefit is better semantic selection or avoiding a more expensive model comparison.
|
|
75
|
+
|
|
76
|
+
API and model references: [TypeSafe API](https://docs.typesafe.ai/api), [models and pricing](https://docs.typesafe.ai/models), [confidence](https://docs.typesafe.ai/confidence), [coding-agent guidance](https://docs.typesafe.ai/introduction/coding-agents).
|
|
77
|
+
|
|
78
|
+
This is an experimental beta. Packaging and automated browser checks do not replace owner Manual QA or establish completed coding-task savings.
|
|
79
|
+
|
|
80
|
+
OpenAI Decisions joins the shared bounded engine in `0.3.4-beta.3`. New settings default off with Automatic provider preference; historical Jev policies retain their vendor consent. Private CLI/dashboard OpenAI key setup, independent text consent, typed conversion and provider-aware costs are described in the [OpenAI Decisions beta guide](openai-decisions-beta.md). Live OpenAI access and completed-task savings remain unmeasured.
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
# OpenAI Decisions beta
|
|
2
|
+
|
|
3
|
+
Experimental `0.3.4-beta.3` adds OpenAI Decisions to the bounded decision assistance introduced with Jev. Start in shadow mode and compare suggestions before explicitly choosing active mode.
|
|
4
|
+
|
|
5
|
+
## Private setup
|
|
6
|
+
|
|
7
|
+
Open Configuration → OpenAI API credentials in the local dashboard, or use a genuine interactive terminal:
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
ewai providers credentials openai configure --project .
|
|
11
|
+
ewai providers credentials openai status --project . --json
|
|
12
|
+
ewai providers credentials openai check --project .
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
The password form and hidden terminal check metadata authentication without inference. This requires models-read permission and does not establish Decisions access, credit or decision quality. A failed replacement retains the saved predecessor. The key is stored outside projects in `~/.ewai/credentials/openai.json`, in an owner-only local file, **unencrypted**. `OPENAI_API_KEY` in the launching environment overrides the saved key. Use that environment route for a restricted key without metadata-read permission. Saving credentials does not enable decision assistance. Do not put keys in chat, arguments, project configuration, SPECS or logs. EWAI never uses Codex OAuth credentials for these calls.
|
|
16
|
+
|
|
17
|
+
Remove only the saved credential, with the revision from status:
|
|
18
|
+
|
|
19
|
+
```sh
|
|
20
|
+
ewai providers credentials openai remove --expected-revision REVISION --yes --project .
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
An environment override remains active after removing a saved key. Unsafe paths, permissions, links or stale replacement requests fail closed. A valid environment override remains usable when saved storage needs repair; local writes stay unavailable until repaired.
|
|
24
|
+
|
|
25
|
+
## Choose participation and provider
|
|
26
|
+
|
|
27
|
+
New projects start **off**, with **Automatic** preference and neither vendor consent. Automatic uses a configured, separately consented OpenAI key first, otherwise a configured, consented Jev key, otherwise local decisions. It is an eligibility choice, not a claim that the account has endpoint access. Installing Codex alone does not authorise API use. Existing Jev policies retain Jev preference and TypeSafe-only consent until explicitly changed.
|
|
28
|
+
|
|
29
|
+
The dashboard has separate OpenAI and TypeSafe text disclosures. Equivalent CLI setup is:
|
|
30
|
+
|
|
31
|
+
```sh
|
|
32
|
+
ewai decisions configure --provider auto --mode shadow --acknowledge-openai-processing --acknowledge-cloud-processing --yes --project .
|
|
33
|
+
ewai decisions status --project . --json
|
|
34
|
+
ewai decisions measurements --project . --json
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
Only acknowledge vendors you authorise. Revoke independently with `--revoke-openai-processing` or `--revoke-cloud-processing` (TypeSafe). If no selected vendor consent remains, also choose `--mode off`. Acknowledgement and revocation for the same vendor cannot be combined. Use `--provider openai` or `--provider jev` for an explicit preference. Existing `ewai jev` commands and `/api/jev` routes remain compatible; `ewai decisions`, `/api/decisions` and `ewai_decisions_*` MCP tools expose the same policy and operations. Model-facing MCP tools cannot capture keys.
|
|
38
|
+
|
|
39
|
+
Both providers are optional. Off makes no paid calls or credential-resolution attempts; unavailable provider credentials do not block the local decision baseline. Ordinary status reports unavailable credentials without losing the other provider’s status. Shadow records suggestions without changing existing automatic decisions. Active can apply eligible context ordering and available persona additions. Required evidence, baseline persona records, mandatory tests, human decisions and canonical impact routes remain protected.
|
|
40
|
+
|
|
41
|
+
## Supported decisions
|
|
42
|
+
|
|
43
|
+
The same seven operations support context ordering, available persona recommendations, skills or answer options, claim support, failure triage, supplemental impact review and supplemental test relevance. For bounded options:
|
|
44
|
+
|
|
45
|
+
```sh
|
|
46
|
+
ewai decisions decide --input decision.json --project .
|
|
47
|
+
ewai decisions personas --focus 'Review accessibility and credential setup' --limit 4 --project .
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
The project-local JSON contract is documented in the [Jev beta guide](jev-beta-experiment.md). Supported LLM comparisons use `origin: "llm"` and the existing `recommendedId`; results are agrees, disagrees or inconclusive, never factual proof. Only locally available core, project, personal and installed premium personas are considered. No premium download occurs, and persona bodies are excluded from automatic requests. Typed advice cannot execute tools or grant work approval.
|
|
51
|
+
|
|
52
|
+
## Transport, accounting and limits
|
|
53
|
+
|
|
54
|
+
OpenAI calls use fixed HTTPS `POST /v1/decisions` with pinned `gpt-6-luna`. The converter maps named choice, zero-based rubric score and predicate arrays into the established operation-facing envelope. Wrong models, names, types, values, distributions, scores, refusals or usage fail safely. Only bounded text is used; no images or external image fetching.
|
|
55
|
+
|
|
56
|
+
Both vendors share one atomic call and conservative input-byte reservation budget, pending-call records, privacy filters and whole-response deadline. Unknown/imported/denied material is excluded. Provider/model, policy revision and credential identity participate in exact caching and freshness. A provider or key change invalidates stale advice. A failed paid request is never retried against a second vendor. Unknown usage stops further calls within that policy budget and remains unknown in cumulative cost reporting, including after a policy revision. Explicitly saving a new policy starts a new bounded budget.
|
|
57
|
+
|
|
58
|
+
Numeric measurements identify provider and model and preserve reported input/cache/output details. The base estimate uses Jev USD0.042/M input or OpenAI USD0.10/M input per recorded call. OpenAI estimates precede free-cache adjustments and regional or long-context premiums; these are **not invoices**. Recent history is capped at 500 events while cumulative known input/cost totals cover all recorded calls. Requests, source, keys, instructions and persona bodies are not retained in telemetry.
|
|
59
|
+
|
|
60
|
+
OpenAI integration is verified with mock transports and synthetic local projects until an owner privately configures a usable key. Live account access, OpenAI quality/latency and completed coding-task token, time or cost savings are unmeasured. Vendor speed statements are not EWAI results. Manual QA and public release remain separate.
|
|
61
|
+
|
|
62
|
+
Primary protocol and pricing sources: [Decisions guide](https://developers.openai.com/api/docs/guides/decisions), [create reference](https://developers.openai.com/api/reference/resources/decisions/methods/create), [TypeSafe models](https://docs.typesafe.ai/models).
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# New in 0.3.4 Beta 6 release
|
|
2
|
+
|
|
3
|
+
**EWAI now supports Jev and the OpenAI Decisions API.**
|
|
4
|
+
|
|
5
|
+
- **Rank options and recommend a choice.** Evaluate available approaches and return an ordered list with a recommended answer.
|
|
6
|
+
- **Check AI recommendations.** Compare supported options returned by a larger language model with a second recommendation.
|
|
7
|
+
- **Recommend suitable personas.** Select from personas available to your project, including core, project, personal and installed premium personas.
|
|
8
|
+
- **Reduce larger-model calls.** Evaluate supported choices through focused decision calls instead of sending the same options to a larger language model.
|
|
9
|
+
- **Prioritise context.** Help order supporting context for coding and review tasks.
|
|
10
|
+
- **Support additional checks.** Assess supplied claim evidence, classify sanitised failures and recommend supplementary impact reviews and tests.
|
|
11
|
+
- **Choose your provider.** Select Jev, OpenAI Decisions or Automatic. Automatic prefers configured and authorised OpenAI access, then Jev.
|
|
12
|
+
- **Configure through the CLI or dashboard.** Save API keys privately, enable either integration and select the uses you want.
|
|
13
|
+
- **Choose shadow or active mode.** Shadow shows recommendations for comparison. Active can apply supported recommendations to context ordering and persona selection.
|
|
14
|
+
- **Track usage and estimated cost.** View decision-call usage and set call and input-token limits.
|
|
15
|
+
|
|
16
|
+
Both integrations are optional and start disabled. EWAI continues to work without either service.
|
|
17
|
+
|
|
18
|
+
Install or update from the npm beta channel:
|
|
19
|
+
|
|
20
|
+
```sh
|
|
21
|
+
npm install --save-dev @thebackstoryis/engineering-with-ai@beta
|
|
22
|
+
```
|
package/README.md
CHANGED
|
@@ -1,5 +1,28 @@
|
|
|
1
1
|
# Engineering With AI
|
|
2
2
|
|
|
3
|
+
## New in 0.3.4 Beta 6 release
|
|
4
|
+
|
|
5
|
+
**EWAI now supports Jev and the OpenAI Decisions API.**
|
|
6
|
+
|
|
7
|
+
- **Rank options and recommend a choice.** Evaluate available approaches and return an ordered list with a recommended answer.
|
|
8
|
+
- **Check AI recommendations.** Compare supported options returned by a larger language model with a second recommendation.
|
|
9
|
+
- **Recommend suitable personas.** Select from personas available to your project, including core, project, personal and installed premium personas.
|
|
10
|
+
- **Reduce larger-model calls.** Evaluate supported choices through focused decision calls instead of sending the same options to a larger language model.
|
|
11
|
+
- **Prioritise context.** Help order supporting context for coding and review tasks.
|
|
12
|
+
- **Support additional checks.** Assess supplied claim evidence, classify sanitised failures and recommend supplementary impact reviews and tests.
|
|
13
|
+
- **Choose your provider.** Select Jev, OpenAI Decisions or Automatic. Automatic prefers configured and authorised OpenAI access, then Jev.
|
|
14
|
+
- **Configure through the CLI or dashboard.** Save API keys privately, enable either integration and select the uses you want.
|
|
15
|
+
- **Choose shadow or active mode.** Shadow shows recommendations for comparison. Active can apply supported recommendations to context ordering and persona selection.
|
|
16
|
+
- **Track usage and estimated cost.** View decision-call usage and set call and input-token limits.
|
|
17
|
+
|
|
18
|
+
Both integrations are optional and start disabled. EWAI continues to work without either service.
|
|
19
|
+
|
|
20
|
+
Install or update from the npm beta channel:
|
|
21
|
+
|
|
22
|
+
```sh
|
|
23
|
+
npm install --save-dev @thebackstoryis/engineering-with-ai@beta
|
|
24
|
+
```
|
|
25
|
+
|
|
3
26
|
Engineering With AI (EWAI) helps you plan, build and review software with an AI assistant. It gives the assistant a shared record of the project, a delivery workflow and checks against your engineering standards. You keep control of the decisions and approve implementation before it starts.
|
|
4
27
|
|
|
5
28
|
[](https://www.conversationalcoding.dev/engineering-with-ai-harness/?utm_source=readme&utm_medium=referral&utm_campaign=ewai)
|
|
@@ -18,7 +41,7 @@ ewai
|
|
|
18
41
|
|
|
19
42
|
`ewai` opens your assistant and walks you through setting up the project. See [Install and start](#install-and-start) for details.
|
|
20
43
|
|
|
21
|
-
|
|
44
|
+
Versions `0.3.2` and `0.3.3` update the package's links and documentation. The features below arrived in `0.3.1`.
|
|
22
45
|
|
|
23
46
|
## New in 0.3.1: concise answers and Grok Build
|
|
24
47
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@thebackstoryis/engineering-with-ai",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.4-beta.6",
|
|
4
4
|
"description": "A human-centred, AI-augmented engineering pipeline",
|
|
5
5
|
"license": "SEE LICENSE IN LICENSE",
|
|
6
6
|
"author": "The Backstory Is",
|
|
@@ -25,7 +25,7 @@
|
|
|
25
25
|
"type": "git",
|
|
26
26
|
"url": "git+https://github.com/TheBackstoryIs/EngineeringWithAIHarness.git"
|
|
27
27
|
},
|
|
28
|
-
"homepage": "https://www.conversationalcoding.dev/
|
|
28
|
+
"homepage": "https://www.conversationalcoding.dev/ewai",
|
|
29
29
|
"bugs": {
|
|
30
30
|
"url": "https://github.com/TheBackstoryIs/EngineeringWithAIHarness/issues"
|
|
31
31
|
},
|
|
@@ -51,7 +51,9 @@
|
|
|
51
51
|
"tests/fixtures/context-benchmarks.json",
|
|
52
52
|
"!config/*-source-manifest.json",
|
|
53
53
|
"!config/workflow-migration-map.json",
|
|
54
|
-
"templates/"
|
|
54
|
+
"templates/",
|
|
55
|
+
"scripts/jev-experiment.mjs",
|
|
56
|
+
"tests/fixtures/jev-synthetic.json"
|
|
55
57
|
],
|
|
56
58
|
"publishConfig": {
|
|
57
59
|
"access": "public",
|
|
@@ -59,7 +61,7 @@
|
|
|
59
61
|
},
|
|
60
62
|
"scripts": {
|
|
61
63
|
"build": "npm run check",
|
|
62
|
-
"precheck": "node scripts/publication-check.mjs --source-only && node --check src/runtime/tree-sitter-index.mjs && node --check src/design-system-application.mjs && node --check src/team-hub-resources.mjs && node --check src/dashboard-preferences.mjs && node --check public/dashboard-navigation.js && node --check src/config-document.mjs && node --check src/coding-providers.mjs && node --check src/runtime/grok-provider.mjs && node --check public/coding-providers.js && node --check src/grok-credentials.mjs && node --check public/grok-credentials.js",
|
|
64
|
+
"precheck": "node scripts/publication-check.mjs --source-only && node --check src/runtime/tree-sitter-index.mjs && node --check src/design-system-application.mjs && node --check src/team-hub-resources.mjs && node --check src/dashboard-preferences.mjs && node --check public/dashboard-navigation.js && node --check src/config-document.mjs && node --check src/coding-providers.mjs && node --check src/runtime/grok-provider.mjs && node --check public/coding-providers.js && node --check src/grok-credentials.mjs && node --check public/grok-credentials.js && node --check src/jev-credentials.mjs && node --check src/jev.mjs && node --check src/runtime/jev-operations.mjs && node --check public/jev.js && node --check public/jev-credentials.js && node --check scripts/jev-experiment.mjs && node --check src/openai-credentials.mjs && node --check src/openai-decisions.mjs && node --check public/openai-credentials.js",
|
|
63
65
|
"publication:check": "node scripts/publication-check.mjs",
|
|
64
66
|
"prepack": "node scripts/publication-check.mjs --quiet",
|
|
65
67
|
"check": "node --check src/error-reporting.mjs && node --check src/team-hub.mjs && node --check src/runtime/team-hub-client.mjs && node --check src/runtime/team-hub-database.mjs && node --check src/runtime/team-hub-server.mjs && node --check src/runtime/team-hub.mjs && node --check src/runtime/error-reporting.mjs && node --check src/launcher.mjs && node --check src/afk-worker.mjs && node --check src/companion-opening.mjs && node --check src/companion.mjs && node --check src/cli.mjs && node --check src/checkin.mjs && node --check src/persona-licence-config.mjs && node --check src/persona-website-provider.mjs && node --check src/persona-zip.mjs && node --check src/persona-entitlements.mjs && node --check src/context.mjs && node --check src/archaeology.mjs && node --check src/evidence-depth.mjs && node --check src/prototype-iterations.mjs && node --check src/solution-readiness.mjs && node --check src/delivery-artifacts.mjs && node --check src/delivery-documents.mjs && node --check src/delivery-gates.mjs && node --check src/delivery.mjs && node --check src/design-systems.mjs && node --check src/design-system-authoring.mjs && node --check src/execution-state.mjs && node --check src/intent-dependencies.mjs && node --check src/task-graph.mjs && node --check src/test-scenarios.mjs && node --check src/security-validation-config.mjs && node --check src/starter-materialisation-contract.mjs && node --check src/validation-config.mjs && node --check src/project.mjs && node --check src/install.mjs && node --check src/intents.mjs && node --check src/intent-maps.mjs && node --check src/discovery.mjs && node --check src/packs.mjs && node --check src/organisation-blueprints.mjs && node --check src/organisation-policies.mjs && node --check src/policy-design-gates.mjs && node --check src/policy-gate-integration.mjs && node --check src/personas.mjs && node --check src/platform-metadata-analysis.mjs && node --check src/power-platform-source-map.mjs && node --check src/salesforce-source-map.mjs && node --check src/repository-source-map.mjs && node --check src/runtime/context-assembly.mjs && node --check src/runtime/context-benchmarks.mjs && node --check src/runtime/database.mjs && node --check src/runtime/execution-leases.mjs && node --check src/runtime/provider-adapters.mjs && node --check src/runtime/security-validation.mjs && node --check src/runtime/starter-materialisation.mjs && node --check src/runtime/policy-workspace.mjs && node --check src/runtime/lifecycle-hooks.mjs && node --check src/runtime/afk-conductor.mjs && node --check src/runtime/dashboard-actions.mjs && node --check src/runtime/dashboard-handoffs.mjs && node --check src/runtime/persona-engagement.mjs && node --check src/runtime/evidence-depth-workspace.mjs && node --check src/runtime/prototype-iterations.mjs && node --check src/runtime/guided-discovery.mjs && node --check src/runtime/guided-intents.mjs && node --check src/runtime/phase-contributions.mjs && node --check src/runtime/impact-analysis.mjs && node --check src/runtime/paths.mjs && node --check src/runtime/intents.mjs && node --check src/runtime/knowledge.mjs && node --check src/runtime/palace.mjs && node --check src/runtime/repository-index.mjs && node --check src/runtime/runs.mjs && node --check src/runtime/work.mjs && node --check src/runtime/version.mjs && node --check src/runtime/dashboard.mjs && node --check src/runtime/dashboard-server.mjs && node --check src/runtime/mcp-config.mjs && node --check src/runtime/mcp-server.mjs && node --check src/autonomy.mjs && node --check src/autonomy-phase-contracts.mjs && node --check src/autonomy-worker.mjs && node --check src/runtime/autonomy-supervisor.mjs && node --check src/runtime/autonomy-operations.mjs && node --check src/runtime/autonomy-workers.mjs && node --check src/runtime/autonomy-workspace.mjs && node --check public/autonomy.js && node --check public/app.js && node --check public/team-hub/app.js && node --check scripts/setup.mjs",
|
package/public/index.html
CHANGED
|
@@ -86,6 +86,65 @@
|
|
|
86
86
|
<output id="grokCredentialNotice" role="status" aria-live="polite"></output>
|
|
87
87
|
</div>
|
|
88
88
|
</section>
|
|
89
|
+
<section class="configuration-group jev-configuration" aria-labelledby="jevTitle">
|
|
90
|
+
<header><div><h3 id="jevTitle">Decision assistance <span class="jev-beta">Beta</span></h3><p>Rank options, compare recommendations and find relevant reviewers before asking a larger model.</p></div></header>
|
|
91
|
+
<div class="jev-body">
|
|
92
|
+
<form id="jevSettingsForm">
|
|
93
|
+
<fieldset id="jevControls" disabled>
|
|
94
|
+
<legend>How decision assistance participates</legend>
|
|
95
|
+
<label class="jev-provider-label" for="jevProvider">Decision provider<select id="jevProvider"><option value="auto">Automatic — OpenAI first</option><option value="openai">OpenAI Decisions</option><option value="jev">Jev (TypeSafe)</option></select></label><p>Automatic uses a configured OpenAI key with OpenAI consent first, otherwise a configured Jev key with TypeSafe consent. API access is established by a decision attempt; Codex sign-in is separate.</p>
|
|
96
|
+
<div class="jev-modes">
|
|
97
|
+
<label><input type="radio" name="jevMode" value="off" checked> Off <small>Use existing local decisions.</small></label>
|
|
98
|
+
<label><input type="radio" name="jevMode" value="shadow"> Shadow — recommended first <small>Measure suggestions; keep existing automatic decisions.</small></label>
|
|
99
|
+
<label><input type="radio" name="jevMode" value="active"> Active <small>Apply eligible context and persona advice. Required evidence and approvals remain.</small></label>
|
|
100
|
+
</div>
|
|
101
|
+
<fieldset><legend>Where decision assistance can help</legend><div id="jevUseCases" class="jev-use-cases"></div></fieldset>
|
|
102
|
+
<div class="jev-disclosure"><strong>Text processing by TypeSafe</strong><p>Enabled decisions send your focus and selected labels or descriptions to TypeSafe. Explicit claim and diagnostic checks can send eligible evidence you supply. Persona bodies and project source files are excluded from automatic ranking.</p><label><input id="jevCloudConsent" type="checkbox"> I accept TypeSafe text processing for enabled decisions.</label></div>
|
|
103
|
+
<div class="jev-disclosure"><strong>Text processing by OpenAI</strong><p>Enabled OpenAI decisions send the same eligible focus, labels, descriptions or supplied evidence to OpenAI. Persona bodies and project source files are excluded from automatic ranking.</p><label><input id="openaiCloudConsent" type="checkbox"> I accept OpenAI text processing for enabled decisions.</label></div>
|
|
104
|
+
<div class="jev-limits">
|
|
105
|
+
<label>Maximum calls per saved policy<input id="jevMaxCalls" type="number" min="1" max="1000" required></label>
|
|
106
|
+
<label>Input-token reservation budget<input id="jevMaxTokens" type="number" min="2000" max="1000000" required></label>
|
|
107
|
+
<label>Timeout, milliseconds<input id="jevTimeout" type="number" min="100" max="10000" required></label>
|
|
108
|
+
<label>Minimum confidence<input id="jevConfidence" type="number" min="0.5" max="1" step="0.01" required></label>
|
|
109
|
+
</div>
|
|
110
|
+
<p>Saving a new policy starts a new bounded budget. Timeouts or unknown usage stop further calls within that budget. Confidence is advisory.</p>
|
|
111
|
+
</fieldset>
|
|
112
|
+
<div class="coding-provider-actions"><button id="jevSettingsSave" type="submit" class="primary-action" disabled>Save decision settings</button><button id="jevRefresh" type="button" class="quiet-button">Refresh settings and measurements</button></div>
|
|
113
|
+
<output id="jevNotice" role="status" aria-live="polite">Loading decision settings…</output>
|
|
114
|
+
</form>
|
|
115
|
+
<section aria-labelledby="jevMeasurementsTitle"><h4 id="jevMeasurementsTitle">Measured results</h4><div id="jevMeasurements" aria-live="polite">Unknown — no measurements loaded.</div><p>Latency and cost describe the selected provider calls. Base estimates precede OpenAI free-cache adjustments and regional or long-context premiums. Downstream token savings and coding speed remain unknown until compared with equivalent work.</p></section>
|
|
116
|
+
</div>
|
|
117
|
+
</section>
|
|
118
|
+
<section class="configuration-group jev-credentials" aria-labelledby="jevCredentialsTitle">
|
|
119
|
+
<header><div><h3 id="jevCredentialsTitle">Jev API credentials</h3><p>Connect TypeSafe for bounded decision assistance. Saving a key leaves Jev off.</p></div></header>
|
|
120
|
+
<div class="jev-credentials-body">
|
|
121
|
+
<p id="jevCredentialStatus" role="status">Checking credential status…</p>
|
|
122
|
+
<form id="jevCredentialForm">
|
|
123
|
+
<label for="jevCredentialKey">TypeSafe API key</label>
|
|
124
|
+
<input id="jevCredentialKey" type="password" autocomplete="off" spellcheck="false" autocapitalize="off" maxlength="512" required aria-describedby="jevCredentialHelp" disabled>
|
|
125
|
+
<p id="jevCredentialHelp">Saved for your account on this computer, across EWAI projects, outside the project in an owner-only local file. The stored key is not encrypted. TYPESAFE_API_KEY overrides it.</p>
|
|
126
|
+
<div class="coding-provider-actions"><button id="jevCredentialSave" type="submit" class="primary-action" disabled>Check and save key</button><button id="jevCredentialCancel" type="button" class="quiet-button" disabled>Cancel key entry</button></div>
|
|
127
|
+
</form>
|
|
128
|
+
<div class="coding-provider-actions"><button id="jevCredentialCheck" type="button" class="quiet-button" disabled>Check connection</button><button id="jevCredentialRemove" type="button" class="quiet-button" disabled>Remove saved key</button><button id="jevCredentialRefresh" type="button" class="quiet-button">Refresh status</button></div>
|
|
129
|
+
<p>Checking authenticates with TypeSafe without generating output. It does not prove available credit or decision quality. Saving a key does not enable Jev.</p>
|
|
130
|
+
<output id="jevCredentialNotice" role="status" aria-live="polite"></output>
|
|
131
|
+
</div>
|
|
132
|
+
</section>
|
|
133
|
+
<section class="configuration-group openai-credentials" aria-labelledby="openaiCredentialsTitle">
|
|
134
|
+
<header><div><h3 id="openaiCredentialsTitle">OpenAI API credentials</h3><p>Connect OpenAI for bounded decision assistance. Saving a key does not enable decision assistance.</p></div></header>
|
|
135
|
+
<div class="openai-credentials-body">
|
|
136
|
+
<p id="openaiCredentialStatus" role="status">Checking credential status…</p>
|
|
137
|
+
<form id="openaiCredentialForm">
|
|
138
|
+
<label for="openaiCredentialKey">OpenAI API key</label>
|
|
139
|
+
<input id="openaiCredentialKey" type="password" autocomplete="off" spellcheck="false" autocapitalize="off" maxlength="512" required aria-describedby="openaiCredentialHelp" disabled>
|
|
140
|
+
<p id="openaiCredentialHelp">Saved for your account on this computer, across EWAI projects, outside the project in an owner-only local file. The stored key is not encrypted. OPENAI_API_KEY overrides it.</p>
|
|
141
|
+
<div class="coding-provider-actions"><button id="openaiCredentialSave" type="submit" class="primary-action" disabled>Check and save key</button><button id="openaiCredentialCancel" type="button" class="quiet-button" disabled>Cancel key entry</button></div>
|
|
142
|
+
</form>
|
|
143
|
+
<div class="coding-provider-actions"><button id="openaiCredentialCheck" type="button" class="quiet-button" disabled>Check connection</button><button id="openaiCredentialRemove" type="button" class="quiet-button" disabled>Remove saved key</button><button id="openaiCredentialRefresh" type="button" class="quiet-button">Refresh status</button></div>
|
|
144
|
+
<p>Checking authenticates with OpenAI without generating output. It does not prove Decisions access, available credit or decision quality. Saving a key does not enable decisions.</p>
|
|
145
|
+
<output id="openaiCredentialNotice" role="status" aria-live="polite"></output>
|
|
146
|
+
</div>
|
|
147
|
+
</section>
|
|
89
148
|
<section class="configuration-group autonomy-configuration" aria-labelledby="autonomyConfigurationTitle">
|
|
90
149
|
<header><div><h3 id="autonomyConfigurationTitle">Delegated delivery</h3><p>Choose an exact pool and limits. EWAI can progress only permitted work between your decisions.</p></div><strong id="autonomyMode" class="autonomy-mode">Checking…</strong></header>
|
|
91
150
|
<div class="autonomy-configuration-body">
|
|
@@ -844,6 +903,9 @@
|
|
|
844
903
|
</form>
|
|
845
904
|
</dialog>
|
|
846
905
|
<script src="/app.js" type="module"></script>
|
|
906
|
+
<script src="/jev.js" type="module"></script>
|
|
907
|
+
<script src="/jev-credentials.js" type="module"></script>
|
|
908
|
+
<script src="/openai-credentials.js" type="module"></script>
|
|
847
909
|
<script src="/coding-providers.js" type="module"></script>
|
|
848
910
|
<script src="/grok-credentials.js" type="module"></script>
|
|
849
911
|
</body>
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
const element=id=>document.getElementById(id),form=element('jevCredentialForm'),field=element('jevCredentialKey');
|
|
2
|
+
let saved=null,busy=false;
|
|
3
|
+
function controls(){
|
|
4
|
+
field.disabled=busy||!saved||!saved.storageSupported;
|
|
5
|
+
element('jevCredentialSave').disabled=field.disabled;
|
|
6
|
+
element('jevCredentialCancel').disabled=busy;
|
|
7
|
+
element('jevCredentialCheck').disabled=busy||!saved?.configured;
|
|
8
|
+
element('jevCredentialRemove').disabled=busy||!saved?.saved;
|
|
9
|
+
element('jevCredentialRefresh').disabled=busy;
|
|
10
|
+
}
|
|
11
|
+
function adopt(data){
|
|
12
|
+
saved=data;
|
|
13
|
+
element('jevCredentialStatus').textContent=data.source==='environment'?'Using TYPESAFE_API_KEY from the launching environment. '+(data.saved?'A saved key is also available.':'No saved key.'):
|
|
14
|
+
data.source==='saved'?'Saved Jev key available for this account.':'No Jev key configured. Recommended: check and save a key below.';
|
|
15
|
+
if(data.storageIssue)element('jevCredentialStatus').textContent='Using TYPESAFE_API_KEY from the launching environment. Saved storage needs repair; saved key availability is unknown.';
|
|
16
|
+
if(data.storageSupported===false)element('jevCredentialStatus').textContent+=' Local key storage is unavailable on this platform; use the launching environment.';
|
|
17
|
+
controls();
|
|
18
|
+
}
|
|
19
|
+
async function request(action='',body){
|
|
20
|
+
const response=await fetch('/api/jev/credentials'+(action?'/'+action:''),{method:action?'POST':'GET',headers:action?{'content-type':'application/json'}:undefined,body:action?JSON.stringify(body):undefined});
|
|
21
|
+
let data;try{data=await response.json();}catch{throw Error('Credential setup could not be reached. Recheck the local dashboard connection.');}
|
|
22
|
+
if(!response.ok)throw Error(data.error??'Credential setup failed. Refresh status and try again.');return data;
|
|
23
|
+
}
|
|
24
|
+
async function load(){
|
|
25
|
+
busy=true;controls();
|
|
26
|
+
try{adopt(await request());}catch{saved=null;element('jevCredentialNotice').textContent='Credential status could not be read safely. Repair private storage or use TYPESAFE_API_KEY in the launching environment, then refresh.';}
|
|
27
|
+
finally{busy=false;controls();}
|
|
28
|
+
}
|
|
29
|
+
async function action(name){
|
|
30
|
+
if(busy||!saved)return;
|
|
31
|
+
const body={confirmed:true,...(name==='check'?{}:{expectedRevision:saved.revision}),...(name==='configure'?{apiKey:field.value}:{})};
|
|
32
|
+
field.value='';busy=true;controls();element('jevCredentialNotice').textContent=name==='remove'?'Removing saved key…':'Checking TypeSafe authentication without generated output…';
|
|
33
|
+
try{const result=await request(name,body);adopt(result);element('jevCredentialNotice').textContent=result.message;}
|
|
34
|
+
catch(error){element('jevCredentialNotice').textContent=error.message+' Refresh status before trying again.';}
|
|
35
|
+
finally{body.apiKey=undefined;field.value='';busy=false;controls();}
|
|
36
|
+
}
|
|
37
|
+
form.addEventListener('submit',event=>{event.preventDefault();void action('configure');});
|
|
38
|
+
element('jevCredentialCancel').addEventListener('click',()=>{field.value='';element('jevCredentialNotice').textContent='Cancelled. Saved key retained.';});
|
|
39
|
+
element('jevCredentialCheck').addEventListener('click',()=>void action('check'));
|
|
40
|
+
element('jevCredentialRemove').addEventListener('click',()=>void action('remove'));
|
|
41
|
+
element('jevCredentialRefresh').addEventListener('click',()=>{field.value='';element('jevCredentialNotice').textContent='';void load();});
|
|
42
|
+
window.addEventListener('pagehide',()=>{field.value='';});
|
|
43
|
+
controls();void load();
|
package/public/jev.js
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
const el=id=>document.getElementById(id);
|
|
2
|
+
const labels={context:'Context ordering',personas:'Available persona recommendations',skill:'Skills and answer options',claim:'Claim support checks',failure:'Failure triage',impact:'Supplemental impact reviews',tests:'Supplemental test relevance'};
|
|
3
|
+
let saved=null,busy=false;
|
|
4
|
+
for(const [id,title]of Object.entries(labels)){const label=document.createElement('label'),input=document.createElement('input');input.type='checkbox';input.name='jevUseCase';input.value=id;label.append(input,document.createTextNode(title));el('jevUseCases').append(label);}
|
|
5
|
+
function controls(){el('jevControls').disabled=busy||!saved;el('jevSettingsSave').disabled=busy||!saved;el('jevRefresh').disabled=busy;}
|
|
6
|
+
async function request(path,body){const response=await fetch('/api/jev/'+path,{method:body?'POST':'GET',headers:body?{'content-type':'application/json'}:undefined,body:body?JSON.stringify(body):undefined});let result;try{result=await response.json();}catch{throw Error('Decision settings could not be reached.');}if(!response.ok)throw Error(result.error??'Refresh Decision settings before trying again.');return result;}
|
|
7
|
+
function adopt(value){saved=value;const p=value.policy;el('jevProvider').value=p.provider;el('openaiCloudConsent').checked=p.openaiConsent;document.querySelector(`input[name=jevMode][value=${p.mode}]`).checked=true;for(const input of document.querySelectorAll('input[name=jevUseCase]'))input.checked=p.useCases.includes(input.value);el('jevCloudConsent').checked=p.cloudConsent;el('jevMaxCalls').value=p.maxCalls;el('jevMaxTokens').value=p.maxInputTokens;el('jevTimeout').value=p.timeoutMs;el('jevConfidence').value=p.minConfidence;}
|
|
8
|
+
async function measurements(){const data=await request('measurements'),s=data.summary;const list=document.createElement('dl');for(const [label,value]of [['Calls in recent history',s.providerCalls],['Cache hits in recent history',s.cacheHits],['Fallbacks in recent history',s.fallbacks],['Total recorded input tokens',s.inputTokens??'Unknown'],['Estimated provider cost, USD',s.estimatedUSD===null?'Unknown':s.estimatedUSD.toFixed(6)],['Median call latency',s.medianMs===null?'Unknown':s.medianMs+' ms'],['P95 call latency',s.p95Ms===null?'Unknown':s.p95Ms+' ms'],['Downstream savings','Unknown — not measured']]){const dt=document.createElement('dt'),dd=document.createElement('dd');dt.textContent=label;dd.textContent=String(value);list.append(dt,dd);}const budget=document.createElement('p');budget.textContent=`Current policy: ${data.budget.callsUsed}/${data.budget.maxCalls} calls; ${data.budget.reservedTokens}/${data.budget.maxInputTokens} reserved input tokens.`;el('jevMeasurements').replaceChildren(budget,list);}
|
|
9
|
+
async function load(){busy=true;controls();try{adopt(await request('settings'));await measurements();el('jevNotice').textContent='Settings loaded. Recommended first enablement: shadow.';}catch(error){el('jevNotice').textContent=error.message;}finally{busy=false;controls();}}
|
|
10
|
+
el('jevSettingsForm').addEventListener('submit',async event=>{event.preventDefault();if(busy||!saved)return;const policy={mode:document.querySelector('input[name=jevMode]:checked').value,useCases:[...document.querySelectorAll('input[name=jevUseCase]:checked')].map(i=>i.value),provider:el('jevProvider').value,openaiConsent:el('openaiCloudConsent').checked,cloudConsent:el('jevCloudConsent').checked,maxCalls:Number(el('jevMaxCalls').value),maxInputTokens:Number(el('jevMaxTokens').value),timeoutMs:Number(el('jevTimeout').value),minConfidence:Number(el('jevConfidence').value)};busy=true;controls();el('jevNotice').textContent='Saving Decision settings…';try{adopt(await request('settings',{confirmed:true,expectedRevision:saved.revision,policy}));await measurements();el('jevNotice').textContent='Decision settings saved. Mode: '+policy.mode+'.';}catch(error){el('jevNotice').textContent=error.message+' Refresh to compare the latest saved policy.';}finally{busy=false;controls();}});
|
|
11
|
+
el('jevRefresh').addEventListener('click',()=>void load());controls();void load();
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
const element=id=>document.getElementById(id),form=element('openaiCredentialForm'),field=element('openaiCredentialKey');
|
|
2
|
+
let saved=null,busy=false;
|
|
3
|
+
function controls(){
|
|
4
|
+
field.disabled=busy||!saved||!saved.storageSupported;
|
|
5
|
+
element('openaiCredentialSave').disabled=field.disabled;
|
|
6
|
+
element('openaiCredentialCancel').disabled=busy;
|
|
7
|
+
element('openaiCredentialCheck').disabled=busy||!saved?.configured;
|
|
8
|
+
element('openaiCredentialRemove').disabled=busy||!saved?.saved;
|
|
9
|
+
element('openaiCredentialRefresh').disabled=busy;
|
|
10
|
+
}
|
|
11
|
+
function adopt(data){
|
|
12
|
+
saved=data;
|
|
13
|
+
element('openaiCredentialStatus').textContent=data.source==='environment'?'Using OPENAI_API_KEY from the launching environment. '+(data.saved?'A saved key is also available.':'No saved key.'):
|
|
14
|
+
data.source==='saved'?'Saved OpenAI key available for this account.':'No OpenAI key configured. Recommended: check and save a key below.';
|
|
15
|
+
if(data.storageIssue)element('openaiCredentialStatus').textContent='Using OPENAI_API_KEY from the launching environment. Saved storage needs repair; saved key availability is unknown.';
|
|
16
|
+
if(data.storageSupported===false)element('openaiCredentialStatus').textContent+=' Local key storage is unavailable on this platform; use the launching environment.';
|
|
17
|
+
controls();
|
|
18
|
+
}
|
|
19
|
+
async function request(action='',body){
|
|
20
|
+
const response=await fetch('/api/openai/credentials'+(action?'/'+action:''),{method:action?'POST':'GET',headers:action?{'content-type':'application/json'}:undefined,body:action?JSON.stringify(body):undefined});
|
|
21
|
+
let data;try{data=await response.json();}catch{throw Error('Credential setup could not be reached. Recheck the local dashboard connection.');}
|
|
22
|
+
if(!response.ok)throw Error(data.error??'Credential setup failed. Refresh status and try again.');return data;
|
|
23
|
+
}
|
|
24
|
+
async function load(){
|
|
25
|
+
busy=true;controls();
|
|
26
|
+
try{adopt(await request());}catch{saved=null;element('openaiCredentialNotice').textContent='Credential status could not be read safely. Repair private storage or use OPENAI_API_KEY in the launching environment, then refresh.';}
|
|
27
|
+
finally{busy=false;controls();}
|
|
28
|
+
}
|
|
29
|
+
async function action(name){
|
|
30
|
+
if(busy||!saved)return;
|
|
31
|
+
const body={confirmed:true,...(name==='check'?{}:{expectedRevision:saved.revision}),...(name==='configure'?{apiKey:field.value}:{})};
|
|
32
|
+
field.value='';busy=true;controls();element('openaiCredentialNotice').textContent=name==='remove'?'Removing saved key…':'Checking OpenAI authentication without generated output…';
|
|
33
|
+
try{const result=await request(name,body);adopt(result);element('openaiCredentialNotice').textContent=result.message;}
|
|
34
|
+
catch(error){element('openaiCredentialNotice').textContent=error.message+' Refresh status before trying again.';}
|
|
35
|
+
finally{body.apiKey=undefined;field.value='';busy=false;controls();}
|
|
36
|
+
}
|
|
37
|
+
form.addEventListener('submit',event=>{event.preventDefault();void action('configure');});
|
|
38
|
+
element('openaiCredentialCancel').addEventListener('click',()=>{field.value='';element('openaiCredentialNotice').textContent='Cancelled. Saved key retained.';});
|
|
39
|
+
element('openaiCredentialCheck').addEventListener('click',()=>void action('check'));
|
|
40
|
+
element('openaiCredentialRemove').addEventListener('click',()=>void action('remove'));
|
|
41
|
+
element('openaiCredentialRefresh').addEventListener('click',()=>{field.value='';element('openaiCredentialNotice').textContent='';void load();});
|
|
42
|
+
window.addEventListener('pagehide',()=>{field.value='';});
|
|
43
|
+
controls();void load();
|
package/public/styles.css
CHANGED
|
@@ -19,6 +19,26 @@
|
|
|
19
19
|
--shadow: 0 12px 32px rgb(23 33 28 / 12%);
|
|
20
20
|
}
|
|
21
21
|
|
|
22
|
+
.jev-body { display: grid; gap: 24px; padding: 20px; }
|
|
23
|
+
.jev-body form, .jev-body fieldset { display: grid; gap: 14px; min-width: 0; }
|
|
24
|
+
.jev-body fieldset { margin: 0; padding: 14px; border: 1px solid var(--line); border-radius: 6px; }
|
|
25
|
+
.jev-body legend, .jev-body h4 { font-weight: 700; }
|
|
26
|
+
.jev-body p { color: var(--muted); line-height: 1.5; }
|
|
27
|
+
.jev-beta { font-size: 11px; color: var(--green); background: var(--green-soft); padding: 4px 7px; border-radius: 6px; }
|
|
28
|
+
.jev-modes { display: grid; gap: 12px; }
|
|
29
|
+
.jev-modes label { display: grid; grid-template-columns: 20px minmax(0, 1fr); align-items: center; min-height: 44px; }
|
|
30
|
+
.jev-modes small { grid-column: 2; color: var(--muted); }
|
|
31
|
+
.jev-use-cases, .jev-limits { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 12px; }
|
|
32
|
+
.jev-use-cases label, .jev-disclosure label { display: flex; align-items: center; gap: 8px; min-height: 44px; }
|
|
33
|
+
.jev-limits label { display: grid; gap: 7px; }
|
|
34
|
+
.jev-limits input { width: 100%; min-height: 42px; padding: 8px; border: 1px solid var(--line-strong); border-radius: 6px; }
|
|
35
|
+
.jev-body input { accent-color: var(--green); }
|
|
36
|
+
.jev-disclosure { padding: 14px; background: var(--surface-2); border: 1px solid var(--line); border-radius: 6px; }
|
|
37
|
+
#jevMeasurements dl { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 1fr); gap: 10px 18px; }
|
|
38
|
+
#jevMeasurements dd { margin: 0; overflow-wrap: anywhere; font-variant-numeric: tabular-nums; }
|
|
39
|
+
.jev-body button:focus-visible, .jev-body input:focus-visible { outline: 2px solid var(--green); outline-offset: 2px; }
|
|
40
|
+
@media (max-width: 600px) { .jev-body { padding: 14px; } .jev-use-cases, .jev-limits { grid-template-columns: minmax(0, 1fr); } }
|
|
41
|
+
|
|
22
42
|
* { box-sizing: border-box; }
|
|
23
43
|
html, body { width: 100%; height: 100%; }
|
|
24
44
|
body {
|
|
@@ -2776,7 +2796,11 @@ button:focus-visible, input:focus-visible, select:focus-visible, textarea:focus-
|
|
|
2776
2796
|
.autonomy-button-row button { width: 100%; }
|
|
2777
2797
|
.autonomy-dialog .action-sheet > footer { flex-wrap: wrap; }
|
|
2778
2798
|
}
|
|
2779
|
-
.grok-credentials-body { padding: 18px; display: grid; gap: 14px; }
|
|
2780
|
-
.grok-credentials-body form { display: grid; gap: 12px; }
|
|
2781
|
-
.grok-credentials-body input { width: 100%; box-sizing: border-box; min-height: 40px; border: 1px solid var(--line, #cfd8d5); border-radius: 6px; padding: 8px 10px; }
|
|
2782
|
-
.grok-credentials-body p { margin: 0; }
|
|
2799
|
+
:is(.grok-credentials-body, .jev-credentials-body, .openai-credentials-body) { padding: 18px; display: grid; gap: 14px; }
|
|
2800
|
+
:is(.grok-credentials-body, .jev-credentials-body, .openai-credentials-body) form { display: grid; gap: 12px; }
|
|
2801
|
+
:is(.grok-credentials-body, .jev-credentials-body, .openai-credentials-body) input { width: 100%; box-sizing: border-box; min-height: 40px; border: 1px solid var(--line, #cfd8d5); border-radius: 6px; padding: 8px 10px; }
|
|
2802
|
+
:is(.grok-credentials-body, .jev-credentials-body, .openai-credentials-body) p { margin: 0; }
|
|
2803
|
+
|
|
2804
|
+
.jev-provider-label { display: grid; gap: 7px; }
|
|
2805
|
+
#jevProvider { width: 100%; min-width: 0; min-height: 44px; box-sizing: border-box; padding: 8px; border: 1px solid var(--line-strong); border-radius: 6px; background: var(--surface); color: var(--ink); }
|
|
2806
|
+
#jevProvider:focus-visible { outline: 2px solid var(--green); outline-offset: 2px; }
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
import {readFileSync} from 'node:fs';
|
|
2
|
+
import {decideWithJev} from '../src/runtime/jev-operations.mjs';
|
|
3
|
+
import {readJevMeasurements} from '../src/jev.mjs';
|
|
4
|
+
|
|
5
|
+
// Import-safe runner: the CLI owns policy and key capture. This never enables
|
|
6
|
+
// Jev, widens budgets, captures credentials or uploads project evidence.
|
|
7
|
+
export async function runJevExperiment(root,options={}){
|
|
8
|
+
const limit=options.limit??6;
|
|
9
|
+
if(!Number.isInteger(limit)||limit<1||limit>6){const e=Error('Choose an experiment limit from 1 to 6.');e.code='jev-input';throw e;}
|
|
10
|
+
const fixtures=JSON.parse(readFileSync(new URL('../tests/fixtures/jev-synthetic.json',import.meta.url),'utf8')).slice(0,limit),results=[];
|
|
11
|
+
const before=readJevMeasurements(root);
|
|
12
|
+
for(const fixture of fixtures){const result=await decideWithJev(root,fixture.input,options);results.push({id:fixture.id,expected:fixture.expected,actual:result.decision,status:result.status,reason:result.reason,correct:result.status==='suggested'?result.decision===fixture.expected:null,validation:result.validation,measurement:result.measurement??null});}
|
|
13
|
+
const after=readJevMeasurements(root);
|
|
14
|
+
return {schema:'ewai.jev-experiment/v1',dataset:'six synthetic challenge cases; not representative coding evaluation',results,summary:{evaluated:results.filter(r=>r.correct!==null).length,correct:results.filter(r=>r.correct===true).length,providerCalls:results.filter(r=>r.measurement?.providerCall).length,inputTokens:results.every(r=>r.measurement?.inputTokens!==null)?results.reduce((sum,r)=>sum+(r.measurement?.inputTokens??0),0):null,downstreamSavings:null},before:before.summary,after:after.summary};
|
|
15
|
+
}
|