@stage5/lumine 0.2.82 → 0.2.84
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/lib/constants.js +5 -0
- package/package.json +1 -1
- package/sdk/BUILD_SDK_INDEX.md +32 -7
- package/sdk/LUMINE_ADMIN.md +122 -10
package/lib/constants.js
CHANGED
|
@@ -176,6 +176,11 @@ Lumine CLI as the source of truth for saving this workspace back to Twinkle.
|
|
|
176
176
|
|
|
177
177
|
## Workflow
|
|
178
178
|
|
|
179
|
+
- When planning a new app or a substantial improvement, read the AI decision design
|
|
180
|
+
guidance in ${SDK_REFERENCE_FILE}. Consider whether JEV through Twinkle.ai.decide
|
|
181
|
+
would improve the requested experience, and choose ordinary code or a text model
|
|
182
|
+
when those fit better. Make this judgment from the app's needs; do not wait for
|
|
183
|
+
the creator to know the technology or add it automatically to every app.
|
|
179
184
|
- Edit only project files in this workspace.
|
|
180
185
|
- Keep /index.html or /index.htm as the entry file.
|
|
181
186
|
- Before editing an existing project, confirm \`.twinkle/lumine-project.json\`
|
package/package.json
CHANGED
package/sdk/BUILD_SDK_INDEX.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# Build SDK Index
|
|
2
2
|
|
|
3
|
-
Version: 1.
|
|
4
|
-
Updated: 2026-09-
|
|
5
|
-
Generated: 2026-09-
|
|
3
|
+
Version: 1.47.0
|
|
4
|
+
Updated: 2026-09-21
|
|
5
|
+
Generated: 2026-09-21T08:06:52.743Z
|
|
6
6
|
|
|
7
7
|
## Notes
|
|
8
8
|
- This SDK is injected into Build iframes via the Build preview/runtime.
|
|
@@ -36,7 +36,15 @@ Generated: 2026-09-19T10:34:03.685Z
|
|
|
36
36
|
- The creator's agent designs the rewards. Declare the economy in a project file `rewards.json` at the root: budgets (userDailyXP, userDailyCoins, optional userDailyClaims; there is no app-wide daily or lifetime budget, only what one learner can earn per day) and rules [{ id, title, xp, coins, verifier: 'numeric-quiz' | 'completion', maxAttempts?, retry?: { xpPercent, coinsPercent, paidAttempts? }, minSeconds? (completion), progression?: 'dated' | 'until-earned' (quiz) }]. Wire the matching Twinkle.rewards calls with those literal rule ids. Questions and answer keys NEVER go in project files (published source is readable by every player): quiz rules get them from the private question sheet uploaded with `lumine rewards sheet <file.json>` ({ rules: { <ruleId>: { questions?, sets? } } }); `lumine rewards check` validates both together. A review request freezes the code and proposes rewards.json merged with the sheet; the administrator reads the code, checks the amounts and whether the app is exploitable, may change any amount, and approves. Creators are kids and teens: show approval status and one Send for review action; do not ask them to fill in technical forms. Every code update that retains rewards needs a new approval before publishing. Removing the SDK automatically clears its gate. Apps read amounts, tries and sets from getStatus, never from their own file.
|
|
37
37
|
- Verifiers: 'numeric-quiz' pays for server-checked numeric answers (retry share, attempt limits, dated sets or until-earned sets that stay up until somebody earns them, after-answer guides). 'completion' pays when the app reports an activity finished — a cleared stage, a finished round — at least minSeconds after start({ ruleId }); the server checks only the elapsed time, once per learner per site day (UTC midnight), and the budgets. Call start when the activity begins and claim({ challengeId }) with no answers when it ends; keep completion amounts and userDailyXP small enough that a player scripting the calls would not matter, because nothing else is verified.
|
|
38
38
|
- Numeric quiz answers are verified on the server; client scores, privateDb state, timers and completion booleans are not verified reward evidence. Limits reset at UTC midnight. Rules are earned once per viewer per UTC day; attempt limits and retry payouts come from the approved rule. Challenges expire at the UTC day boundary. Budgets apply across release changes.
|
|
39
|
-
- Optional reward rule controls: maxLifetimeClaims caps one learner’s receipts for that rule across every day and release; completionProof: classic-tower-v1 requires a server-simulated Classic Tower finish in addition to minSeconds. These are server-enforced controls. Existing completion rules without completionProof still verify elapsed time only.
|
|
39
|
+
- Optional reward rule controls: maxLifetimeClaims caps one learner’s receipts for that rule across every day and release; completionProof: classic-tower-v1 requires a server-simulated Classic Tower finish in addition to minSeconds. These are server-enforced controls. Existing completion rules without completionProof still verify elapsed time only. Registered proof profiles also include breadface-v1 (server-simulated Breadface inputs) and study-record-v1 (a private study record reviewed by JEV, billed to the learner’s AI Energy). Profiles are platform-owned; an app cannot invent a verifier or authorize its own reward.
|
|
40
|
+
|
|
41
|
+
## AI decision design
|
|
42
|
+
- When planning a new app or an improvement, consider whether model-based judgments would materially improve the requested experience. Twinkle.ai.decide runs JEV, a model for narrow decisions over supplied text or structured state. Potential uses include interpreting a player's request to an NPC, choosing among legal game actions, classifying user content, ranking supplied candidates, and adapting an activity to evidence about the learner. These are examples to reason from, not a keyword checklist or a requirement to add AI to every app.
|
|
43
|
+
- Choose the mechanism by the work: use ordinary code for known rules, arithmetic, physics, legal moves, storage, and permissions; use ai.decide for contextual choices, yes/no probabilities, and scores; use ai.chat or ai.generateObject for generated dialogue, explanations, open-ended content, and multi-step reasoning. JEV does not generate text, browse the web, inspect images, or supply missing app data. Combine these tools when it benefits the actual request, without silently turning every decision into a more expensive text-model call.
|
|
44
|
+
- Before adopting JEV, identify the concrete decision, the context available at that moment, the allowed outcomes, and why latency plus AI Energy use are worthwhile. Use your model judgment to make that choice. Do not substitute regex or keyword matching for understanding human-language intent. Explain the user-visible benefit simply; the creator does not need to select a provider, enter an API key, or design question schemas.
|
|
45
|
+
- Send related independent questions about the same state together in one ai.decide request (up to 32). Write complete instructions and distinct option or level descriptions. Include an other/uncertain option when the choices might not cover the input. A second call is useful only when it needs newly obtained context or choices that depend on the first result; answers in one batch do not see each other.
|
|
46
|
+
- Keep code in control. Choice returns an option and its probability distribution; score returns a position on the declared levels; noul is the probability of yes, with 0.5 meaning uncertainty rather than medium strength. Confidence describes the distribution and is not proof of correctness. Test the app's criteria and thresholds on clear, ambiguous, and out-of-scope examples, and provide a suitable uncertain outcome. JEV decisions never authorize XP, Coins, purchases, access, or other server-owned state; use the appropriate existing SDK and server approval paths.
|
|
47
|
+
- Call at meaningful user actions or discrete app decision points, not every render, animation frame, keystroke, or background poll. Batch work, debounce changing input, keep one request active per decision flow, and discard results whose app state changed while waiting. Keep rendering, movement, and controls local and responsive. Signed-in viewers use AI Energy based on measured JEV cost. Handle unavailable, timeout, exhausted Energy, and rate-limit errors with a clear retry or an explicitly local fallback; never fabricate an AI answer or run an automatic retry loop. Consider the guest and offline experience before making AI essential.
|
|
40
48
|
|
|
41
49
|
## Token Scopes
|
|
42
50
|
files:read, media:read, media:write, live:read, live:write, user:read, users:read, dailyReflections:read, content:read, content:write, sharedDb:read, sharedDb:write, privateDb:read, privateDb:write, files:write, chat:read, chat:write, notifications:read, notifications:write, notifications:emit, reminders:read, reminders:write, rewards:claim
|
|
@@ -467,6 +475,19 @@ renderBattery(policy?.energyPercent, policy?.energySegmentsRemaining);
|
|
|
467
475
|
- Use this for in-app AI replies instead of creating or fetching app-local endpoints such as /api/chat.
|
|
468
476
|
- Example: const chatHistory = conversation.slice(-12).map((entry) => ({ role: entry.role === 'assistant' ? 'assistant' : 'user', content: entry.text }));
|
|
469
477
|
const result = await Twinkle.ai.chat({ message, history: chatHistory, systemPrompt: 'You are a cheerful pirate helper who answers in one sentence.', onText: (text, meta) => renderReply(text), onStatus: (status) => setThinking(status === 'thinking') });
|
|
478
|
+
- async decide({ state, questions }) | scopes: none
|
|
479
|
+
- Returns: { answers, model, provider, aiUsagePolicy }
|
|
480
|
+
- Use JEV for fast contextual choices, yes/no probabilities, and scores inside a running app. Send several independent questions about the same state together.
|
|
481
|
+
- Read the AI decision design guidance when deciding whether this capability benefits the app. This method evaluates supplied context; it does not generate prose, browse, inspect images, run tools, grant access, or award XP/Coins.
|
|
482
|
+
- state is non-empty text, a JSON object, or a JSON array. questions maps your question IDs to { type, instructions, criteria? }. Instructions may be non-empty text or structured JSON. Every question sees the same state and answers independently; put the complete question in instructions, not just its ID.
|
|
483
|
+
- Choice: { type: 'choice', instructions, criteria: { optionName: description, other: description } }. Supply 2–255 named options. Each description is text, structured JSON, or null. Returns { type: 'choice', choice, probabilities, confidence } under answers[questionId].
|
|
484
|
+
- Score: { type: 'score', instructions, criteria: [lowestLevelDescription, ..., highestLevelDescription] }. Supply 2–10 concrete ordered descriptions. Returns { type: 'score', score, probabilities, confidence, legend }. Levels start at 0; score may be fractional and is a probability-weighted position between 0 and criteria.length - 1.
|
|
485
|
+
- Yes/no probability: { type: 'noul', instructions, criteria?: { true: description, false: description } }. Returns { type: 'noul', noul }, where noul is the probability of yes between 0 and 1. There is no separate confidence field. If criteria is supplied, describe both true and false.
|
|
486
|
+
- A request accepts 1–32 questions, at most 64 KB of JSON and 20 levels of nesting. Question IDs and option names must be non-empty, at most 128 characters, and cannot be __proto__, constructor, or prototype. Descriptions may use structured JSON; state must contain only data the app is allowed to access.
|
|
487
|
+
- Requires a signed-in viewer and uses the normal runtime AI rate limits. Twinkle selects and calls JEV on its server; never put provider credentials or an external endpoint in app code. Each successful decision settles measured JEV usage through the existing AI Energy policy and returns the canonical aiUsagePolicy. Empty Energy blocks provider work. Authorized App MCP invocations retain the existing system-covered billing policy.
|
|
488
|
+
- Errors include invalid_ai_decision (bad input), ai_decision_unavailable, ai_decision_timeout, ai_decision_rate_limited, ai_decision_invalid_response, and ai_usage_unavailable, plus the existing auth, access, Energy, and rate-limit errors. Invalid answers are rejected. There is no automatic provider retry, fabricated answer, or paid LLM fallback. Do not automatically retry a rejected request.
|
|
489
|
+
- Call on meaningful app events with bounded frequency, batch questions, and keep rendering and deterministic game rules in local code. Ignore a result if the app state or turn changed while it was pending. Test uncertain inputs and choose a fallback appropriate to the experience; confidence is not a correctness guarantee.
|
|
490
|
+
- Example: const { answers } = await Twinkle.ai.decide({ state: { playerRequest, availableActions }, questions: { action: { type: 'choice', instructions: 'Which available companion action best fits playerRequest? Use wait when unclear.', criteria: { follow: 'Follow the player', guard: 'Stay and keep watch', wait: 'Do nothing until clarified' } }, needsClarification: { type: 'noul', instructions: 'Is playerRequest too ambiguous to act on?' } } }); const action = answers.needsClarification.noul > 0.5 ? 'wait' : answers.action.choice;
|
|
470
491
|
- async generateObject({ prompt, expectedStructure, thinkingMode, mode, model, instructions, systemPrompt, webSearch, requestId, onText, onStatus, onReasoning } = {}) | scopes: none
|
|
471
492
|
- Returns: { object, result, model, provider, thinkingMode, requestedThinkingMode, requestedModel, webSearch, aiUsagePolicy }
|
|
472
493
|
- Generate a validated structured JSON object for app decisions, routing, grading, and game-state logic, with optional live output/status callbacks and web search.
|
|
@@ -1023,18 +1044,21 @@ world.updatePresence({ x, y, z, facing });
|
|
|
1023
1044
|
- Returns: { mode: "live", status: "awarded" | "pending" | "expired" | "not_found", receipt: { id, challengeId, ruleId, reviewId, artifactVersionId, dayKey, xp, coins, attempt, createdAt } | null, balances: { xp, coins } } | { mode: "preview", status: "not_found", receipt: null, message }
|
|
1024
1045
|
- Read an existing receipt for this app and signed-in viewer by server-issued challengeId, including previous UTC days and previous approved versions. Requires the current approved published release and runtime grant; a stale frame must reload first. Never awards, retries a claim, returns answer keys, or restores removed rewards permission.
|
|
1025
1046
|
- Reconcile a durable local reward outbox after a lost claim reply: awarded confirms the exact payment; pending means no receipt yet for a current unexpired challenge, so retry the same challengeId. expired or not_found confirms no paid receipt and no claim possible for that ID under the current release. Never refund app items just because getStatus history omitted an older claim or a network request failed. Preview has no durable paid receipts; keep it separate from live recovery.
|
|
1026
|
-
- await Twinkle.rewards.start({ ruleId }) | scopes: rewards:claim
|
|
1047
|
+
- await Twinkle.rewards.start({ ruleId, levelIndex? }) | scopes: rewards:claim
|
|
1027
1048
|
- Returns: { mode: "live", challengeId, questions: [{ prompt, hint?, guide? }], setKey, reward: { xp, coins }, retryReward: { xp, coins }, attempts, maxAttempts, attemptsRemaining, firstTryAvailable, expiresAt }
|
|
1028
1049
|
- Creates or resumes a server-issued challenge for the signed-in viewer. Render its questions (prompt and optional hint) and collect numeric answers in the same order. One daily challenge per rule/review; repeat starts cannot reset attempts. A challenge stays open until the site's daily reset (UTC midnight, 9:00 AM in Korea) (expiresAt). Resuming after a wrong answer includes each question's guide.
|
|
1029
1050
|
- Errors: build_reward_not_scheduled when the rule has no questions for today; build_reward_daily_claims_reached when the viewer already earned today’s cap. attemptsRemaining is null for unlimited rules.
|
|
1030
1051
|
- For a completion rule call start when the activity begins (the moment the stage starts); the challenge's age is what the claim is measured against. In preview mode start also works for the owner (a stateless simulation).
|
|
1031
1052
|
- For completionProof: classic-tower-v1, start also returns completion { profile, token, maxFrames, completed, failed }. A new start resets only the simulated climb to its canonical spawn; it cannot reset daily or lifetime rewards. Record inputs from the first physics frame. The completion token is bound to the viewer, challenge, rule and published release.
|
|
1032
|
-
-
|
|
1033
|
-
|
|
1053
|
+
- For breadface-v1, pass the zero-based canonical levelIndex. Record [dt, inputBits] from the first physics frame; start returns its server token and maxFrames. For study-record-v1, start returns completion { profile, usesAiEnergy: true }; there is no client-authored proof token.
|
|
1054
|
+
- await Twinkle.rewards.progress({ challengeId, completionToken?, frames?, record?, requestId? }) | scopes: rewards:claim
|
|
1055
|
+
- Returns: { mode: "live" | "preview", completion: { profile, token?, completed, failed?, decision?, message?, maxFrames? }, aiUsagePolicy? }
|
|
1034
1056
|
- Verify a bounded batch of inputs for an approved server-simulated climb.
|
|
1035
1057
|
- Only for completionProof: classic-tower-v1. Send 1 to 600 chronological physics frames, each [dt, moveX, moveY, cameraForwardX, cameraForwardZ, jumpPressed, jumpHeld, speedMultiplier]. dt is in seconds, at most 0.05; movement axes are -1 through 1; the camera values are the horizontal components before normalization; jump flags are 0 or 1; speedMultiplier is an existing Classic Tower trail speed (1, 1.1, 1.2, 1.3, 1.35 or 1.4). Geometry and player state are owned by the registered server simulation.
|
|
1036
1058
|
- Send occasional batches with at most one request in flight. Keep the previous token and the exact batch until a response confirms it; an identical retry is safe. Use the returned token for the next batch. Simulation time cannot outrun wall time. A completed token is evidence of a legal simulated run, not proof that a human played or that inputs were not automated.
|
|
1037
1059
|
- On respawn or a return from another world, begin a fresh run at the canonical spawn via start. Preserve other worlds and gameplay. Do not submit positions, scores, secret keys, or a client completion flag. Preview tokens can never be redeemed in the published app.
|
|
1060
|
+
- For breadface-v1, send 1–1,000 chronological [dt, inputBits] frames (dt > 0 and <= 0.033). Bits are left=1, right=2, jumpHeld=4, jumpQueued=8, fireHeld=16. The server replays the registered frozen game; only a legitimate goal and any rule-specific bonus qualify. Honor maxFrames and retain unacknowledged inputs for a bounded retry.
|
|
1061
|
+
- For study-record-v1, send record { work, learning, nextStep? } and one stable requestId (16–64 ASCII letters, numbers, hyphens or underscores, e.g. crypto.randomUUID()). The combined record is at most 4,000 UTF-8 bytes. Show that each check uses AI Energy before sending; accept, revise and uncertain decisions all incur measured check cost. Reuse the requestId for a lost response; do not poll in a loop. An accepted, billed review returns completion.completed and a bound token; use that token with claim. A preview charges for the check but never awards real XP/Coins. Keep the record and show completion.message on a refusal or uncertain result.
|
|
1038
1062
|
- await Twinkle.rewards.claim({ challengeId, answers?: [number], completionToken?: string }) | scopes: rewards:claim
|
|
1039
1063
|
- Returns: { awarded: false, attempts, attemptsRemaining, questions: [{ prompt, hint?, guide? }] } | { awarded: true, duplicate, receipt: { ruleId, xp, coins, attempt, firstTry }, questions: [{ prompt, hint?, guide? }], balances: { xp, coins } }
|
|
1040
1064
|
- Twinkle verifies every answer, approval, current published artifact and budget before atomically recording XP and Coins. The receipt’s xp/coins are what was actually paid: the full amounts on a first try, the retry share after a wrong answer (attempt > 1). Retry the same challengeId after a lost response; a confirmed claim returns its original receipt without another award. Never update balance UI optimistically. Under retry.paidAttempts a correct answer past that attempt returns awarded: true with a zero receipt: solved, not paid.
|
|
@@ -1042,6 +1066,7 @@ world.updatePresence({ x, y, z, facing });
|
|
|
1042
1066
|
- A wrong answer within two seconds of the previous one is refused with build_reward_throttled (HTTP 429) and does not count; wait for the person to try again rather than retry-looping.
|
|
1043
1067
|
- Completion rules take no answers: claim({ challengeId }) when the activity is finished. build_reward_too_fast (HTTP 409) means fewer than minSeconds passed since start; show nothing and let play continue. In preview mode the receipt carries preview: true and nothing is paid.
|
|
1044
1068
|
- A completionProof rule also requires the signed completionToken from a successful rewards.progress response. The server simulates the registered game physics and must reach the goal. A timer, forged position, client win flag, altered inventory or token from another viewer, challenge or release cannot authorize payment. maxLifetimeClaims is enforced from receipts in the same award transaction. build_reward_lifetime_claims_reached means all rewards for this rule have been collected; do not retry it.
|
|
1069
|
+
- For study-record-v1, the server verifies the settled private review row and its viewer, app, exact release, challenge and day binding. It never trusts the client’s decision, requested award amount or AI answer. Each successful daily study record is claimable once; display only canonical receipt/balances.
|
|
1045
1070
|
- await Twinkle.rewards.getLeaderboard({ metric?: "xp" | "coins", period?: "day" | "week" | "all", limit? }) | scopes: rewards:claim
|
|
1046
1071
|
- Returns: { mode: "live", metric, period, limit, dayKey, from, available: { xp, coins }, entries: [{ rank, userId, username, profilePicUrl, xp, coins, claims, lastAt }], me: { rank, xp, coins, claims } | null } | { mode: "preview", metric, period, available, entries: [], me: null, message }
|
|
1047
1072
|
- Standings of who earned the most XP or Coins in THIS app, computed by Twinkle from its own receipts (never from anything the app submits). period 'day' is today (site day, UTC), 'week' the last 7 site days, 'all' (default) every day since approval. limit defaults to 20, max 100.
|
package/sdk/LUMINE_ADMIN.md
CHANGED
|
@@ -2752,8 +2752,10 @@ primary review does not cover the target. An open review does not block a
|
|
|
2752
2752
|
deployment or host hold. Its files, lease and database boundaries persist;
|
|
2753
2753
|
active requests use the normal drain. A held or unavailable owner returns a
|
|
2754
2754
|
retryable failure, so keep the session and retry when that host is available
|
|
2755
|
-
again. Release operators
|
|
2756
|
-
|
|
2755
|
+
again. Release operators review final shutdown deltas through the deployment
|
|
2756
|
+
workflow's private SSM/S3 snapshots (or interactive management access during
|
|
2757
|
+
explicit recovery) and record their evidence. Raw logs never belong in GitHub
|
|
2758
|
+
output. An active review keeps ownership of clearing;
|
|
2757
2759
|
otherwise API stderr is cleared with the existing guarded
|
|
2758
2760
|
`npm run logs:clear-errors` plus post-clear re-read. A stopped target whose final logs
|
|
2759
2761
|
were reviewed does not need to be started for daily management; starting EC2
|
|
@@ -2995,7 +2997,7 @@ Mikey"** section carrying only
|
|
|
2995
2997
|
the deltas and anomalies worth his time, next to the escalation list. Never
|
|
2996
2998
|
dump raw sections at him.
|
|
2997
2999
|
|
|
2998
|
-
### Jev
|
|
3000
|
+
### Jev serving and audits (standing duty, every full daily review; updated 2026-09-20)
|
|
2999
3001
|
|
|
3000
3002
|
Read `data.jevPilot` from `lumine admin brief --json` and carry it into
|
|
3001
3003
|
the full report for Mikey. The active `daily-run report --json` also includes
|
|
@@ -3009,17 +3011,64 @@ seven completed days, and keep the in-progress day separate. Include paired
|
|
|
3009
3011
|
decision counts, disagreements (especially Jev react / baseline respond),
|
|
3010
3012
|
p50/p95 latency for each model, provider errors/timeouts, pending observations,
|
|
3011
3013
|
known incremental cost, unknown-cost requests, ledger gaps and cap status.
|
|
3012
|
-
|
|
3013
|
-
|
|
3014
|
-
|
|
3014
|
+
Separate `bySurface.comment` from `bySurface.chat`. For chat, report
|
|
3015
|
+
`routingFieldDisagreements`, `candidateSkippedBaselineRequired`, and missing
|
|
3016
|
+
routing-comparison evidence. Equal reply actions do not prove equal tool/history
|
|
3017
|
+
routing. Chat's baseline also extracts structured plans, while Jev compares seven
|
|
3018
|
+
routing choices plus reaction emoji; these latencies do not establish an end-to-end speedup.
|
|
3019
|
+
Mikey authorized production serving on September 20 for the tested comment and
|
|
3020
|
+
text-chat routing decisions. Include `serving.jevDecisions`, baseline and
|
|
3021
|
+
unreserved fallbacks with reasons, audit coverage, served disagreements,
|
|
3022
|
+
`serving.decisionLatencyMs`, chat added wait, and comment baseline calls avoided.
|
|
3023
|
+
Chat retains the existing full planner for outputs outside Jev's tested scope;
|
|
3024
|
+
comments run a 5% independent background baseline audit. Distinguish actual
|
|
3025
|
+
selected routes from unused comparisons and identify Turtle's deployment tests.
|
|
3026
|
+
Chat's `baseline_requires_reply` fallback preserves the planner's written reply
|
|
3027
|
+
when Jev would only react; report its frequency and review those disagreements.
|
|
3028
|
+
Reaction-only chat responses require both models to agree.
|
|
3029
|
+
Jev chooses the emoji from all 18 supported reactions in `chat-routing-v2`.
|
|
3030
|
+
Report `reactionChoices` usage by source, paired emoji disagreements, and missing
|
|
3031
|
+
legacy evidence; review whether the chosen tone fits the canonical conversation.
|
|
3032
|
+
Candidate and served emoji are in `reviewCandidates` routing objects. Earlier
|
|
3033
|
+
`chat-routing-v1` rows have no emoji comparison and must not count as agreement.
|
|
3034
|
+
Mikey explicitly requested all eligible requests with no daily request cap
|
|
3035
|
+
(`dailyLimit: 0`) and a cost report during every full website-management run.
|
|
3036
|
+
Run `lumine admin ai-costs day YYYY-MM-DD --json` for the last completed UTC day.
|
|
3037
|
+
Report the canonical `data.dailyAiCosts.byOperation` USD totals for `jev_reply_gate_serve`,
|
|
3038
|
+
`jev_chat_routing_serve`, any `_shadow` operations, and `jev_reply_gate_audit`.
|
|
3039
|
+
Separate comment/chat provider spend from background baseline-audit spend.
|
|
3040
|
+
Report unfinished selection/baseline-audit telemetry; synthetic probes are
|
|
3041
|
+
excluded from performance metrics but included in daily request and cost counts.
|
|
3042
|
+
If `telemetryStatus: partial` or `telemetryComplete: false`, detail metrics are a
|
|
3043
|
+
bounded recent sample, not full-day performance or cost; use the canonical daily
|
|
3044
|
+
AI-cost report for complete cost totals and record the coverage gap.
|
|
3045
|
+
Known Jev spend and `jev_reply_gate_audit` calls are already in application AI
|
|
3046
|
+
costs: never add them again or infer net savings from Jev cost alone. Agreement
|
|
3047
|
+
is not accuracy; confidence is not a measured success rate.
|
|
3015
3048
|
|
|
3016
3049
|
Privately inspect the bounded `reviewCandidates` when needed, name what was
|
|
3017
|
-
actually reviewed, and account for edited comments.
|
|
3050
|
+
actually reviewed, and account for edited comments or chat messages. Use the
|
|
3051
|
+
candidate's surface and target ID to find the correct canonical record. Ordinary model disagreements
|
|
3018
3052
|
are evaluation findings; outages, stuck telemetry or missing ledger entries are
|
|
3019
3053
|
operational findings. Report a recommendation to continue, adjust or stop, without
|
|
3020
|
-
automatically
|
|
3021
|
-
|
|
3022
|
-
|
|
3054
|
+
automatically changing mode, scope or caps.
|
|
3055
|
+
|
|
3056
|
+
Mikey's September 21 reporting requirement: include a private case-by-case
|
|
3057
|
+
comparison for the reviewed disagreements. For each event give the minimum
|
|
3058
|
+
relevant excerpt/context, field name and plain-language meaning, exact baseline
|
|
3059
|
+
and JEV values, selected value/source and fallback, observed outcome, and your
|
|
3060
|
+
assessment with evidence. Explicitly allow “both defensible” or “insufficient
|
|
3061
|
+
evidence”; the existing LLM is not ground truth. State reviewed/total coverage and
|
|
3062
|
+
rubric version. A bad delivered answer does not establish which routing choice
|
|
3063
|
+
caused it, and the alternative model's answer was not necessarily generated.
|
|
3064
|
+
`requiresPreviousMessages` means additional retrieval beyond supplied recent
|
|
3065
|
+
context; `requiresMathVerification` also covers answer/tutoring verification in
|
|
3066
|
+
non-math subjects. Historical v1/v2 telemetry records both decisions and source
|
|
3067
|
+
IDs, not original conversation text, model explanations or human verdicts. Read
|
|
3068
|
+
canonical context privately and qualify historical reconstruction when edited,
|
|
3069
|
+
deleted or missing. The September 21 expansion adds bounded, expiring private
|
|
3070
|
+
input snapshots as described below.
|
|
3071
|
+
See `twinkle-api/JEV_PILOT.md` for configuration,
|
|
3023
3072
|
the synthetic evaluation step and release checks.
|
|
3024
3073
|
|
|
3025
3074
|
### Application AI calendar-month cost (standing duty, every full daily review)
|
|
@@ -3170,6 +3219,34 @@ reload while waiting in the queue. The current UTC day is returned with `inProgr
|
|
|
3170
3219
|
**Headline `lastCompletedDay` (its exact `dayKey`) — never the in-progress
|
|
3171
3220
|
day**, exactly as the closed-day AI-cost duty does.
|
|
3172
3221
|
|
|
3222
|
+
Since Mikey's 2026-09-20 decision, keep current Energy policy and worker
|
|
3223
|
+
capacity while observing. During every full run, supplement these counters
|
|
3224
|
+
with the read-only per-request diagnostic report (from the local API checkout):
|
|
3225
|
+
|
|
3226
|
+
```bash
|
|
3227
|
+
ssh api-primary.twinkle.network \
|
|
3228
|
+
'cd /home/ec2-user/server && timeout 75s node --max-old-space-size=128 -' \
|
|
3229
|
+
< scripts/build-energy-daily.cjs
|
|
3230
|
+
```
|
|
3231
|
+
|
|
3232
|
+
This uses telemetry already recorded in queue jobs, canonical run sessions,
|
|
3233
|
+
provider-turn budget metadata and reservation usage; no new collection or API
|
|
3234
|
+
restart is needed. Save its JSON privately. Headline its last completed UTC
|
|
3235
|
+
day and compare the complete days in its seven-day window: queue wait p50/p90/
|
|
3236
|
+
maximum, starts waiting over 60 seconds, cancellations before start, unchanged
|
|
3237
|
+
budget stops, and the separate `handoff_only`, work-without-save and unknown
|
|
3238
|
+
patterns. Keep the two explicit denominators separate: unchanged stops / all
|
|
3239
|
+
budget stops, and unchanged stops / completed manual runs in the same cohort.
|
|
3240
|
+
Do not divide by usage reservations or assume busy-refusal counts measure waits.
|
|
3241
|
+
|
|
3242
|
+
Inspect the stop cases' observed starting budget, recorded work/handoff turns,
|
|
3243
|
+
remaining Energy and final-reservation spend. Missing lineage is unknown, not
|
|
3244
|
+
zero work or zero cost; final-reservation cost can exclude earlier planning
|
|
3245
|
+
reservations. The oldest day may be partial under rolling seven-day retention.
|
|
3246
|
+
These observations do not by themselves establish a bug or authorize an
|
|
3247
|
+
admission floor, extra worker capacity, budget cuts or model changes. Update
|
|
3248
|
+
todo 52 with the latest completed-day observations and any concrete regression.
|
|
3249
|
+
|
|
3173
3250
|
`flags` lists every tripped check with its exact numbers: `overflow_usd`
|
|
3174
3251
|
(overflow above $1 on a completed day), `budget_stop_unchanged_ratio` (more
|
|
3175
3252
|
than 30% of at least 5 budget stops ended with nothing saved),
|
|
@@ -4301,3 +4378,38 @@ contracts.
|
|
|
4301
4378
|
Legacy aliases such as `subjects list`, `subjects get`, `subjects featured`,
|
|
4302
4379
|
`comments get`, and `recommend` remain accepted, but the singular command forms
|
|
4303
4380
|
shown above are the canonical interface.
|
|
4381
|
+
|
|
4382
|
+
September 21 routing expansion: read `jevRoutingShadow` alongside `jevPilot` in
|
|
4383
|
+
both the admin brief and full daily report. The reviewed `chat-routing-v3` path
|
|
4384
|
+
is primary; report actual selection latency, chat baseline calls avoided, audit
|
|
4385
|
+
coverage and `jev_chat_routing_audit` cost separately from v1/v2. New routing
|
|
4386
|
+
families remain comparison-only until Mikey reviews each one and explicitly
|
|
4387
|
+
promotes it. Report every registered family, including zero-sample and skipped
|
|
4388
|
+
families; no traffic is not a pass. Show paired counts, exact differing fields,
|
|
4389
|
+
both values, private input snapshots when available, baseline-controlled outcome,
|
|
4390
|
+
provider/LLM timing, failure/skip reasons, stale comparisons, and USD spend from
|
|
4391
|
+
canonical `jev_<family>_shadow` cost operations. Context snapshots are bounded,
|
|
4392
|
+
private and expire after eight days; do not copy unrelated private content into
|
|
4393
|
+
public output. Raw model choices precede existing application guards and are not
|
|
4394
|
+
proof an action was performed. Do not call agreement accuracy or infer speedups
|
|
4395
|
+
from shadow timings. Full daily costs must include these operations once only.
|
|
4396
|
+
|
|
4397
|
+
September 21 Auto exception: Mikey approved JEV as Lumine Auto's primary model
|
|
4398
|
+
selector immediately, with an independent LLM comparison for every choice.
|
|
4399
|
+
Auto is the new default; stored manual preferences remain manual. Review
|
|
4400
|
+
`jevRoutingShadow` / `byRoute.lumine_model` separately from the eight
|
|
4401
|
+
comparison-only families: selected model/effort, both decisions, exact selection
|
|
4402
|
+
context, confidence, fallback, actual selection latency, missing evidence and
|
|
4403
|
+
observed task outcome. Report `jev_lumine_model_serve` and the additional
|
|
4404
|
+
`jev_lumine_model_audit` spend separately, using canonical AI-cost totals without
|
|
4405
|
+
double-counting. JEV choice confidence and LLM agreement are not correctness
|
|
4406
|
+
scores. The other eight families still require review before promotion.
|
|
4407
|
+
|
|
4408
|
+
|
|
4409
|
+
### September 21 verified reward follow-up
|
|
4410
|
+
|
|
4411
|
+
After the Study review migration is deployed, the full daily website run also reads the API-host report `node scripts/build-study-reviews-daily.cjs --days 1`. It is indexed, read-only and bounded; it reports decisions, known measured cost, unknown-usage requests, p50/p95 latency, settled checks, claims, pending rows older than 60 seconds and graded-but-unsettled rows. `--examples` includes at most 30 private full study records and their exact prompt version for administrator quality review; keep that output private. No traffic is not a quality pass. The 12 synthetic pre-release records are a small initial check, not measured real-user accuracy.
|
|
4412
|
+
|
|
4413
|
+
Study's registered `study-record-v1` checks use learner AI Energy for accepted, revise and uncertain results. Preview checks use real AI Energy but cannot award XP/Coins. The agreed proposal pays 2,000 XP + 500 Coins at most once per learner per UTC day. Breadface's `breadface-v1` uses deterministic server physics, not a model. Both creator proposals must be accepted to publish; inspect the canonical review state and acceptance/rejection events during follow-up.
|
|
4414
|
+
|
|
4415
|
+
For total JEV cost, include `dailyAiCosts.byProviderModel` for TypeSafe/JEV across both AI Energy and system-covered ledgers. Report serving/shadow/audit operations separately as above; the Build SDK's system-covered/failed calls use `jev_decision`, while ordinary viewer SDK checks are already counted by the AI Energy ledger. Do not add the Study report's known-cost sum on top of the canonical overall AI-cost total: it is a diagnostic breakdown of those same calls.
|