coding-artifact-evaluation-shell 0.1.0-eidolon.a3d53b93d02d8 → 0.1.0-eidolon.af1b35bd65ae9

package/README.md

@@ -53,5 +53,3 @@ The next CLI invocation loads updated code; agents reread updated SOP paths. Fro
 
 
 For the explicit coding entry, durable evidence contract and legacy identity compatibility, see [SKILL.md](SKILL.md#coding-scope-and-durable-evidence). New package imports use `coding-artifact-evaluation-shell/run`.
-
-For current coding runs, explicit nonempty `codingScope.groups` also asks the same judge to compare every applicable caller-owned leaf, including after an overall FAIL. It does not add a role, scoring protocol, evidence or new requirements. An absent claim is unreviewed, distinct from an explicit indeterminate claim. Historical and ungrouped minimal FAIL decisions retain their original meaning; completeness and semantic correctness still require actual evidence.

package/SKILL.md

@@ -26,5 +26,3 @@ This evaluator assesses code-development artifacts and their explicitly required
 New integrations import `runCodingEvaluation` from `coding-artifact-evaluation-shell/run` and pass `codingScope:{kind:'coding',description,groups?}` plus a new absolute `retainTo` directory whose parent exists. Groups reuse the existing root projection and never change strict overall judgment. `verifyRetainedEvaluation(path)` verifies a data-only archive after temporary inputs are removed. The archive contains code, complete target bytes, observations, original model outputs/logs, decisions and costs; it excludes runtime HOME/auth/bin/live channels and never acts as a resume checkpoint. Failed partial closures remain partial.
 
 `runEvaluation`, `artifact-evaluation/v1` and `ArtifactEvaluation.*` FQNs remain compatibility identities for existing coding consumers and immutable historical records. They do not broaden the current product scope. Do not rename historical results or infer semantic PASS from retention verification.
-
-For current coding runs, explicit nonempty `codingScope.groups` also asks the same judge to compare every applicable caller-owned leaf, including after an overall FAIL. It does not add a role, scoring protocol, evidence or new requirements. An absent claim is unreviewed, distinct from an explicit indeterminate claim. Historical and ungrouped minimal FAIL decisions retain their original meaning; completeness and semantic correctness still require actual evidence.

package/capsule.ts

@@ -112,7 +112,7 @@ export async function runEvaluation(manifest:Manifest,output:string,options:Eval
   let delegate:JudgePort;
   if(options.judge)delegate=options.judge;
   else if(options.noModel)delegate={invocations:()=>[],judge:async()=>({judgment:'indeterminate',obligations:manifest.obligations??[],claims:[],findings:[],sourceReviews:sources.map(source=>({sourceId:source.id,complete:false,omissions:['No-model smoke does not make a semantic judgment'],ambiguities:[]}))})};
-  else delegate=createJudge(run,budget,state,manifest.includeCommitments,options.experimentalCounterReview?{reservedInvocations:1}:undefined,judgeAgent,Boolean(options.codingScope?.groups?.length));
+  else delegate=createJudge(run,budget,state,manifest.includeCommitments,options.experimentalCounterReview?{reservedInvocations:1}:undefined,judgeAgent);
   let reviewed:Proposal|undefined,reviewError:JudgeProtocolError|undefined;
   try {
     try {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "coding-artifact-evaluation-shell",
-  "version": "0.1.0-eidolon.a3d53b93d02d8",
+  "version": "0.1.0-eidolon.af1b35bd65ae9",
   "private": false,
   "type": "module",
   "description": "Code-development artifact evaluator and code-first Halfcode SkillApp",

package/resources/evaluate.md

@@ -22,7 +22,3 @@ Collect facts through the runner's owned process/HTTP/browser/file capabilities.
 The dynamic Evidence, Check and Groups LocalFunctions are read-only tools over explicit input paths. They do not grant execution rights, write verdicts or approve model budgets. Automated isolated model turns use the same implementation compiled into their owned scratch, not a generic host with broader permissions.
 
 For grouped results, the caller explicitly supplies `{id,label,rootIds}` entries selecting actual root obligation IDs from the admitted decision. Use `projectTargetGroups(input, decision, groups)` from the package, or discover the Groups LocalFunction and invoke it with absolute native `inputPath`, `decisionPath`, and `groups`. It reuses the original graph, shows ungrouped/shared roots, and preserves the overall result. Non-root/unknown selections are rejected for admitted decisions; unfinished or rejected drafts never yield partial PASS. There are no built-in domains or automatic categories. Labels do not certify unlisted requirements: a group of selected calculations is not proof of an entire product. Missing historical process evidence remains missing; a group view cannot supply it or repair a mistaken original judgment.
-
-For current coding runs, explicit nonempty `codingScope.groups` also asks the same judge to compare every applicable caller-owned leaf, including after an overall FAIL. It does not add a role, scoring protocol, evidence or new requirements. An absent claim is unreviewed, distinct from an explicit indeterminate claim. Historical and ungrouped minimal FAIL decisions retain their original meaning; completeness and semantic correctness still require actual evidence.
-
-Before collection, describe the caller's intended scope and method/depth in its trusted codingScope description and capability declarations; report the work actually completed. See [Review](review.md) for the shared evidence-depth guidance used by both collection and judgment. The existing claim method/conditions/reasoning/limitations support scope, actual investigation, coverage and gaps; there is no new wire field or depth score. A code-review-only selection cannot certify an original full-delivery requirement left unobserved. Index/search tools are optional, explicitly supplied locators whose version, resolution and coverage must be checked against frozen source; they do not substitute for execution or unavailable historical facts.

package/resources/review.md

@@ -13,11 +13,3 @@ Goal → fact → comparison:
 Read COMPLETE originals first; preserve each independent qualifier, modality and quantifier. Exclusion is not all-member selection; do not transfer a qualifier between objects without a source relation. A neighboring attribute cannot satisfy another property. Split independent conditions under all/any instead of partially certifying a broad leaf. Compare the ACTUAL CONTENT of static deliverables, not their titles/existence. Challenge PASS: could cited facts remain unchanged if the required property were absent? Challenge absence FAIL: inspect the bounded candidate set and relevant alternative implementations/call sites; unread/uncited supplied facts are not unavailable; ambiguous wording cannot become a stronger mechanism requirement. No extra planning file is required.
 Evidence must distinguish the promised consumer PURPOSE, not just setup/access/empty state. Genuine tests count for what their real seams/assertions exercise: cite BOTH their definition and native execution; a replaced entrance is not proven even when other dependencies are real. Ask which required failure a test detects. Temporal/causal/lifecycle requirements need observations distinguishing those conditions, not final values alone. Free-text probe conditions describe intent; neither a word such as "settled" nor tool completion proves business completion. For absence require established prerequisites and an observed completion/deadline, not immediate silence: unknown completion means indeterminate; an unavailable environment prerequisite is not a business capability failure. Representative acceptance is finite sampling; disclose limits without waiving material conditions. No exhaustive branch checklist is required; equivalent observable semantics count unless originals mandate the form.
 Derive expected outcomes before comparison; artifact output, exit 0, coverage labels and self-assertions never define the goal. PASS needs all required conditions; FAIL needs an applicable original contradiction surviving valid alternative readings/implementations; missing or ambiguous evidence is indeterminate. Keep judgments consistent with comparisons and findings. Actual facts and finite sampling are not universal proofs.
-
-Describe evidence depth per comparison, using existing fields: `method` names what actually happened (source inspection, symbol/call tracing, executed tests, integration or consumer observation); `conditions` identifies the assessed boundary, artifact/version and actual execution or historical conditions; `reasoning` states which required distinctions the evidence covers; `limitations` identifies material uncovered distinctions and why. This supports a scope | actual method/depth | coverage | judgment | gap view without extra output keys, a second score, or mandatory repeated checklists. A requested depth label is an operational plan, never evidence that the work happened. Keep explanations concise and reuse evidence references.
-
-Scope, method, depth and coverage are independent: scope selects the authorized object/requirements; methods obtain facts; depth describes how far the actual investigation followed relevant dependencies, transitions, error paths or counterexamples; coverage states inspected versus unassessed conditions. Choose the least costly sufficient evidence for each original requirement, deepening only for a material unresolved distinction within available capabilities and the same budget. Static architecture/deliverable requirements can be established by adequate source inspection without executing E2E; passing tests or a successful consumer journey do not establish unrelated architecture or all branches. Neither add generic testing/E2E/process obligations nor silently waive frozen original conditions. A deliberately unperformed method is not itself a failed requirement; an original behavioral condition still lacking support remains unresolved.
-
-Symbol indexes, call graphs and search results locate candidate evidence. When supplied, check their binding to the frozen artifact, source/file hashes or matching revision, supported languages, parser/resolver confidence, exclusions, unresolved edges and truncation; follow relevant locators back to actual source before making a code claim. Do not infer freshness from HEAD alone for dirty/non-git artifacts. A missing hit/path is not proof of absence when resolution, scope or traversal is incomplete, and a static graph is not an executed trace. Use only explicitly available capabilities; do not discover/install an indexer or invent a tool.
-
-Separate actual counterevidence from unperformed observations, unavailable/lost inputs, unresolved interpretation and exhausted investigation. State the specific missing fact in `limitations`; do not label already supplied unread evidence unavailable. Current source or a new execution cannot recreate a missing historical development/review event. Localize ambiguities and gaps to their actual requirement; preserve supported subcomparisons instead of describing successful test execution as unperformed. Original aggregate judgments and source-review admission rules still apply; do not turn a selected positive view into full-delivery PASS.

package/support/judge.ts

@@ -17,20 +17,16 @@ ${ENVIRONMENT_GUIDANCE}`;
 export const SERVICE_FACT_GUIDANCE='Service launch/argv/cwd/port/directory facts are conditions, NOT additional application requirements; directory creation is not application state. Do not demand a specific storage/persistence/restart mechanism unless COMPLETE originals actually require it; assess its actual source and native execution evidence.';
 
 /** Existing contracts only; a closed review is not taught a now-unavailable feedback route. */
-export function judgeGuidance(canSupplement:boolean,hasNeeds:boolean,completeClaims=false):string {
-  let choice='Capabilities are closed; do not request or execute new observations. Submit a final comparison of current native facts.';
-  if(canSupplement){
-    const failScope=completeClaims?'overall FAIL with complete requested leaf comparisons':'minimal FAIL';
-    choice='Check ALL currently material gaps together FIRST. If safely obtainable, return evidence requests promptly, not a provisional scorecard. A supported required contradiction already permits '+failScope+'.';
-  }
-  const coverage=completeClaims
-    ? 'The caller requested coding dimension views. Compare EVERY applicable leaf in the complete caller-owned obligation graph, even after a decisive FAIL. Submit one supported claim per leaf, or an explicit indeterminate claim explaining its missing material fact. Do not stop at the first contradiction or omit other dimensions; never invent positive claims to fill the table. Preserve all original scope, source reviews and needs. Overall still folds the same original graph; groups cannot change it. Keep each comparison concise; do not repeat the full logs.'
-    : 'Keep output proportional to the decision. A supported required FAIL needs its minimal witness/finding; an indeterminate needs explicit missing material scope, not gratuitous positive scores of every other area. Preserve caller obligations, source reviews and every need in either case. This never certifies unassessed scope. PASS still requires all original material conditions. Inspect explicit static deliverables early; a supported omission can be decisive. Use one comparison per claim; no extra planning document.';
-  return [JUDGE_GUIDANCE,SERVICE_FACT_GUIDANCE,choice,coverage,
+export function judgeGuidance(canSupplement:boolean,hasNeeds:boolean):string {
+  const choice=canSupplement
+    ? 'Check ALL currently material gaps together FIRST. If safely obtainable, return evidence requests promptly, not a provisional scorecard. A supported required contradiction already permits minimal FAIL.'
+    : 'Capabilities are closed; do not request or execute new observations. Submit a final comparison of current native facts.';
+  return [JUDGE_GUIDANCE,SERVICE_FACT_GUIDANCE,choice,
+    'Keep output proportional to the decision. A supported required FAIL needs its minimal witness/finding; an indeterminate needs explicit missing material scope, not gratuitous positive scores of every other area. Preserve caller obligations, source reviews and every need in either case. This never certifies unassessed scope. PASS still requires all original material conditions. Inspect explicit static deliverables early; a supported omission can be decisive. Use one comparison per claim; no extra planning document.',
     ...(canSupplement?[FEEDBACK_GUIDANCE]:[]),PROPOSAL_GUIDANCE,...(hasNeeds?[NEED_GUIDANCE]:[])].join('\n');
 }
 
-export function createJudge(run:AgentRun,budget:Budget,state:ProcessState,includeCommitments=false,experiment?:{window?:ModelWindow;reservedInvocations:number},agent?:AgentRuntime,completeClaims=false):JudgePort {
+export function createJudge(run:AgentRun,budget:Budget,state:ProcessState,includeCommitments=false,experiment?:{window?:ModelWindow;reservedInvocations:number},agent?:AgentRuntime):JudgePort {
   const invocations:Invocation[]=[];
   let finalWindow=experiment?.window;
   function assertCandidateSlot():void {
@@ -44,7 +40,7 @@ export function createJudge(run:AgentRun,budget:Budget,state:ProcessState,includ
     assertCandidateSlot();
     const effectiveWindow=canSupplement?window:closedWindow()??window;
     const payload={...input,policy:{includeCommitments},canSupplement,window:effectiveWindow};
-    const value=await modelTurn({run,budget,state,invocations,agent},{evaluation:input,payload,guidance:judgeGuidance(canSupplement,!!input.needs?.length,completeClaims),phase:'judgment',window:effectiveWindow});
+    const value=await modelTurn({run,budget,state,invocations,agent},{evaluation:input,payload,guidance:judgeGuidance(canSupplement,!!input.needs?.length),phase:'judgment',window:effectiveWindow});
     if(['evidence-request-v1','evidence-request-v2'].includes((value as any)?.format)){
       if(!canSupplement)throw new JudgeProtocolError('Judge requested evidence after final bounded review');
       return decodeFeedback(value,input,includeCommitments);
@@ -53,7 +49,7 @@ export function createJudge(run:AgentRun,budget:Budget,state:ProcessState,includ
   },async judge(input:EvaluationInput,repair) {
     assertCandidateSlot();
     const payload={...input,policy:{includeCommitments},repair:repair??null};
-    const guidance=repair?EVIDENCE_REPAIR_GUIDANCE:judgeGuidance(false,!!input.needs?.length,completeClaims);
+    const guidance=repair?EVIDENCE_REPAIR_GUIDANCE:judgeGuidance(false,!!input.needs?.length);
     const value=await modelTurn({run,budget,state,invocations,agent},{evaluation:input,payload,guidance,phase:repair?'representation-repair':'judgment',window:closedWindow()});
     return repair?decodeEvidenceRepair(value,repair):decodeProposal(value,input);
   }};