agent-apprenticeship 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (83) hide show
  1. package/README.md +269 -3
  2. package/bin/agent-apprenticeship.js +1 -2
  3. package/package.json +4 -4
  4. package/pyproject.toml +10 -4
  5. package/schemas/environment_replay.schema.json +41 -0
  6. package/schemas/eval_package.schema.json +37 -0
  7. package/schemas/experience_compilation.schema.json +53 -0
  8. package/schemas/runtime_training.schema.json +33 -0
  9. package/schemas/source_evidence_map.schema.json +25 -0
  10. package/schemas/trace.schema.json +904 -0
  11. package/schemas/trace_step.schema.json +622 -0
  12. package/schemas/training_rows.schema.json +29 -0
  13. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/__init__.py +1 -1
  14. package/src/agent_apprenticeship/agent_env_exporter.py +53 -0
  15. package/src/agent_apprenticeship/agent_env_verify.py +32 -0
  16. package/src/agent_apprenticeship/atif.py +152 -0
  17. package/src/agent_apprenticeship/atif_models/LICENSE +201 -0
  18. package/src/agent_apprenticeship/atif_models/__init__.py +38 -0
  19. package/src/agent_apprenticeship/atif_models/agent.py +35 -0
  20. package/src/agent_apprenticeship/atif_models/content.py +159 -0
  21. package/src/agent_apprenticeship/atif_models/final_metrics.py +43 -0
  22. package/src/agent_apprenticeship/atif_models/metrics.py +47 -0
  23. package/src/agent_apprenticeship/atif_models/observation.py +19 -0
  24. package/src/agent_apprenticeship/atif_models/observation_result.py +45 -0
  25. package/src/agent_apprenticeship/atif_models/step.py +142 -0
  26. package/src/agent_apprenticeship/atif_models/subagent_trajectory_ref.py +88 -0
  27. package/src/agent_apprenticeship/atif_models/tool_call.py +34 -0
  28. package/src/agent_apprenticeship/atif_models/trajectory.py +196 -0
  29. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/batch_runner.py +1 -1
  30. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/bundle_exporter.py +8 -1
  31. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/certification.py +1 -1
  32. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/cli.py +26 -7
  33. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/codex_runner.py +17 -17
  34. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/config.py +2 -2
  35. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/ecosystem_r2.py +3 -1
  36. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/evaluator.py +8 -27
  37. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/grader.py +6 -5
  38. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/integration_status.py +1 -1
  39. package/src/agent_apprenticeship/io.py +104 -0
  40. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/learning.py +7 -1
  41. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/loop.py +5 -5
  42. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/loop_review.py +1 -1
  43. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/mentor_checkpoints.py +2 -2
  44. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/package_exporter.py +5 -3
  45. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/progress.py +1 -1
  46. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/public_run.py +1 -1
  47. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/public_sanitizer.py +2 -2
  48. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/release_exporter.py +7 -4
  49. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/rubric_generation.py +1 -1
  50. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/schemas.py +17 -4
  51. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/session_events.py +4 -4
  52. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/task_intake.py +1 -1
  53. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/tdo.py +5 -1
  54. package/src/agent_apprenticeship/trace_normalizer.py +103 -0
  55. package/src/agent_apprenticeship/trace_prompt.py +48 -0
  56. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/training_signals.py +1 -1
  57. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/validation.py +3 -3
  58. package/src/{agent_apprenticeship_trace → agent_apprenticeship}/verifier.py +6 -5
  59. package/docs/bioactive_evidence_mapping_schema.md +0 -113
  60. package/docs/dataset_schema.md +0 -160
  61. package/docs/local_usage.md +0 -191
  62. package/docs/scaling_to_1000_tasks.md +0 -3
  63. package/docs/trace_schema.md +0 -192
  64. package/docs/v0.2_release_notes.md +0 -61
  65. package/src/agent_apprenticeship_trace/io.py +0 -20
  66. package/src/agent_apprenticeship_trace/trace_normalizer.py +0 -1026
  67. package/src/agent_apprenticeship_trace/trace_prompt.py +0 -78
  68. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/actual_outputs_normalizer.py +0 -0
  69. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/apprentice_adapters.py +0 -0
  70. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/artifact_capture.py +0 -0
  71. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/artifact_previews.py +0 -0
  72. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/artifact_resolver.py +0 -0
  73. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/command_discovery.py +0 -0
  74. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/contract_diagnostics.py +0 -0
  75. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/contract_verifier.py +0 -0
  76. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/env.py +0 -0
  77. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/lesson_extractor.py +0 -0
  78. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/llm_output_normalizer.py +0 -0
  79. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/openai_structured.py +0 -0
  80. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/recipes.py +0 -0
  81. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/revision.py +0 -0
  82. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/role_runners.py +0 -0
  83. /package/src/{agent_apprenticeship_trace → agent_apprenticeship}/version.py +0 -0
@@ -6,7 +6,7 @@ from datetime import datetime, timezone
6
6
  from pathlib import Path
7
7
  from typing import Any, Callable, Literal
8
8
 
9
- from .io import append_jsonl, read_json, read_jsonl, write_json
9
+ from .io import append_jsonl, read_workflow_json as read_json, read_workflow_jsonl as read_jsonl, write_json
10
10
 
11
11
  RUN_TERMINAL_STATUSES = {"completed", "partial", "failed"}
12
12
 
@@ -10,7 +10,7 @@ from pathlib import Path
10
10
  from .bundle_exporter import create_contribution_bundle
11
11
  from .config import Settings, apprentice_agent_display_name, apprentice_agent_readiness_status, get_settings, settings_override
12
12
  from .env import redact_secrets
13
- from .io import read_json, write_json
13
+ from .io import read_workflow_json as read_json, write_json
14
14
  from .loop import run_task
15
15
  from .loop_review import record_followup_review_update, record_human_loop_feedback
16
16
  from .mentor_checkpoints import write_mentor_checkpoints
@@ -3,9 +3,9 @@ import hashlib, json, re, shutil
3
3
  from pathlib import Path
4
4
  from typing import Any
5
5
  from .env import redact_secrets, contains_secret
6
- from .io import read_jsonl, append_jsonl, write_json
6
+ from .io import read_workflow_jsonl as read_jsonl, append_jsonl, write_json
7
7
 
8
- PROMPT_TEMPLATE_ID='agent_apprenticeship_trace_worker_v0'
8
+ PROMPT_TEMPLATE_ID='agent_apprenticeship_worker_v0'
9
9
  PROMPT_TEMPLATE_VERSION='0.1'
10
10
  PROMPT_PUBLICATION_STATUS='template_available_in_codebase'
11
11
  RAW_LOG_NAMES=set()
@@ -1,10 +1,11 @@
1
1
  from __future__ import annotations
2
2
  import shutil
3
3
  from pathlib import Path
4
- from .io import read_json, write_json, append_jsonl, read_jsonl
4
+ from .io import read_workflow_json as read_json, write_json, append_jsonl, read_workflow_jsonl as read_jsonl
5
5
  from .public_sanitizer import create_public_release
6
6
  from .package_exporter import IGNORED_RELEASE_DIR_NAMES, is_ignored_release_path, public_task_record
7
7
  from .trace_normalizer import normalize_trace_for_export
8
+ from .atif import to_workflow
8
9
 
9
10
  RELEASE_FILES=['full_task_records.jsonl','tasks.jsonl','task_intake_specs.jsonl','rubrics.jsonl','rubric_items.jsonl','raw_agent_traces.jsonl','agent_traces.jsonl','trace_normalization_reports.jsonl','actual_outputs_normalization_reports.jsonl','actual_outputs.jsonl','grader_results.jsonl','verifier_results.jsonl','evaluator_feedback.jsonl','revision_plans.jsonl','hillclimb_results.jsonl','lessons.jsonl','training_signals.jsonl','process_supervision.jsonl','reward_modeling.jsonl','verifier_training.jsonl','revision_preference_pairs.jsonl','role_results_index.jsonl','artifacts_index.json','packages_index.jsonl','forsy_like_collections.jsonl']
10
11
 
@@ -141,7 +142,7 @@ def _propagate_release_status_fields(release_root):
141
142
  _write_jsonl(path, rows)
142
143
 
143
144
  # Do not inject task_status into agent_traces.jsonl here.
144
- # Trace rows are governed by the AgentTrace schema; attempt_status belongs
145
+ # Trace rows are governed by the WorkflowTrace schema; attempt_status belongs
145
146
  # there, but task_status is task/package-level metadata and is exported via
146
147
  # tasks.jsonl and packages_index.jsonl.
147
148
 
@@ -190,14 +191,14 @@ def create_release(run_root: Path, release_root: Path) -> Path:
190
191
  if norm is not None:
191
192
  normalized_trace_count += 1; normalized_trace_step_count += len(norm.get('steps') or [])
192
193
  if tr is not None:
193
- tr_row=normalize_trace_for_export(dict(tr), report)
194
+ tr_row=to_workflow(normalize_trace_for_export(dict(tr), report))
194
195
  attempt_statuses.append(tr_row.get('attempt_status'))
195
196
  tr_row['iteration_index']=0 if a == 'baseline' else 1
196
197
  tr_row['previous_attempt_id']=None if a == 'baseline' else manifest.get('baseline_attempt_id')
197
198
  tr_row['revision_group_id']=tid
198
199
  tr_row['completion_reason']=manifest.get('loop_stop_reason')
199
200
  append_jsonl(release_root/'agent_traces.jsonl', tr_row)
200
- append_jsonl(release_root/'forsy_like_collections.jsonl', {'collection_id': tr.get('collection_id') or tid, 'trace_id': tr.get('trace_id'), 'attempt_kind': a, 'iteration_index': tr_row['iteration_index'], 'previous_attempt_id': tr_row['previous_attempt_id'], 'steps': tr.get('steps', []), 'trace_ref': f'packages/{tid}/attempts/{a}/agent_trace.json'})
201
+ append_jsonl(release_root/'forsy_like_collections.jsonl', {'collection_id': tr.get('collection_id') or tid, 'trace_id': tr.get('trace_id'), 'attempt_kind': a, 'iteration_index': tr_row['iteration_index'], 'previous_attempt_id': tr_row['previous_attempt_id'], 'trajectory': normalize_trace_for_export(tr, report), 'trace_ref': f'packages/{tid}/attempts/{a}/agent_trace.json'})
201
202
  else:
202
203
  missing_traces += 1; blockers.append(f'missing attempts/{a}/agent_trace.json')
203
204
  attempt_trace_refs[a]={'raw': f'packages/{tid}/attempts/{a}/agent_trace.raw.json' if (pkg/f'attempts/{a}/agent_trace.raw.json').exists() else None, 'normalized': f'packages/{tid}/attempts/{a}/agent_trace.normalized.json' if (pkg/f'attempts/{a}/agent_trace.normalized.json').exists() else None, 'canonical': f'packages/{tid}/attempts/{a}/agent_trace.json' if (pkg/f'attempts/{a}/agent_trace.json').exists() else None, 'normalization_report': f'packages/{tid}/attempts/{a}/trace_normalization_report.json' if (pkg/f'attempts/{a}/trace_normalization_report.json').exists() else None}
@@ -256,4 +257,6 @@ def create_release(run_root: Path, release_root: Path) -> Path:
256
257
  except Exception:
257
258
  pass
258
259
  _propagate_release_status_fields(release_root)
260
+ from .agent_env_exporter import export_agent_env
261
+ export_agent_env(release_root)
259
262
  return release_root
@@ -4,7 +4,7 @@ from pathlib import Path
4
4
  from pydantic import BaseModel
5
5
  from .schemas import TaskIntakeSpec, RubricItem, RubricSpec, RubricQualityReport
6
6
  from .config import apprentice_agent_display_name, get_settings
7
- from .io import read_json
7
+ from .io import read_workflow_json as read_json
8
8
  from .openai_structured import get_model_provider_status, run_structured_role
9
9
  from .public_sanitizer import sanitize_public_obj, sha256_text
10
10
 
@@ -102,7 +102,7 @@ class ActualOutputs(StrictModel):
102
102
  error_message: str | None = None
103
103
  metadata_json: DictAny = Field(default_factory=dict)
104
104
 
105
- class AgentTraceStep(StrictModel):
105
+ class WorkflowTraceStep(StrictModel):
106
106
  step: int
107
107
  turn: int
108
108
  actor: str
@@ -393,8 +393,18 @@ class AgentTraceStep(StrictModel):
393
393
  raise ValueError("operation is required for non-user steps")
394
394
  return self
395
395
 
396
- class AgentTrace(StrictModel):
397
- schema_version: str = "aa-trace-v0.1"
396
+ class WorkflowTrace(StrictModel):
397
+ """Internal task/evaluation projection. Persist using ATIF I/O only."""
398
+
399
+ @model_validator(mode="before")
400
+ @classmethod
401
+ def from_atif(cls, value):
402
+ from .atif import to_workflow
403
+ if isinstance(value, dict) and "agent" in value and "steps" in value:
404
+ return to_workflow(value)
405
+ return value
406
+
407
+ schema_version: str = "ATIF-v1.8"
398
408
  trace_id: str
399
409
  collection_id: str | None = None
400
410
  run_id: str | None = None
@@ -425,7 +435,7 @@ class AgentTrace(StrictModel):
425
435
  final_output_summary: str | None = None
426
436
  learning: str | None = None
427
437
  termination_reason: Literal["task_complete","verifier_passed","verifier_failed","max_iterations_reached","agent_blocked","timeout","error_unrecoverable","partial_then_stopped","provider_usage_limit","other"]
428
- steps: list[AgentTraceStep]
438
+ steps: list[WorkflowTraceStep]
429
439
  actual_outputs: ActualOutputs | None = None
430
440
  artifacts: list[ArtifactRef] = Field(default_factory=list)
431
441
  iteration_index: int | None = None
@@ -646,3 +656,6 @@ class TDOReport(StrictModel):
646
656
  generation_notes: list[str] = Field(default_factory=list)
647
657
  recommended_uses: list[str] = Field(default_factory=list)
648
658
  metadata_json: DictAny = Field(default_factory=dict)
659
+
660
+ # Public trace models use the Harbor interchange contract.
661
+ from .atif_models import Trajectory as AgentTrace, Step as AgentTraceStep
@@ -7,7 +7,7 @@ from typing import Any, Literal
7
7
 
8
8
  from pydantic import BaseModel, ConfigDict, Field
9
9
 
10
- from .io import append_jsonl, read_jsonl
10
+ from .io import append_jsonl, read_workflow_jsonl as read_jsonl
11
11
 
12
12
 
13
13
  SessionEventType = Literal[
@@ -84,9 +84,9 @@ def backfill_session_event_task_ids(run_root: Path, task_id: str | None) -> None
84
84
  changed = True
85
85
  if not changed:
86
86
  return
87
- with path.open("w") as f:
88
- for row in rows:
89
- f.write(json.dumps(row, sort_keys=True) + "\n")
87
+ path.write_text("")
88
+ for row in rows:
89
+ append_jsonl(path, row)
90
90
 
91
91
 
92
92
  def next_followup_index(run_root: Path) -> int:
@@ -4,7 +4,7 @@ from pathlib import Path
4
4
  from pydantic import BaseModel
5
5
  from .schemas import RawTaskRecord, TaskIntakeSpec, TaskIntakeQualityReport
6
6
  from .config import apprentice_agent_display_name, get_settings
7
- from .io import read_json, write_json
7
+ from .io import read_workflow_json as read_json, write_json
8
8
  from .openai_structured import extract_json_object, get_model_provider_status, run_structured_role
9
9
  from .public_sanitizer import sanitize_public_obj, sha256_text
10
10
 
@@ -14,7 +14,7 @@ from typing import Any
14
14
 
15
15
  from .config import Settings, apprentice_agent_display_name, configured_model_provider_ready
16
16
  from .env import contains_secret, redact_secrets
17
- from .io import append_jsonl, read_json, read_jsonl, write_json
17
+ from .io import append_jsonl, read_workflow_json as read_json, read_workflow_jsonl as read_jsonl, write_json
18
18
  from .progress import ProgressCallback, append_progress_event
19
19
  from .public_sanitizer import sanitize_public_obj, sanitize_public_text
20
20
  from .schemas import TDOJudgeResult, TDOReport
@@ -3277,6 +3277,8 @@ def run_tdo_for_package(package_root: Path, settings: Settings, *, run_root: Pat
3277
3277
  )
3278
3278
  write_json(tdo_dir / "tdo_report.json", report)
3279
3279
  write_json(tdo_dir / "tdo_manifest.json", {"schema_version": TDO_VERSION, "tdo_status": "disabled"})
3280
+ from .agent_env_exporter import export_agent_env
3281
+ export_agent_env(package_root)
3280
3282
  return report.model_dump(mode="json")
3281
3283
  max_rounds = _int_env("AA_EXPERIENCE_COMPILER_MAX_ROUNDS", _int_env("AA_TDO_MAX_ROUNDS", 3))
3282
3284
  min_quality = _float_env("AA_EXPERIENCE_COMPILER_MIN_QUALITY", _float_env("AA_TDO_MIN_QUALITY_SCORE", 0.65))
@@ -3466,6 +3468,8 @@ def run_tdo_for_package(package_root: Path, settings: Settings, *, run_root: Pat
3466
3468
  write_artifacts_index(package_root)
3467
3469
  except Exception:
3468
3470
  pass
3471
+ from .agent_env_exporter import export_agent_env
3472
+ export_agent_env(package_root)
3469
3473
  return report.model_dump(mode="json")
3470
3474
 
3471
3475
 
@@ -0,0 +1,103 @@
1
+ """Validate inbound ATIF; never guess or repair an incompatible trace schema."""
2
+ from __future__ import annotations
3
+ from pathlib import Path
4
+ from typing import Any
5
+ from pydantic import BaseModel, Field
6
+ from .schemas import ActualOutputs, WorkflowTrace
7
+ from .atif import load_trace, to_workflow, from_workflow, validate_trace
8
+
9
+ class TraceNormalizationContext(BaseModel):
10
+ task_id: str
11
+ attempt_id: str
12
+ attempt_kind: str
13
+ task: str
14
+ actual_outputs: ActualOutputs | None = None
15
+ trace_id_prefix: str = 'trace'
16
+
17
+ class TraceNormalizationReport(BaseModel):
18
+ task_id: str
19
+ attempt_id: str
20
+ attempt_kind: str
21
+ raw_trace_ref: str | None = None
22
+ normalized_trace_ref: str | None = None
23
+ canonical_trace_ref: str | None = None
24
+ trace_schema_valid: bool = False
25
+ trace_normalized: bool = False
26
+ trace_lossless: bool = True
27
+ fallback_trace: bool = False
28
+ raw_step_count: int = 0
29
+ normalized_step_count: int = 0
30
+ discarded_step_count: int = 0
31
+ raw_trace_parse_error: bool = False
32
+ trace_normalization_error: bool = False
33
+ trace_normalization_partial: bool = False
34
+ validation_errors: list[str] = Field(default_factory=list)
35
+ warnings: list[str] = Field(default_factory=list)
36
+ field_mappings: dict[str,str] = Field(default_factory=dict)
37
+ input_dict_to_string_count: int = 0
38
+ output_dict_to_string_count: int = 0
39
+ causal_type_repair_count: int = 0
40
+ operation_repair_count: int = 0
41
+ action_repair_count: int = 0
42
+ metadata_json: dict[str, Any] = Field(default_factory=dict)
43
+
44
+ class NormalizationResult(BaseModel):
45
+ normalized_trace: dict[str, Any] | None = None
46
+ report: TraceNormalizationReport
47
+ fallback_required: bool = False
48
+ parse_error: str | None = None
49
+
50
+
51
+ def infer_attempt_status(trace: dict[str, Any], report: dict[str, Any] | None = None) -> str:
52
+ existing=trace.get('attempt_status')
53
+ if existing in {'completed', 'failed', 'blocked', 'fallback', 'partial'}:
54
+ return existing
55
+ termination=trace.get('termination_reason')
56
+ md=trace.get('metadata_json') if isinstance(trace.get('metadata_json'), dict) else {}
57
+ report=report or {}
58
+ if termination == 'task_complete':
59
+ return 'completed'
60
+ if termination == 'agent_blocked':
61
+ return 'blocked'
62
+ actual=trace.get('actual_outputs') if isinstance(trace.get('actual_outputs'), dict) else {}
63
+ if termination in {'timeout','error_unrecoverable','provider_usage_limit','verifier_failed'} or actual.get('status') in {'failed','timeout','error'}:
64
+ return 'failed'
65
+ if report.get('fallback_trace') or md.get('fallback_trace') or md.get('fallback_trace_created') or md.get('fallback_reason') or report.get('fallback_reason') or (isinstance(report.get('metadata_json'), dict) and report['metadata_json'].get('fallback_reason')):
66
+ return 'fallback'
67
+ steps=trace.get('steps') or []
68
+ if any(s.get('step_outcome') == 'blocked' for s in steps if isinstance(s, dict)):
69
+ return 'blocked'
70
+ if any(s.get('step_outcome') == 'failed' or s.get('success') is False for s in steps if isinstance(s, dict)):
71
+ return 'failed' if not any(s.get('success') is True for s in steps if isinstance(s, dict)) else 'partial'
72
+ if steps:
73
+ return 'partial'
74
+ return 'failed'
75
+
76
+ def normalize_trace_for_export(trace: dict, report: dict | None = None) -> dict:
77
+ projection = to_workflow(trace) if "agent" in trace else dict(trace)
78
+ projection["attempt_status"] = infer_attempt_status(projection, report)
79
+ return from_workflow(projection)
80
+
81
+
82
+ def repair_agent_trace_file(raw_trace_path: Path, context: TraceNormalizationContext) -> NormalizationResult:
83
+ report = TraceNormalizationReport(task_id=context.task_id, attempt_id=context.attempt_id,
84
+ attempt_kind=context.attempt_kind, raw_trace_ref=raw_trace_path.name)
85
+ try:
86
+ atif = load_trace(raw_trace_path)
87
+ trace = to_workflow(atif)
88
+ trace.update(task_id=context.task_id, attempt_id=context.attempt_id,
89
+ attempt_kind=context.attempt_kind, task=context.task)
90
+ if context.actual_outputs is not None:
91
+ trace['actual_outputs'] = context.actual_outputs.model_dump(mode='json')
92
+ WorkflowTrace.model_validate(trace)
93
+ normalized = from_workflow(trace)
94
+ except (ValueError, TypeError, OSError) as error:
95
+ report.trace_normalization_error = True
96
+ report.trace_lossless = False
97
+ report.validation_errors.append(str(error))
98
+ return NormalizationResult(report=report, fallback_required=True, parse_error=str(error))
99
+ report.raw_step_count = report.normalized_step_count = len(atif['steps'])
100
+ report.trace_schema_valid = report.trace_normalized = True
101
+ report.normalized_trace_ref = 'agent_trace.normalized.json'
102
+ report.canonical_trace_ref = 'agent_trace.json'
103
+ return NormalizationResult(normalized_trace=normalized, report=report)
@@ -0,0 +1,48 @@
1
+ from __future__ import annotations
2
+
3
+ TRACE_SCHEMA_GUIDANCE = """Write ./agent_trace.json as a complete Harbor ATIF v1.8 trajectory:
4
+ {
5
+ "schema_version": "ATIF-v1.8",
6
+ "agent": {"name": "your-agent", "version": "your-agent-version"},
7
+ "steps": [
8
+ {"step_id": 1, "source": "user", "message": "Task instruction"},
9
+ {"step_id": 2, "source": "agent", "message": "Read task input",
10
+ "tool_calls": [{"tool_call_id": "call_2", "function_name": "read_file", "arguments": {"path": "input/task.md"}}],
11
+ "observation": {"results": [{"source_call_id": "call_2", "content": "Observed result"}]}}
12
+ ]
13
+ }
14
+ Use sequential integer step_id values starting at 1; source is system, user, or agent.
15
+ Record each meaningful interaction faithfully. Tool results belong in observation.results,
16
+ with source_call_id matching a tool_call_id on the same step. Preserve parallel calls.
17
+ Record available timestamps, model_name, metrics, final_metrics and subagent_trajectories.
18
+ Embedded subagents need unique trajectory_id values. References use trajectory_id or
19
+ trajectory_path. Image/audio message parts use type and source {media_type, path};
20
+ audio may include duration_sec. Never embed media bytes or invent results or metrics.
21
+ Put project-specific task IDs, attempt IDs, artifacts, causal links, surface fields,
22
+ and outcome evidence in extra.agent_apprenticeship at the root or step level.
23
+ Do not add non-ATIF top-level fields. Do not include hidden chain-of-thought or secrets.
24
+ Read inputs under ./input/ and write deliverables under ./artifacts/.
25
+ Also write ./actual_outputs.json with task_id, attempt_id, attempt_kind, status,
26
+ output_summary, primary_output_ref, deliverable_refs, artifact_refs and files_created.
27
+ Use package-relative artifact references. Evaluation is performed separately.
28
+ """
29
+
30
+ MASKING_GUIDANCE = """Sensitive info masking is standard for this run. Do not expose API keys, private tokens, credentials, or secrets in traces, logs, or artifacts. Preserve useful operational context while masking super-sensitive values."""
31
+
32
+
33
+ def build_worker_prompt(
34
+ task_instruction: str,
35
+ rubric_md: str = "",
36
+ attempt_kind: str = "baseline",
37
+ input_files: list[str] | None = None,
38
+ deliverables: list[str] | None = None,
39
+ sensitive_info_masking: str = "standard",
40
+ workspace_path: str | None = None,
41
+ ) -> str:
42
+ input_files = input_files or []
43
+ deliverables = deliverables or []
44
+ input_section = "\n".join(f"- ./input/{name}" for name in input_files) or "- No explicit input files declared."
45
+ deliverable_section = "\n".join(f"- ./artifacts/{name}" for name in deliverables) or "- Write required task deliverables under ./artifacts/."
46
+ workspace_section = f"\nCurrent task workspace: {workspace_path}\n" if workspace_path else ""
47
+ masking_section = f"\n\n## Sensitive info masking\n{MASKING_GUIDANCE}" if sensitive_info_masking == "standard" else ""
48
+ return f"# Agent Apprenticeship Workflow Task\n\nAttempt: {attempt_kind}\n{workspace_section}\n{TRACE_SCHEMA_GUIDANCE}{masking_section}\n\n## Input files\n{input_section}\n\n## Required output contract\nYou must write these files in the current task workspace before finishing:\n- ./agent_trace.json\n- ./actual_outputs.json\n- ./artifacts/\n{deliverable_section}\n\nBefore finishing, verify ./agent_trace.json, ./actual_outputs.json, and ./artifacts/ exist. Do not only answer in chat/stdout. Do not say the task is complete unless those files exist.\n\n## Task\n{task_instruction}\n\n## Worker-visible evaluation rubric\n{rubric_md}\n\nFinal instruction: write the required files in this workspace, verify they exist, then stop.\n"
@@ -11,7 +11,7 @@ def _current_evaluator_feedback_text(grader: GraderResult | None, evaluator: Eva
11
11
  return 'Evaluator feedback omitted because it referenced stale artifact-contract state.'
12
12
  return text
13
13
 
14
- def process_supervision_from_trace(trace: AgentTrace, grader: GraderResult | None=None, verifier: VerifierResult | None=None, evaluator: EvaluatorFeedback | None=None) -> list[ProcessSupervisionExample]:
14
+ def process_supervision_from_trace(trace: WorkflowTrace, grader: GraderResult | None=None, verifier: VerifierResult | None=None, evaluator: EvaluatorFeedback | None=None) -> list[ProcessSupervisionExample]:
15
15
  final=grader.score if grader else None
16
16
  evaluator_text=_current_evaluator_feedback_text(grader, evaluator)
17
17
  out=[]
@@ -1,8 +1,8 @@
1
1
  from __future__ import annotations
2
2
  import json
3
3
  from pathlib import Path
4
- from .schemas import AgentTrace
5
- from .io import read_jsonl
4
+ from .schemas import WorkflowTrace
5
+ from .io import read_workflow_jsonl as read_jsonl
6
6
  from .env import contains_secret
7
7
  from .public_sanitizer import has_prompt_leak
8
8
  from .artifact_resolver import artifact_ref_resolves, is_artifact_evidence_ref, normalize_artifact_ref
@@ -64,7 +64,7 @@ def validate_release(root: Path) -> dict[str, object]:
64
64
  counters['score_source_counts']=sc; counters['hillclimb_result_count']=len(read_jsonl(root/'hillclimb_results.jsonl')); counters['process_supervision_count']=len(read_jsonl(root/'process_supervision.jsonl')); counters['reward_modeling_count']=len(read_jsonl(root/'reward_modeling.jsonl')); counters['revision_preference_pair_count']=len(read_jsonl(root/'revision_preference_pairs.jsonl'))
65
65
  traces=read_jsonl(root/'agent_traces.jsonl'); counters['trace_count']=len(traces)
66
66
  for t in traces:
67
- try: AgentTrace.model_validate(t); counters['trace_valid_count']+=1
67
+ try: WorkflowTrace.model_validate(t); counters['trace_valid_count']+=1
68
68
  except Exception: counters['trace_invalid_count']+=1; counters['release_valid']=False
69
69
  steps=t.get('steps') or []; counters['trace_step_count_total'] += len(steps)
70
70
  counters['trace_steps_with_tool_count'] += sum(1 for s in steps if s.get('tool'))
@@ -1,9 +1,10 @@
1
1
  from __future__ import annotations
2
+ from .atif import from_workflow
2
3
  import json
3
4
  from pathlib import Path
4
- from .schemas import VerifierResult, GraderResult, ActualOutputs, AgentTrace
5
+ from .schemas import VerifierResult, GraderResult, ActualOutputs, WorkflowTrace
5
6
  from .config import get_settings
6
- from .io import read_json
7
+ from .io import read_workflow_json as read_json
7
8
  from .openai_structured import get_model_provider_status, run_structured_role
8
9
  from .public_sanitizer import sha256_text
9
10
  from .artifact_previews import build_artifact_previews
@@ -30,13 +31,13 @@ def deterministic_verify(grader: GraderResult, outputs: ActualOutputs) -> Verifi
30
31
  issues.append('hidden reference leakage')
31
32
  return VerifierResult(verifier_result_id=f'verifier_{outputs.attempt_id}', task_id=outputs.task_id, attempt_id=outputs.attempt_id, attempt_kind=outputs.attempt_kind, grader_result_id=grader.grader_result_id, verification_status='not_run', artifact_contract_ok=artifact_ok, evidence_grounding_ok=bool(grader.evidence_refs), score_consistency_ok=score_ok, hidden_reference_leaked=grader.hidden_reference_leaked, issues=issues, confidence=0.6, verifier_notes='Structural guardrail checked artifact refs, score bounds, and leakage flags only; Mentor verification did not run.', semantic_evidence_grounding_ok=None, unsupported_claims=[], leakage_check_ok=not grader.hidden_reference_leaked, model=None, provider=None, metadata_json={'score_source': grader.score_source, 'structural_guardrail': True, 'role_status': 'mentor_verifier_unavailable'})
32
33
 
33
- def _verifier_prompt(grader: GraderResult, outputs: ActualOutputs, trace: AgentTrace | None, artifact_preview_bundle: dict | None=None) -> str:
34
+ def _verifier_prompt(grader: GraderResult, outputs: ActualOutputs, trace: WorkflowTrace | None, artifact_preview_bundle: dict | None=None) -> str:
34
35
  return """Return only valid JSON matching VerifierResult. Do not include markdown.
35
36
  Check whether the grader evidence is grounded in actual_outputs.json, artifact refs, trace evidence, and especially the artifact content previews. Check score consistency, unsupported claims, and hidden-reference leakage. Do not treat Apprentice Agent self-assessment or self evaluation as outcome evidence. Do not use deterministic fallback judgement; structural guardrails are package-integrity checks only and are not verifier judgments. Return exactly the VerifierResult schema fields. Do not add top-level fields not in schema. If you want to report extra checks such as reproducible, within_time_limit, uses_only_allowed_apis, passed_checks, or within_memory_limit, put them inside metadata_json.extra_model_fields.
36
37
  GraderResult JSON:
37
- """ + json.dumps(grader.model_dump(), sort_keys=True) + "\nActualOutputs JSON:\n" + json.dumps(outputs.model_dump(), sort_keys=True) + "\nArtifact content previews JSON:\n" + json.dumps(artifact_preview_bundle or {}, sort_keys=True)[:16000] + "\nTrace summary JSON:\n" + json.dumps((trace.model_dump() if trace else {}), sort_keys=True)[:12000]
38
+ """ + json.dumps(grader.model_dump(), sort_keys=True) + "\nActualOutputs JSON:\n" + json.dumps(outputs.model_dump(), sort_keys=True) + "\nArtifact content previews JSON:\n" + json.dumps(artifact_preview_bundle or {}, sort_keys=True)[:16000] + "\nTrace summary JSON:\n" + json.dumps((from_workflow(trace.model_dump()) if trace else {}), sort_keys=True)
38
39
 
39
- def verify_attempt(grader: GraderResult, outputs: ActualOutputs, trace: AgentTrace | None=None, role_root: Path | None=None, package_root: Path | None=None) -> VerifierResult:
40
+ def verify_attempt(grader: GraderResult, outputs: ActualOutputs, trace: WorkflowTrace | None=None, role_root: Path | None=None, package_root: Path | None=None) -> VerifierResult:
40
41
  settings=get_settings(); pre=deterministic_verify(grader, outputs); role_root=role_root or Path('outputs/roles')
41
42
  artifact_preview_bundle=build_artifact_previews(package_root, grader.evidence_refs or outputs.deliverable_refs or outputs.artifact_refs or outputs.files_created)
42
43
  if _model_verifier_active(settings) and settings.llm_verifier_enabled:
@@ -1,113 +0,0 @@
1
- # Bioactive Evidence Mapping Extension Schema
2
-
3
- This optional Agent Apprenticeship schema describes evidence-mapping rows for specialized research and decision-support datasets. It is a general Experience Compilation extension and does not change the core product into a wellness-only tool.
4
-
5
- These rows are for provenance, uncertainty tracking, citation grounding, mechanism mapping, and verifier training. They are not medical advice, diagnosis, treatment guidance, cure claims, or sensitive personal records.
6
-
7
- ## Row Families
8
-
9
- ### evidence_mapping_sft_rows
10
-
11
- LoRA-ready instruction-tuning view for extracting grounded ingredient, compound, product, evidence, mechanism, safety/caution, and uncertainty fields from provided local source material.
12
-
13
- Required fields:
14
-
15
- - `row_id`
16
- - `schema_version`
17
- - `data_type`
18
- - `source_task_id`
19
- - `domain`
20
- - `subdomain`
21
- - `workflow_family`
22
- - `instruction`
23
- - `input_context`
24
- - `target_output`
25
- - `source_text_or_artifact_ref`
26
- - `source_evidence_ref_ids`
27
- - `quality_score`
28
- - `confidence`
29
-
30
- ### ingredient_claim_verifier_rows
31
-
32
- Verifier-training view for checking whether a claim is supported by visible source evidence.
33
-
34
- Required fields:
35
-
36
- - `row_id`
37
- - `claim`
38
- - `ingredient_or_compound_entity`
39
- - `visible_evidence`
40
- - `expected_contract`
41
- - `verifier_decision`
42
- - `failure_reason`
43
- - `confidence`
44
- - `source_evidence_ref_ids`
45
-
46
- ### product_ingredient_extraction_rows
47
-
48
- Extraction view for mapping product labels or specification tables into structured ingredient/product entities.
49
-
50
- Required fields:
51
-
52
- - `row_id`
53
- - `product_entity`
54
- - `ingredient_or_compound_entities`
55
- - `source_text_or_artifact_ref`
56
- - `extraction_contract`
57
- - `citation_or_provenance_refs`
58
- - `source_evidence_ref_ids`
59
- - `confidence`
60
-
61
- ### mechanism_mapping_rows
62
-
63
- Mechanism evidence view for mapping a claim or source passage into a mechanism category with uncertainty.
64
-
65
- Required fields:
66
-
67
- - `row_id`
68
- - `ingredient_or_compound_entity`
69
- - `mechanism_category`
70
- - `evidence_summary`
71
- - `uncertainty`
72
- - `citation_or_provenance_refs`
73
- - `source_evidence_ref_ids`
74
- - `confidence`
75
-
76
- ### safety_caution_rows
77
-
78
- Safety/caution evidence view for neutral flags and context constraints. Use abstract personal-context constraints only, never sensitive identity data.
79
-
80
- Required fields:
81
-
82
- - `row_id`
83
- - `ingredient_or_compound_entity`
84
- - `safety_caution_category`
85
- - `personal_context_constraints`
86
- - `evidence_summary`
87
- - `uncertainty`
88
- - `citation_or_provenance_refs`
89
- - `source_evidence_ref_ids`
90
- - `confidence`
91
-
92
- ### citation_grounding_rows
93
-
94
- Grounding view for checking whether generated summaries cite visible evidence correctly.
95
-
96
- Required fields:
97
-
98
- - `row_id`
99
- - `generated_claim_or_summary`
100
- - `citation_or_provenance_refs`
101
- - `visible_evidence`
102
- - `verifier_decision`
103
- - `failure_reason`
104
- - `source_evidence_ref_ids`
105
- - `confidence`
106
-
107
- ## Safety Rules
108
-
109
- - Use only provided local source material or package evidence refs.
110
- - Keep personal context as abstract constraints such as `pregnancy_context`, `medication_interaction_context`, or `allergy_context`; do not store identity or sensitive user details.
111
- - Record uncertainty and provenance for every extracted claim.
112
- - Do not infer treatment, cure, diagnosis, or patient-specific recommendations.
113
- - Do not mark unsupported claims as accepted verifier rows.