agent-harness 0.22.5 → 0.23.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 754470184c69b02fa37de51afaba8376d554b26869579dd9494d7e720ee0d425
4
- data.tar.gz: 9ec4b741ed03b43859cb3b2565a0e760cbcf2f5e1d80dadf287ae90a18230c25
3
+ metadata.gz: '0874add2f55550c3c9f9b723c84a4036a2dd79dd717dd62069062cfbde630820'
4
+ data.tar.gz: 53c6335c3a47e4e0e18ea3151dad061f9302519013e27591c7541ebf6dfe51db
5
5
  SHA512:
6
- metadata.gz: 24a4027c7f0594760a512456a995f206170e3e41386fc9e4c98386758982cb1b501a9d7bca9beb730a1784453f0436dc079a8923aa6f07bd5a0bd2a900b4f09c
7
- data.tar.gz: 9074fe5570d062bed53fe712bae4cdf95ec0d81fd01ed474cc64b3afb040535ca751fe7c79c5c69b49c91714f4fc2c2ff8dbe1c4827eb3a29e683fe944a49a8b
6
+ metadata.gz: a7297580e4a5a34631421730cab4c0fc5712448864bacefe8c8f3c43c7652b5a42cb9f7c2a603f585f308d2a54117dff918804a45a30c3bb302ec223e545eed4
7
+ data.tar.gz: 36428c8e91bee3395d235cd4ae309f6a43c177acc6991cdf3cd33ed4f0cc433a566849a8b282137466a455da5becdc63820a87a01a36eb8399f857ecbe457ef7
@@ -1,3 +1,3 @@
1
1
  {
2
- ".": "0.22.5"
2
+ ".": "0.23.0"
3
3
  }
data/CHANGELOG.md CHANGED
@@ -1,5 +1,17 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ### Features
4
+
5
+ * add runner model compatibility contract (`AgentHarness.model_compatibility`) with structured `ModelCompatibility::Result` outcomes. Codex exposes static facts for CLI-gated models (e.g. `gpt-5.5` requires Codex CLI `>= 0.116.0`), a baseline supported-model list, supported auth modes, and a `DEFAULT_COMPATIBLE_MODEL_ID` fallback so downstream orchestrators can validate tier/model assignments before scheduling agent runs ([#259](https://github.com/viamin/agent-harness/issues/259)).
6
+
7
+
8
+ ## [0.23.0](https://github.com/viamin/agent-harness/compare/agent-harness/v0.22.5...agent-harness/v0.23.0) (2026-06-15)
9
+
10
+
11
+ ### Features
12
+
13
+ * Add runner model compatibility contracts for Codex and CLI-gated models ([#260](https://github.com/viamin/agent-harness/issues/260)) ([4c192a7](https://github.com/viamin/agent-harness/commit/4c192a72f323ecaaea96e87e8ead6634e33669c0))
14
+
3
15
  ## [0.22.5](https://github.com/viamin/agent-harness/compare/agent-harness/v0.22.4...agent-harness/v0.22.5) (2026-06-12)
4
16
 
5
17
 
data/README.md CHANGED
@@ -326,6 +326,54 @@ tests assert that the expected binary remains aligned with that contract.
326
326
  `source_type`, `package_name`, version fields, and install commands so
327
327
  downstream apps do not need provider-specific branching.
328
328
 
329
+ ### Runner Model Compatibility
330
+
331
+ Downstream orchestrators that map tiers or task requirements to concrete
332
+ models should consult the runner/model compatibility contract before
333
+ scheduling a run. The contract surfaces the runtime dimensions that
334
+ materially affect whether a model will run — runner identity, model id,
335
+ auth mode, and installed CLI version — and returns a structured result
336
+ instead of forcing callers to parse error strings:
337
+
338
+ ```ruby
339
+ result = AgentHarness.model_compatibility(
340
+ runner: :codex,
341
+ model_id: "gpt-5.5",
342
+ auth_mode: :subscription,
343
+ cli_version: "0.115.0"
344
+ )
345
+
346
+ result.supported? # => false
347
+ result.reason # => :cli_version_too_old
348
+ result.minimum_cli_version # => "0.116.0"
349
+ result.fallback_model_id # => "gpt-5-codex"
350
+ result.source # => :static_contract
351
+ ```
352
+
353
+ Outcomes follow three explicit shapes:
354
+
355
+ - **Supported** — `result.supported?` is `true`. The runner contract
356
+ confirms the model can be driven under the given constraints.
357
+ - **Unsupported** — `result.unsupported?` is `true` with a stable
358
+ symbolic `reason` such as `:cli_version_too_old` or
359
+ `:auth_mode_not_supported`, plus any version requirement and a
360
+ contract-owned `fallback_model_id` to use instead.
361
+ - **Unknown** — `result.unknown?` is `true`. The runner cannot answer
362
+ definitively — either the model is not in the runner's static
363
+ contract (`reason: :unknown_model`), or the model is CLI-gated and the
364
+ caller did not supply a comparable `cli_version`
365
+ (`reason: :cli_version_unknown`, with `minimum_cli_version` attached).
366
+ Callers must treat unknown as "ask the provider" — never as implicit
367
+ approval — so a missing CLI version cannot silently schedule a run
368
+ onto an older CLI that the model does not support.
369
+
370
+ Providers that have not opted in to a static contract return an unknown
371
+ result by default, so callers always get a normalized
372
+ `AgentHarness::ModelCompatibility::Result` without provider-specific
373
+ branching. The Codex runner ships static facts for CLI-gated models
374
+ (for example `gpt-5.5` requires Codex CLI `>= 0.116.0`) and a baseline
375
+ list of always-supported models.
376
+
329
377
  ### Custom Providers
330
378
 
331
379
  ```ruby
@@ -0,0 +1,145 @@
1
+ # frozen_string_literal: true
2
+
3
+ module AgentHarness
4
+ # Structured runner/model compatibility contract.
5
+ #
6
+ # ModelCompatibility is the single source of truth for whether a given
7
+ # runner (provider) can execute a particular model under specific runtime
8
+ # constraints such as authentication mode or installed CLI version.
9
+ # Downstream orchestrators (for example Paid's RDR-040 tier-to-model
10
+ # mapping) consume this contract before validating tier models, selecting
11
+ # runners, and starting agent runs — instead of inferring compatibility
12
+ # from scattered facts like CLI version pins, smoke-test overrides, or
13
+ # observed runtime errors.
14
+ #
15
+ # @example Query Codex compatibility for a CLI-gated model
16
+ # AgentHarness.model_compatibility(
17
+ # runner: :codex,
18
+ # model_id: "gpt-5.5",
19
+ # auth_mode: :subscription,
20
+ # cli_version: "0.116.0"
21
+ # )
22
+ # # => #<AgentHarness::ModelCompatibility::Result supported=true ...>
23
+ module ModelCompatibility
24
+ # Sentinel for "the runner declares no opinion about this model."
25
+ UNKNOWN_REASON = :unknown
26
+ # Issued when a runner does not advertise the requested model at all.
27
+ UNKNOWN_MODEL_REASON = :unknown_model
28
+ # Issued when the runner needs a comparable CLI version to answer
29
+ # definitively for a CLI-gated model but the caller did not supply one
30
+ # (or it could not be parsed). Pairs with :minimum_cli_version on the
31
+ # result. Distinct from :unknown_model — the runner *does* know the
32
+ # model; it just cannot confirm the installed CLI is new enough.
33
+ UNKNOWN_CLI_VERSION_REASON = :cli_version_unknown
34
+ # Issued when the runner supports the model but the installed CLI is too
35
+ # old. Pairs with :minimum_cli_version on the result.
36
+ UNSUPPORTED_CLI_VERSION_REASON = :cli_version_too_old
37
+ # Issued when the runner supports the model but the requested auth mode
38
+ # is not part of the runner's contract for it.
39
+ UNSUPPORTED_AUTH_MODE_REASON = :auth_mode_not_supported
40
+ # Default supported reason.
41
+ SUPPORTED_REASON = :supported
42
+
43
+ # Sources for a compatibility decision. Static contracts are baked into
44
+ # the provider; live probes hit the provider CLI/API; entitlement checks
45
+ # gate on subscription state. Downstream orchestrators can use this to
46
+ # decide whether to cache the answer.
47
+ SOURCES = %i[static_contract live_provider_probe entitlement_check unknown].freeze
48
+
49
+ # Structured compatibility outcome.
50
+ #
51
+ # The struct intentionally exposes whatever facts the runner contract
52
+ # knows. Callers should treat unknown fields as nil rather than fail.
53
+ Result = Struct.new(
54
+ :runner,
55
+ :model_id,
56
+ :auth_mode,
57
+ :cli_version,
58
+ :supported,
59
+ :reason,
60
+ :minimum_cli_version,
61
+ :cli_version_requirement,
62
+ :fallback_model_id,
63
+ :source,
64
+ :details,
65
+ keyword_init: true
66
+ ) do
67
+ # @return [Boolean] true when compatibility is known to be supported
68
+ def supported? = supported == true
69
+
70
+ # @return [Boolean] true when compatibility is known to be unsupported
71
+ def unsupported? = supported == false
72
+
73
+ # @return [Boolean] true when the runner did not return a definite answer
74
+ def unknown? = supported.nil?
75
+
76
+ def to_h
77
+ super.compact
78
+ end
79
+ end
80
+
81
+ class << self
82
+ # Build a Result with sensible defaults from a partial Hash.
83
+ #
84
+ # Provider implementations call this helper to return a normalized
85
+ # Result without having to instantiate Struct fields by hand.
86
+ #
87
+ # @param runner [Symbol] canonical provider/runner name
88
+ # @param model_id [String, Symbol, nil] requested model id
89
+ # @param attributes [Hash] additional Result attributes
90
+ # @return [Result]
91
+ def build_result(runner:, model_id:, **attributes)
92
+ normalized_model_id = model_id.is_a?(Symbol) ? model_id.to_s : model_id
93
+ source = attributes.fetch(:source, :static_contract)
94
+ supported = attributes[:supported]
95
+
96
+ if supported == false && attributes[:reason].nil?
97
+ raise ArgumentError,
98
+ "AgentHarness::ModelCompatibility.build_result requires an explicit " \
99
+ "`reason:` when `supported: false`. Pass a specific reason " \
100
+ "(e.g. :cli_version_too_old, :auth_mode_not_supported) so " \
101
+ "callers can distinguish unsupported from :unknown."
102
+ end
103
+
104
+ reason = attributes[:reason] || default_reason_for(supported)
105
+
106
+ Result.new(
107
+ runner: runner.to_sym,
108
+ model_id: normalized_model_id,
109
+ auth_mode: attributes[:auth_mode],
110
+ cli_version: attributes[:cli_version],
111
+ supported: supported,
112
+ reason: reason,
113
+ minimum_cli_version: attributes[:minimum_cli_version],
114
+ cli_version_requirement: attributes[:cli_version_requirement],
115
+ fallback_model_id: attributes[:fallback_model_id],
116
+ source: source,
117
+ details: attributes[:details]
118
+ )
119
+ end
120
+
121
+ # Build an explicit "unknown" Result. Use when the runner has no
122
+ # static contract for the requested model and cannot probe.
123
+ def unknown_result(runner:, model_id:, fallback_model_id: nil, **attributes)
124
+ build_result(
125
+ runner: runner,
126
+ model_id: model_id,
127
+ supported: nil,
128
+ reason: attributes.delete(:reason) || UNKNOWN_REASON,
129
+ fallback_model_id: fallback_model_id,
130
+ source: attributes.delete(:source) || :unknown,
131
+ **attributes
132
+ )
133
+ end
134
+
135
+ private
136
+
137
+ def default_reason_for(supported)
138
+ case supported
139
+ when true then SUPPORTED_REASON
140
+ else UNKNOWN_REASON
141
+ end
142
+ end
143
+ end
144
+ end
145
+ end
@@ -709,6 +709,63 @@ module AgentHarness
709
709
  nil
710
710
  end
711
711
 
712
+ # Runner/model compatibility contract.
713
+ #
714
+ # Returns a structured {AgentHarness::ModelCompatibility::Result}
715
+ # describing whether this runner can execute +model_id+ under the
716
+ # given +auth_mode+ and installed +cli_version+. Providers that own
717
+ # CLI-gated or entitlement-gated models should override this to
718
+ # expose stable, structured facts so downstream orchestrators do not
719
+ # have to parse error strings or rediscover provider knowledge.
720
+ #
721
+ # The default implementation returns an explicit +:unknown+ result
722
+ # rather than collapsing to "supported" — callers are expected to
723
+ # handle unknown outcomes explicitly.
724
+ #
725
+ # @param model_id [String, Symbol] the requested model identifier
726
+ # @param auth_mode [Symbol, nil] caller's auth mode (e.g. :api_key,
727
+ # :subscription)
728
+ # @param cli_version [String, Gem::Version, nil] installed CLI
729
+ # version, when known
730
+ # @return [AgentHarness::ModelCompatibility::Result]
731
+ def model_compatibility(model_id:, auth_mode: nil, cli_version: nil)
732
+ AgentHarness::ModelCompatibility.unknown_result(
733
+ runner: provider_name,
734
+ model_id: model_id,
735
+ auth_mode: auth_mode,
736
+ cli_version: normalize_cli_version_for_compatibility(cli_version),
737
+ fallback_model_id: default_compatible_model_id
738
+ )
739
+ end
740
+
741
+ # Optional default model the runner can recommend as a fallback when
742
+ # a requested model is unsupported or unknown. Providers override
743
+ # this when they own a stable default model for smoke tests or
744
+ # downstream tier fallback.
745
+ #
746
+ # @return [String, nil]
747
+ def default_compatible_model_id
748
+ nil
749
+ end
750
+
751
+ # Helper for {#model_compatibility} implementations: coerce a CLI
752
+ # version string into a +Gem::Version+ when possible, returning the
753
+ # raw value otherwise. Keeps Result#cli_version stable across
754
+ # callers that pass either form.
755
+ def normalize_cli_version_for_compatibility(value)
756
+ return nil if value.nil?
757
+ return value if value.is_a?(Gem::Version)
758
+
759
+ str = value.respond_to?(:strip) ? value.strip : value.to_s
760
+ return nil if str.empty?
761
+
762
+ begin
763
+ Gem::Version.new(str)
764
+ rescue ArgumentError
765
+ str
766
+ end
767
+ end
768
+
712
769
  private
713
770
 
714
771
  def versioned_installation_contract(version)
@@ -20,6 +20,43 @@ module AgentHarness
20
20
 
21
21
  SUPPORTED_CLI_VERSION = "0.122.0"
22
22
  SUPPORTED_CLI_REQUIREMENT = Gem::Requirement.new(">= #{SUPPORTED_CLI_VERSION}", "< 0.123.0").freeze
23
+
24
+ # Default model recommended by the Codex runner contract when callers
25
+ # have no explicit preference. Used as the {AgentHarness::ModelCompatibility::Result#fallback_model_id}
26
+ # for unsupported/unknown model lookups so downstream orchestrators
27
+ # (smoke tests, tier fallback) have a stable, contract-backed choice
28
+ # instead of relying on whichever model the CLI's default points at.
29
+ DEFAULT_COMPATIBLE_MODEL_ID = "gpt-5-codex"
30
+
31
+ # Known CLI-gated model facts. Each entry expresses the minimum Codex
32
+ # CLI version required to drive that model. Keep entries here only when
33
+ # the requirement is durable runner contract knowledge — not
34
+ # provider-side experiments or one-off CLI defaults.
35
+ #
36
+ # The +gpt-5.5+ entry tracks the failure class observed in
37
+ # viamin/agent-harness#245 and viamin/agent-harness#250: older Codex
38
+ # CLI builds (e.g. 0.115.x) could not drive the +gpt-5.5+ family.
39
+ MODEL_COMPATIBILITY_FACTS = {
40
+ "gpt-5.5" => {minimum_cli_version: "0.116.0"},
41
+ "gpt-5.5-codex" => {minimum_cli_version: "0.116.0"}
42
+ }.each_value(&:freeze).freeze
43
+
44
+ # Models that the runner contract considers supported on every Codex
45
+ # CLI release we ship. Used by {.model_compatibility} so callers can
46
+ # distinguish "unknown to the contract" from "explicitly supported."
47
+ BASELINE_SUPPORTED_MODELS = %w[
48
+ gpt-5
49
+ gpt-5-codex
50
+ gpt-5-mini
51
+ gpt-4o
52
+ gpt-4o-mini
53
+ o4-mini
54
+ ].freeze
55
+
56
+ # Auth modes the Codex runner accepts. Compatibility checks for an
57
+ # unrecognised auth mode return :auth_mode_not_supported rather than
58
+ # silently approving the request.
59
+ SUPPORTED_AUTH_MODES = %i[api_key subscription].freeze
23
60
  OAUTH_REFRESH_FAILURE_PATTERNS = [
24
61
  /refresh_token_reused/i,
25
62
  /failed to refresh token\b.*\b401\b/im,
@@ -232,6 +269,143 @@ module AgentHarness
232
269
  Base::DEFAULT_SMOKE_TEST_CONTRACT
233
270
  end
234
271
 
272
+ def default_compatible_model_id
273
+ DEFAULT_COMPATIBLE_MODEL_ID
274
+ end
275
+
276
+ # Structured Codex compatibility contract.
277
+ #
278
+ # Returns an {AgentHarness::ModelCompatibility::Result} for the
279
+ # combination of +model_id+, +auth_mode+, and installed
280
+ # +cli_version+. The contract surfaces three concrete outcomes:
281
+ #
282
+ # 1. **Supported** — the model is in {BASELINE_SUPPORTED_MODELS} (or
283
+ # a known CLI-gated model whose minimum CLI version is met).
284
+ # 2. **Unsupported due to CLI version** — the model is known to need
285
+ # a newer Codex CLI than was supplied; the result carries
286
+ # +:minimum_cli_version+ so callers can act on it.
287
+ # 3. **Unknown / dynamic** — the model is not in this static
288
+ # contract. Callers must treat this as "ask the provider" rather
289
+ # than as approval.
290
+ #
291
+ # @param model_id [String, Symbol]
292
+ # @param auth_mode [Symbol, nil] :api_key or :subscription
293
+ # @param cli_version [String, Gem::Version, nil]
294
+ # @return [AgentHarness::ModelCompatibility::Result]
295
+ def model_compatibility(model_id:, auth_mode: nil, cli_version: nil)
296
+ normalized_model_id = model_id.to_s
297
+ normalized_auth_mode = auth_mode&.to_sym
298
+ normalized_cli_version = normalize_cli_version_for_compatibility(cli_version)
299
+
300
+ if normalized_auth_mode && !SUPPORTED_AUTH_MODES.include?(normalized_auth_mode)
301
+ return AgentHarness::ModelCompatibility.build_result(
302
+ runner: provider_name,
303
+ model_id: normalized_model_id,
304
+ auth_mode: normalized_auth_mode,
305
+ cli_version: normalized_cli_version,
306
+ supported: false,
307
+ reason: AgentHarness::ModelCompatibility::UNSUPPORTED_AUTH_MODE_REASON,
308
+ fallback_model_id: DEFAULT_COMPATIBLE_MODEL_ID,
309
+ source: :static_contract,
310
+ details: {supported_auth_modes: SUPPORTED_AUTH_MODES}
311
+ )
312
+ end
313
+
314
+ gated_fact = MODEL_COMPATIBILITY_FACTS[normalized_model_id]
315
+ if gated_fact
316
+ minimum_version = gated_fact[:minimum_cli_version]
317
+ requirement = Gem::Requirement.new(">= #{minimum_version}")
318
+ comparable_version = comparable_cli_version(normalized_cli_version)
319
+
320
+ # A CLI-gated model without a comparable installed version must
321
+ # stay explicit. Returning :supported here would re-introduce the
322
+ # exact `gpt-5.5` failure class this contract is designed to
323
+ # prevent — a caller that cannot supply a version would get
324
+ # `supported? == true` and may still schedule a run onto an old
325
+ # CLI (e.g. 0.115.x). Surface :unknown with the requirement
326
+ # attached so callers can decide deliberately.
327
+ if comparable_version.nil?
328
+ return AgentHarness::ModelCompatibility.unknown_result(
329
+ runner: provider_name,
330
+ model_id: normalized_model_id,
331
+ auth_mode: normalized_auth_mode,
332
+ cli_version: normalized_cli_version,
333
+ reason: AgentHarness::ModelCompatibility::UNKNOWN_CLI_VERSION_REASON,
334
+ minimum_cli_version: minimum_version,
335
+ cli_version_requirement: requirement.to_s,
336
+ fallback_model_id: DEFAULT_COMPATIBLE_MODEL_ID,
337
+ source: :static_contract
338
+ )
339
+ end
340
+
341
+ unless requirement.satisfied_by?(comparable_version)
342
+ return AgentHarness::ModelCompatibility.build_result(
343
+ runner: provider_name,
344
+ model_id: normalized_model_id,
345
+ auth_mode: normalized_auth_mode,
346
+ cli_version: normalized_cli_version,
347
+ supported: false,
348
+ reason: AgentHarness::ModelCompatibility::UNSUPPORTED_CLI_VERSION_REASON,
349
+ minimum_cli_version: minimum_version,
350
+ cli_version_requirement: requirement.to_s,
351
+ fallback_model_id: DEFAULT_COMPATIBLE_MODEL_ID,
352
+ source: :static_contract
353
+ )
354
+ end
355
+
356
+ return AgentHarness::ModelCompatibility.build_result(
357
+ runner: provider_name,
358
+ model_id: normalized_model_id,
359
+ auth_mode: normalized_auth_mode,
360
+ cli_version: normalized_cli_version,
361
+ supported: true,
362
+ reason: AgentHarness::ModelCompatibility::SUPPORTED_REASON,
363
+ minimum_cli_version: minimum_version,
364
+ cli_version_requirement: requirement.to_s,
365
+ source: :static_contract
366
+ )
367
+ end
368
+
369
+ if BASELINE_SUPPORTED_MODELS.include?(normalized_model_id)
370
+ return AgentHarness::ModelCompatibility.build_result(
371
+ runner: provider_name,
372
+ model_id: normalized_model_id,
373
+ auth_mode: normalized_auth_mode,
374
+ cli_version: normalized_cli_version,
375
+ supported: true,
376
+ reason: AgentHarness::ModelCompatibility::SUPPORTED_REASON,
377
+ source: :static_contract
378
+ )
379
+ end
380
+
381
+ AgentHarness::ModelCompatibility.unknown_result(
382
+ runner: provider_name,
383
+ model_id: normalized_model_id,
384
+ auth_mode: normalized_auth_mode,
385
+ cli_version: normalized_cli_version,
386
+ reason: AgentHarness::ModelCompatibility::UNKNOWN_MODEL_REASON,
387
+ fallback_model_id: DEFAULT_COMPATIBLE_MODEL_ID
388
+ )
389
+ end
390
+
391
+ # Coerce normalized CLI version into a +Gem::Version+ usable for
392
+ # requirement comparison. Returns +nil+ when the value is missing or
393
+ # cannot be parsed — callers treat that as "no installed-version
394
+ # signal," not as a failure.
395
+ def comparable_cli_version(value)
396
+ return value if value.is_a?(Gem::Version)
397
+ return nil if value.nil?
398
+
399
+ str = value.respond_to?(:strip) ? value.strip : value.to_s
400
+ return nil if str.empty?
401
+
402
+ begin
403
+ Gem::Version.new(str)
404
+ rescue ArgumentError
405
+ nil
406
+ end
407
+ end
408
+
235
409
  def parse_cli_jsonl_transcript(raw_output, max_events: nil)
236
410
  return parser_instance.send(:parse_jsonl_output, "") if max_events && max_events <= 0
237
411
 
@@ -181,6 +181,29 @@ module AgentHarness
181
181
  end
182
182
  end
183
183
 
184
+ # Query runner/model compatibility contract for a provider.
185
+ #
186
+ # Delegates to the provider class's +.model_compatibility+ method,
187
+ # which returns a structured {AgentHarness::ModelCompatibility::Result}.
188
+ # The default implementation in {AgentHarness::Providers::Adapter}
189
+ # returns +:unknown+ for providers that have not opted in to a
190
+ # static contract.
191
+ #
192
+ # @param name [Symbol, String] the provider name or alias
193
+ # @param model_id [String, Symbol] requested model identifier
194
+ # @param auth_mode [Symbol, nil] caller's auth mode
195
+ # @param cli_version [String, Gem::Version, nil] installed CLI version
196
+ # @return [AgentHarness::ModelCompatibility::Result]
197
+ # @raise [ConfigurationError] if the provider name is not registered
198
+ def model_compatibility(name, model_id:, auth_mode: nil, cli_version: nil)
199
+ klass = get(name)
200
+ klass.model_compatibility(
201
+ model_id: model_id,
202
+ auth_mode: auth_mode,
203
+ cli_version: cli_version
204
+ )
205
+ end
206
+
184
207
  # Get smoke-test metadata for a provider.
185
208
  #
186
209
  # @param name [Symbol, String] the provider name
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module AgentHarness
4
- VERSION = "0.22.5"
4
+ VERSION = "0.23.0"
5
5
  end
data/lib/agent_harness.rb CHANGED
@@ -257,6 +257,33 @@ module AgentHarness
257
257
  Providers::Registry.instance.smoke_test_contracts
258
258
  end
259
259
 
260
+ # Query runner/model compatibility contract.
261
+ #
262
+ # Returns a structured {AgentHarness::ModelCompatibility::Result}
263
+ # describing whether the named runner can execute +model_id+ under
264
+ # the requested runtime constraints. Downstream orchestrators should
265
+ # consume this contract before validating tier models, selecting a
266
+ # runner, or scheduling work — rather than inferring compatibility
267
+ # from scattered CLI version pins, smoke-test overrides, or runtime
268
+ # error strings.
269
+ #
270
+ # @param runner [Symbol, String] provider/runner name (e.g. :codex)
271
+ # @param model_id [String, Symbol] requested model identifier
272
+ # @param auth_mode [Symbol, nil] caller's auth mode (e.g. :api_key,
273
+ # :subscription)
274
+ # @param cli_version [String, Gem::Version, nil] installed CLI
275
+ # version, when known
276
+ # @return [AgentHarness::ModelCompatibility::Result]
277
+ # @raise [ConfigurationError] if the runner is not registered
278
+ def model_compatibility(runner:, model_id:, auth_mode: nil, cli_version: nil)
279
+ Providers::Registry.instance.model_compatibility(
280
+ runner,
281
+ model_id: model_id,
282
+ auth_mode: auth_mode,
283
+ cli_version: cli_version
284
+ )
285
+ end
286
+
260
287
  # Check if authentication is valid for a provider
261
288
  # @param provider_name [Symbol] the provider name
262
289
  # @return [Boolean] true if auth is valid
@@ -395,6 +422,7 @@ require_relative "agent_harness/authentication"
395
422
  require_relative "agent_harness/provider_health_check"
396
423
  require_relative "agent_harness/release_registry"
397
424
  require_relative "agent_harness/dependency_updater"
425
+ require_relative "agent_harness/model_compatibility"
398
426
 
399
427
  # Provider layer
400
428
  require_relative "agent_harness/providers/registry"
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: agent-harness
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.22.5
4
+ version: 0.23.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Bart Agapinan
@@ -113,6 +113,7 @@ files:
113
113
  - lib/agent_harness/mcp_config_loader.rb
114
114
  - lib/agent_harness/mcp_config_translator.rb
115
115
  - lib/agent_harness/mcp_server.rb
116
+ - lib/agent_harness/model_compatibility.rb
116
117
  - lib/agent_harness/openai_compatible_transport.rb
117
118
  - lib/agent_harness/orchestration/circuit_breaker.rb
118
119
  - lib/agent_harness/orchestration/conductor.rb