agent_sessions 0.4.0 → 0.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +5 -0
- data/doc/Agent/Sessions/Readers/Codex.md +7 -7
- data/doc/CHANGELOG.md +5 -0
- data/lib/agent/sessions/readers/codex.rb +39 -26
- data/lib/agent/sessions/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 96029e1b23c71e1b1703abf8bcd1d09640a35c469da194d6e2cd0879a2da0786
|
|
4
|
+
data.tar.gz: 91f556777c0afd1af418606a8fe4fa71329688c2a6f40090e152b1cef308dbab
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: b6eaa333075f8dc86b887b7b00bf02c06591bfe158b217c0d0d0f439520eeb99a755c38b92c06e276512825fe2b912dff87b97b68a2fa980d1039fc94999ff21
|
|
7
|
+
data.tar.gz: d8c484f661db60b910a650617192737051c0f275f74363e78447bea5ee10480253285a001c661ec7cc7207a5d397181f06090d027fd16f498cece2452592b940
|
data/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,8 @@
|
|
|
1
|
+
## 0.4.1 (2026-09-10)
|
|
2
|
+
|
|
3
|
+
- Recognize Codex `token_usage_record` accounting metadata without adding conversation messages, round trips, or unknown-record warnings, including with `include_events: true`.
|
|
4
|
+
- Read Codex session usage from the latest usable cumulative totals in file order across legacy `token_count` events and new `thread_token_usage` records, without double-counting. Malformed or empty totals preserve earlier valid usage; existing token normalization and missing-value semantics are retained.
|
|
5
|
+
|
|
1
6
|
## 0.4.0 (2026-09-10)
|
|
2
7
|
|
|
3
8
|
- Round-trip grouping in every reader: `RoundTrip` (`index`, `messages`, `usage`, `recorded`, plus `parts`, `calls`, `roles`), `reader.each_round_trip` (streams), `reader.round_trips` (eager), `reader.round_trips_recorded?`
|
|
@@ -39,8 +39,8 @@ caller must learn to ignore is worse than no warning.
|
|
|
39
39
|
|
|
40
40
|
### `NON_MESSAGE_TYPES` <a id="constant-NON_MESSAGE_TYPES"></a> <a id="NON_MESSAGE_TYPES-constant"></a>
|
|
41
41
|
Known, and deliberately not messages: the session header, per-turn
|
|
42
|
-
configuration,
|
|
43
|
-
a judgement, not an oversight —
|
|
42
|
+
configuration, state records, and cumulative token accounting. Silence here is
|
|
43
|
+
a judgement, not an oversight —these are not conversation, and warning about
|
|
44
44
|
them would train a caller to ignore warnings.
|
|
45
45
|
|
|
46
46
|
### `ROLES` <a id="constant-ROLES"></a> <a id="ROLES-constant"></a>
|
|
@@ -61,11 +61,11 @@ Not documented.
|
|
|
61
61
|
## Public Instance Methods
|
|
62
62
|
### `usage()` <a id="method-i-usage"></a> <a id="usage-instance_method"></a>
|
|
63
63
|
Session totals. Codex writes no usage on its messages; it writes token_count
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
turn.
|
|
64
|
+
events with info.total_token_usage, or token_usage_record records with
|
|
65
|
+
thread_token_usage (observed 2026-09-10). Both hold cumulative totals. The
|
|
66
|
+
legacy format was verified on 2026-08-24: consecutive records report total
|
|
67
|
+
33,751 then 69,135 while their last_token_usage differ, so the latest usable
|
|
68
|
+
record is the session and summing would multiply-count every earlier turn.
|
|
69
69
|
|
|
70
70
|
Two normalizations, both from that same file:
|
|
71
71
|
|
data/doc/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,8 @@
|
|
|
1
|
+
## 0.4.1 (2026-09-10)
|
|
2
|
+
|
|
3
|
+
- Recognize Codex `token_usage_record` accounting metadata without adding conversation messages, round trips, or unknown-record warnings, including with `include_events: true`.
|
|
4
|
+
- Read Codex session usage from the latest usable cumulative totals in file order across legacy `token_count` events and new `thread_token_usage` records, without double-counting. Malformed or empty totals preserve earlier valid usage; existing token normalization and missing-value semantics are retained.
|
|
5
|
+
|
|
1
6
|
## 0.4.0 (2026-09-10)
|
|
2
7
|
|
|
3
8
|
- Round-trip grouping in every reader: `RoundTrip` (`index`, `messages`, `usage`, `recorded`, plus `parts`, `calls`, `roles`), `reader.each_round_trip` (streams), `reader.round_trips` (eager), `reader.round_trips_recorded?`
|
|
@@ -15,11 +15,12 @@ module Agent
|
|
|
15
15
|
# been inference. A reference implementation has to be falsifiable.
|
|
16
16
|
class Codex < Base
|
|
17
17
|
# Known, and deliberately not messages: the session header, per-turn
|
|
18
|
-
# configuration,
|
|
19
|
-
# here is a judgement, not an oversight —
|
|
20
|
-
# warning about them would train a
|
|
18
|
+
# configuration, state records, and cumulative token accounting.
|
|
19
|
+
# Silence here is a judgement, not an oversight —
|
|
20
|
+
# these are not conversation, and warning about them would train a
|
|
21
|
+
# caller to ignore warnings.
|
|
21
22
|
NON_MESSAGE_TYPES = %w[session_meta turn_context world_state
|
|
22
|
-
inter_agent_communication_metadata].freeze
|
|
23
|
+
inter_agent_communication_metadata token_usage_record].freeze
|
|
23
24
|
|
|
24
25
|
# "developer" is what Codex writes where the normalized vocabulary says
|
|
25
26
|
# :system. It is 101 of 292 role-bearing records in the sample, so this is
|
|
@@ -60,11 +61,12 @@ module Agent
|
|
|
60
61
|
CALL_OUTPUTS = %w[output tools].freeze
|
|
61
62
|
|
|
62
63
|
# Session totals. Codex writes no usage on its messages; it writes
|
|
63
|
-
# token_count
|
|
64
|
-
#
|
|
64
|
+
# token_count events with info.total_token_usage, or token_usage_record
|
|
65
|
+
# records with thread_token_usage (observed 2026-09-10). Both hold
|
|
66
|
+
# cumulative totals. The legacy format was verified on 2026-08-24:
|
|
65
67
|
# consecutive records report total 33,751 then 69,135 while their
|
|
66
|
-
# last_token_usage differ, so the
|
|
67
|
-
# would multiply-count every earlier turn.
|
|
68
|
+
# last_token_usage differ, so the latest usable record is the session
|
|
69
|
+
# and summing would multiply-count every earlier turn.
|
|
68
70
|
#
|
|
69
71
|
# Two normalizations, both from that same file:
|
|
70
72
|
#
|
|
@@ -79,25 +81,12 @@ module Agent
|
|
|
79
81
|
# any bucket it landed in would be double-counted by a caller summing
|
|
80
82
|
# buckets.
|
|
81
83
|
def usage
|
|
82
|
-
|
|
84
|
+
mapped = nil
|
|
83
85
|
each_record do |record, _line_number|
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
candidate = record.dig("payload", "info", "total_token_usage")
|
|
87
|
-
info = candidate if record.dig("payload", "type") == "token_count" && candidate.is_a?(Hash)
|
|
86
|
+
candidate = cumulative_usage(record)
|
|
87
|
+
mapped = candidate if candidate
|
|
88
88
|
end
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
input = count_from(info["input_tokens"])
|
|
92
|
-
cached = count_from(info["cached_input_tokens"])
|
|
93
|
-
mapped = Usage.new(input: input && cached ? [input - cached, 0].max : input,
|
|
94
|
-
output: count_from(info["output_tokens"]),
|
|
95
|
-
cache_read: cached,
|
|
96
|
-
cache_creation: count_from(info["cache_write_input_tokens"]),
|
|
97
|
-
reasoning: count_from(info["reasoning_output_tokens"]))
|
|
98
|
-
# Same rule as Claude's usage_from: a token_count record whose every
|
|
99
|
-
# field failed the count check answers nil, not an all-nil Usage.
|
|
100
|
-
mapped.to_h.each_value.any? ? mapped : nil
|
|
89
|
+
mapped
|
|
101
90
|
end
|
|
102
91
|
|
|
103
92
|
private
|
|
@@ -198,7 +187,31 @@ module Agent
|
|
|
198
187
|
def event_message(record)
|
|
199
188
|
return nil unless include_events
|
|
200
189
|
|
|
201
|
-
|
|
190
|
+
payload = record["payload"]
|
|
191
|
+
type = payload.is_a?(Hash) ? payload["type"] : nil
|
|
192
|
+
build(record, :system, [Part.new(type: :unknown, text: type)])
|
|
193
|
+
end
|
|
194
|
+
|
|
195
|
+
def cumulative_usage(record)
|
|
196
|
+
source = case record["type"]
|
|
197
|
+
when "event_msg"
|
|
198
|
+
payload = record["payload"]
|
|
199
|
+
info = payload["info"] if payload.is_a?(Hash) && payload["type"] == "token_count"
|
|
200
|
+
info["total_token_usage"] if info.is_a?(Hash)
|
|
201
|
+
when "token_usage_record"
|
|
202
|
+
payload = record["payload"]
|
|
203
|
+
payload["thread_token_usage"] if payload.is_a?(Hash)
|
|
204
|
+
end
|
|
205
|
+
return unless source.is_a?(Hash)
|
|
206
|
+
|
|
207
|
+
input = count_from(source["input_tokens"])
|
|
208
|
+
cached = count_from(source["cached_input_tokens"])
|
|
209
|
+
result = Usage.new(input: input && cached ? [input - cached, 0].max : input,
|
|
210
|
+
output: count_from(source["output_tokens"]),
|
|
211
|
+
cache_read: cached,
|
|
212
|
+
cache_creation: count_from(source["cache_write_input_tokens"]),
|
|
213
|
+
reasoning: count_from(source["reasoning_output_tokens"]))
|
|
214
|
+
result if result.to_h.each_value.any?
|
|
202
215
|
end
|
|
203
216
|
|
|
204
217
|
def unknown_message(record)
|