dspy 0.4.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 06ba0ad132367bef01b9dccd24cac12433eaed02c47b96ae78b460370b21c85b
4
- data.tar.gz: 86baa5b7e136c1a0527e880915a9dfd34ad6093927ad4979f45a0dc44b3bbd9c
3
+ metadata.gz: fd8c98014ede76d7f232ebbb1e46789efa57fdca83203548d0e9b564054d859d
4
+ data.tar.gz: f654cf96ac2976fbdd101e6d9d3b5b69346a5a32857cfcff2683e659173c9483
5
5
  SHA512:
6
- metadata.gz: f0e499582d6a3593b3e71b2bb587db6b8599fa1d50cd265cb14a9ac9eb4ea72fc8d5bfa580c67a8f8c5065eef29ef67b26010b316fb07a87e10011a5be5cd7d5
7
- data.tar.gz: 765749d9edd708965a61ecdb71c20150c934fd252b2e67913acb41bc7c1f8c33ccf96709a96530c9120dab51643997690aa073dcb3e36425eaa3ed91739895ad
6
+ metadata.gz: 425bf005503285726f84aff2009e53cfcad853afc5b11cce7b600a978e5546ddf8cddd8d9e50d9e54701c26834ff47511d34719bf1a2f78c60f8e8b17a5d92e2
7
+ data.tar.gz: 6c2f6eeb7686a9e116642f8097154aee5610cfe75542973e973b4ba851f1c7428e71f6241a9502b7489fad40a59f35921fc377aa3af008fd506a17d18043320a
data/README.md CHANGED
@@ -14,23 +14,19 @@ The result? LLM applications that actually scale and don't break when you sneeze
14
14
  - **Signatures** - Define input/output schemas using Sorbet types
15
15
  - **Predict** - Basic LLM completion with structured data
16
16
  - **Chain of Thought** - Step-by-step reasoning for complex problems
17
- - **ReAct** - Tool-using agents that can actually get things done
18
- - **RAG** - Context-enriched responses from your data
19
- - **Multi-stage Pipelines** - Compose multiple LLM calls into workflows
17
+ - **ReAct** - Tool-using agents with basic tool integration
18
+ - **Manual Composition** - Combine multiple LLM calls into workflows
20
19
 
21
20
  **Optimization & Evaluation:**
22
21
  - **Prompt Objects** - Manipulate prompts as first-class objects instead of strings
23
22
  - **Typed Examples** - Type-safe training data with automatic validation
24
- - **Evaluation Framework** - Systematic testing with built-in metrics
25
- - **MIPROv2 Optimizer** - State-of-the-art automatic prompt optimization
26
- - **Simple Optimizer** - Random/grid search for quick experimentation
23
+ - **Evaluation Framework** - Basic testing with simple metrics
24
+ - **Basic Optimization** - Simple prompt optimization techniques
27
25
 
28
26
  **Production Features:**
29
- - **Storage System** - Persistent optimization result storage with search and filtering
30
- - **Registry System** - Version control for optimized signatures with deployment tracking
27
+ - **File-based Storage** - Basic optimization result persistence
31
28
  - **Multi-Platform Observability** - OpenTelemetry, New Relic, and Langfuse integration
32
- - **Auto-deployment** - Intelligent deployment based on performance improvements
33
- - **Rollback Protection** - Automatic rollback on performance degradation
29
+ - **Basic Instrumentation** - Event tracking and logging
34
30
 
35
31
  **Developer Experience:**
36
32
  - LLM provider support using official Ruby clients:
@@ -106,21 +102,20 @@ puts result.confidence # => 0.85
106
102
  - **[Examples & Validation](docs/core-concepts/examples.md)** - Type-safe training data
107
103
 
108
104
  ### Optimization
109
- - **[Evaluation Framework](docs/optimization/evaluation.md)** - Systematic testing with metrics
105
+ - **[Evaluation Framework](docs/optimization/evaluation.md)** - Basic testing with simple metrics
110
106
  - **[Prompt Optimization](docs/optimization/prompt-optimization.md)** - Manipulate prompts as objects
111
- - **[MIPROv2 Optimizer](docs/optimization/miprov2.md)** - State-of-the-art automatic optimization
112
- - **[Simple Optimizer](docs/optimization/simple-optimizer.md)** - Quick experimentation with random/grid search
107
+ - **[MIPROv2 Optimizer](docs/optimization/miprov2.md)** - Basic automatic optimization
108
+ - **[Simple Optimizer](docs/optimization/simple-optimizer.md)** - Random search experimentation
113
109
 
114
110
  ### Production Features
115
- - **[Storage System](docs/enterprise/storage.md)** - Persist and search optimization results
116
- - **[Registry & Versions](docs/enterprise/registry.md)** - Version control with deployment tracking
117
- - **[Observability](docs/enterprise/observability.md)** - Multi-platform monitoring and metrics
111
+ - **[Storage System](docs/production/storage.md)** - Basic file-based persistence
112
+ - **[Observability](docs/production/observability.md)** - Multi-platform monitoring and metrics
118
113
 
119
114
  ### Advanced Usage
120
- - **[Complex Types](docs/advanced/complex-types.md)** - Enums, optional fields, and defaults
121
- - **[Multi-stage Pipelines](docs/advanced/pipelines.md)** - Advanced composition patterns
122
- - **[RAG Implementation](docs/advanced/rag.md)** - Retrieval Augmented Generation
123
- - **[Custom Metrics](docs/advanced/custom-metrics.md)** - Domain-specific evaluation logic
115
+ - **[Complex Types](docs/advanced/complex-types.md)** - Basic Sorbet type integration
116
+ - **[Manual Pipelines](docs/advanced/pipelines.md)** - Manual module composition patterns
117
+ - **[RAG Patterns](docs/advanced/rag.md)** - Manual RAG implementation with external services
118
+ - **[Custom Metrics](docs/advanced/custom-metrics.md)** - Proc-based evaluation logic
124
119
 
125
120
  ## What's Next
126
121
 
@@ -5,49 +5,22 @@ require 'sorbet-runtime'
5
5
  require_relative 'predict'
6
6
  require_relative 'signature'
7
7
  require_relative 'instrumentation'
8
+ require_relative 'mixins/struct_builder'
8
9
 
9
10
  module DSPy
10
11
  # Enhances prediction by encouraging step-by-step reasoning
11
12
  # before providing a final answer using Sorbet signatures.
12
13
  class ChainOfThought < Predict
13
14
  extend T::Sig
15
+ include Mixins::StructBuilder
14
16
 
15
17
  FieldDescriptor = DSPy::Signature::FieldDescriptor
16
18
 
17
19
  sig { params(signature_class: T.class_of(DSPy::Signature)).void }
18
20
  def initialize(signature_class)
19
21
  @original_signature = signature_class
20
-
21
- # Create enhanced output struct with reasoning
22
- enhanced_output_struct = create_enhanced_output_struct(signature_class)
23
-
24
- # Create enhanced signature class
25
- enhanced_signature = Class.new(DSPy::Signature) do
26
- # Set the description
27
- description "#{signature_class.description} Think step by step."
28
-
29
- # Use the same input struct and copy field descriptors
30
- @input_struct_class = signature_class.input_struct_class
31
- @input_field_descriptors = signature_class.instance_variable_get(:@input_field_descriptors) || {}
32
-
33
- # Use the enhanced output struct and create field descriptors for it
34
- @output_struct_class = enhanced_output_struct
35
-
36
- # Create field descriptors for the enhanced output struct
37
- @output_field_descriptors = {}
38
-
39
- # Copy original output field descriptors
40
- original_output_descriptors = signature_class.instance_variable_get(:@output_field_descriptors) || {}
41
- @output_field_descriptors.merge!(original_output_descriptors)
42
-
43
- # Add reasoning field descriptor (ChainOfThought always provides this)
44
- @output_field_descriptors[:reasoning] = FieldDescriptor.new(String, "Step by step reasoning process")
45
-
46
- class << self
47
- attr_reader :input_struct_class, :output_struct_class
48
- end
49
- end
50
-
22
+ enhanced_signature = build_enhanced_signature(signature_class)
23
+
51
24
  # Call parent constructor with enhanced signature
52
25
  super(enhanced_signature)
53
26
  @signature_class = enhanced_signature
@@ -81,9 +54,8 @@ module DSPy
81
54
 
82
55
  sig { override.params(instruction: String).returns(ChainOfThought) }
83
56
  def with_instruction(instruction)
84
- # Ensure ChainOfThought behavior is preserved
85
- cot_instruction = instruction.include?("Think step by step") ? instruction : "#{instruction} Think step by step."
86
- super(cot_instruction)
57
+ enhanced_instruction = ensure_chain_of_thought_instruction(instruction)
58
+ super(enhanced_instruction)
87
59
  end
88
60
 
89
61
  sig { override.params(examples: T::Array[FewShotExample]).returns(ChainOfThought) }
@@ -113,43 +85,96 @@ module DSPy
113
85
  # Override forward_untyped to add ChainOfThought-specific instrumentation
114
86
  sig { override.params(input_values: T.untyped).returns(T.untyped) }
115
87
  def forward_untyped(**input_values)
116
- # Prepare instrumentation payload
117
- input_fields = input_values.keys.map(&:to_s)
118
-
119
- # Instrument ChainOfThought lifecycle
120
- result = Instrumentation.instrument('dspy.chain_of_thought', {
121
- signature_class: @original_signature.name,
122
- model: lm.model,
123
- provider: lm.provider,
124
- input_fields: input_fields
125
- }) do
88
+ instrument_prediction('dspy.chain_of_thought', @original_signature, input_values) do
126
89
  # Call parent prediction logic
127
90
  prediction_result = super(**input_values)
128
91
 
129
92
  # Analyze reasoning if present
130
- if prediction_result.respond_to?(:reasoning) && prediction_result.reasoning
131
- reasoning_content = prediction_result.reasoning.to_s
132
- reasoning_length = reasoning_content.length
133
- reasoning_steps = count_reasoning_steps(reasoning_content)
134
-
135
- # Emit reasoning analysis event
136
- Instrumentation.emit('dspy.chain_of_thought.reasoning_complete', {
137
- signature_class: @original_signature.name,
138
- reasoning_steps: reasoning_steps,
139
- reasoning_length: reasoning_length,
140
- has_reasoning: !reasoning_content.empty?
141
- })
142
- end
93
+ analyze_reasoning(prediction_result)
143
94
 
144
95
  prediction_result
145
96
  end
146
-
147
- result
148
97
  end
149
98
 
150
99
  private
151
100
 
101
+ # Builds enhanced signature with reasoning capabilities
102
+ sig { params(signature_class: T.class_of(DSPy::Signature)).returns(T.class_of(DSPy::Signature)) }
103
+ def build_enhanced_signature(signature_class)
104
+ enhanced_output_struct = create_enhanced_output_struct(signature_class)
105
+ create_signature_class(signature_class, enhanced_output_struct)
106
+ end
107
+
108
+ # Creates signature class with enhanced description and reasoning field
109
+ sig { params(signature_class: T.class_of(DSPy::Signature), enhanced_output_struct: T.class_of(T::Struct)).returns(T.class_of(DSPy::Signature)) }
110
+ def create_signature_class(signature_class, enhanced_output_struct)
111
+ Class.new(DSPy::Signature) do
112
+ description "#{signature_class.description} Think step by step."
113
+
114
+ # Use the same input struct and copy field descriptors
115
+ @input_struct_class = signature_class.input_struct_class
116
+ @input_field_descriptors = signature_class.instance_variable_get(:@input_field_descriptors) || {}
117
+
118
+ # Use the enhanced output struct and create field descriptors for it
119
+ @output_struct_class = enhanced_output_struct
120
+
121
+ # Create field descriptors for the enhanced output struct
122
+ @output_field_descriptors = {}
123
+
124
+ # Copy original output field descriptors
125
+ original_output_descriptors = signature_class.instance_variable_get(:@output_field_descriptors) || {}
126
+ @output_field_descriptors.merge!(original_output_descriptors)
127
+
128
+ # Add reasoning field descriptor (ChainOfThought always provides this)
129
+ @output_field_descriptors[:reasoning] = FieldDescriptor.new(String, "Step by step reasoning process")
130
+
131
+ class << self
132
+ attr_reader :input_struct_class, :output_struct_class
133
+ end
134
+ end
135
+ end
136
+
137
+ # Creates enhanced output struct with reasoning field
138
+ sig { params(signature_class: T.class_of(DSPy::Signature)).returns(T.class_of(T::Struct)) }
139
+ def create_enhanced_output_struct(signature_class)
140
+ output_props = signature_class.output_struct_class.props
141
+
142
+ build_enhanced_struct(
143
+ { output: output_props },
144
+ { reasoning: [String, "Step by step reasoning process"] }
145
+ )
146
+ end
147
+
148
+ # Ensures instruction includes chain of thought prompt
149
+ sig { params(instruction: String).returns(String) }
150
+ def ensure_chain_of_thought_instruction(instruction)
151
+ instruction.include?("Think step by step") ? instruction : "#{instruction} Think step by step."
152
+ end
153
+
154
+ # Analyzes reasoning in prediction result and emits instrumentation events
155
+ sig { params(prediction_result: T.untyped).void }
156
+ def analyze_reasoning(prediction_result)
157
+ return unless prediction_result.respond_to?(:reasoning) && prediction_result.reasoning
158
+
159
+ reasoning_content = prediction_result.reasoning.to_s
160
+ return if reasoning_content.empty?
161
+
162
+ emit_reasoning_analysis(reasoning_content)
163
+ end
164
+
165
+ # Emits reasoning analysis instrumentation event
166
+ sig { params(reasoning_content: String).void }
167
+ def emit_reasoning_analysis(reasoning_content)
168
+ Instrumentation.emit('dspy.chain_of_thought.reasoning_complete', {
169
+ signature_class: @original_signature.name,
170
+ reasoning_steps: count_reasoning_steps(reasoning_content),
171
+ reasoning_length: reasoning_content.length,
172
+ has_reasoning: true
173
+ })
174
+ end
175
+
152
176
  # Count reasoning steps by looking for step indicators
177
+ sig { params(reasoning_text: String).returns(Integer) }
153
178
  def count_reasoning_steps(reasoning_text)
154
179
  return 0 if reasoning_text.nil? || reasoning_text.empty?
155
180
 
@@ -170,50 +195,5 @@ module DSPy
170
195
  # Fallback: count sentences if no clear steps
171
196
  max_count > 0 ? max_count : reasoning_text.split(/[.!?]+/).reject(&:empty?).length
172
197
  end
173
-
174
- sig { params(signature_class: T.class_of(DSPy::Signature)).returns(T.class_of(T::Struct)) }
175
- def create_enhanced_output_struct(signature_class)
176
- # Get original output props
177
- original_props = signature_class.output_struct_class.props
178
-
179
- # Create new struct class with reasoning added
180
- Class.new(T::Struct) do
181
- # Add all original fields
182
- original_props.each do |name, prop|
183
- # Extract the type and other options
184
- type = prop[:type]
185
- options = prop.except(:type, :type_object, :accessor_key, :sensitivity, :redaction)
186
-
187
- # Handle default values
188
- if options[:default]
189
- const name, type, default: options[:default]
190
- elsif options[:factory]
191
- const name, type, factory: options[:factory]
192
- else
193
- const name, type
194
- end
195
- end
196
-
197
- # Add reasoning field (ChainOfThought always provides this)
198
- const :reasoning, String
199
-
200
- # Add to_h method to serialize the struct to a hash
201
- define_method :to_h do
202
- hash = {}
203
-
204
- # Start with input values if available
205
- if self.instance_variable_defined?(:@input_values)
206
- hash.merge!(self.instance_variable_get(:@input_values))
207
- end
208
-
209
- # Then add output properties
210
- self.class.props.keys.each do |key|
211
- hash[key] = self.send(key)
212
- end
213
-
214
- hash
215
- end
216
- end
217
- end
218
198
  end
219
199
  end
@@ -28,9 +28,9 @@ module DSPy
28
28
  return {} unless usage.is_a?(Hash)
29
29
 
30
30
  {
31
- tokens_input: usage[:prompt_tokens] || usage['prompt_tokens'],
32
- tokens_output: usage[:completion_tokens] || usage['completion_tokens'],
33
- tokens_total: usage[:total_tokens] || usage['total_tokens']
31
+ input_tokens: usage[:prompt_tokens] || usage['prompt_tokens'],
32
+ output_tokens: usage[:completion_tokens] || usage['completion_tokens'],
33
+ total_tokens: usage[:total_tokens] || usage['total_tokens']
34
34
  }
35
35
  end
36
36
 
@@ -44,9 +44,9 @@ module DSPy
44
44
  output_tokens = usage[:output_tokens] || usage['output_tokens'] || 0
45
45
 
46
46
  {
47
- tokens_input: input_tokens,
48
- tokens_output: output_tokens,
49
- tokens_total: input_tokens + output_tokens
47
+ input_tokens: input_tokens,
48
+ output_tokens: output_tokens,
49
+ total_tokens: input_tokens + output_tokens
50
50
  }
51
51
  end
52
52
  end
@@ -9,27 +9,27 @@ module DSPy
9
9
  # Provides extension points for logging, OpenTelemetry, New Relic, Langfuse, and custom monitoring
10
10
  module Instrumentation
11
11
  # Get a logger subscriber instance (creates new instance each time)
12
- def self.logger_subscriber
12
+ def self.logger_subscriber(**options)
13
13
  require_relative 'subscribers/logger_subscriber'
14
- DSPy::Subscribers::LoggerSubscriber.new
14
+ DSPy::Subscribers::LoggerSubscriber.new(**options)
15
15
  end
16
16
 
17
17
  # Get an OpenTelemetry subscriber instance (creates new instance each time)
18
- def self.otel_subscriber
18
+ def self.otel_subscriber(**options)
19
19
  require_relative 'subscribers/otel_subscriber'
20
- DSPy::Subscribers::OtelSubscriber.new
20
+ DSPy::Subscribers::OtelSubscriber.new(**options)
21
21
  end
22
22
 
23
23
  # Get a New Relic subscriber instance (creates new instance each time)
24
- def self.newrelic_subscriber
24
+ def self.newrelic_subscriber(**options)
25
25
  require_relative 'subscribers/newrelic_subscriber'
26
- DSPy::Subscribers::NewrelicSubscriber.new
26
+ DSPy::Subscribers::NewrelicSubscriber.new(**options)
27
27
  end
28
28
 
29
29
  # Get a Langfuse subscriber instance (creates new instance each time)
30
- def self.langfuse_subscriber
30
+ def self.langfuse_subscriber(**options)
31
31
  require_relative 'subscribers/langfuse_subscriber'
32
- DSPy::Subscribers::LangfuseSubscriber.new
32
+ DSPy::Subscribers::LangfuseSubscriber.new(**options)
33
33
  end
34
34
 
35
35
  def self.notifications
@@ -115,9 +115,8 @@ module DSPy
115
115
  enhanced_payload = payload.merge(
116
116
  duration_ms: ((end_time - start_time) * 1000).round(2),
117
117
  cpu_time_ms: ((end_cpu - start_cpu) * 1000).round(2),
118
- status: 'success',
119
- timestamp: Time.now.iso8601
120
- )
118
+ status: 'success'
119
+ ).merge(generate_timestamp)
121
120
 
122
121
  self.emit_event(event_name, enhanced_payload)
123
122
  result
@@ -130,9 +129,8 @@ module DSPy
130
129
  cpu_time_ms: ((end_cpu - start_cpu) * 1000).round(2),
131
130
  status: 'error',
132
131
  error_type: error.class.name,
133
- error_message: error.message,
134
- timestamp: Time.now.iso8601
135
- )
132
+ error_message: error.message
133
+ ).merge(generate_timestamp)
136
134
 
137
135
  self.emit_event(event_name, error_payload)
138
136
  raise
@@ -145,9 +143,8 @@ module DSPy
145
143
  payload ||= {}
146
144
 
147
145
  enhanced_payload = payload.merge(
148
- timestamp: Time.now.iso8601,
149
146
  status: payload[:status] || 'success'
150
- )
147
+ ).merge(generate_timestamp)
151
148
 
152
149
  self.emit_event(event_name, enhanced_payload)
153
150
  end
@@ -175,7 +172,102 @@ module DSPy
175
172
  end
176
173
 
177
174
  def self.setup_subscribers
178
- # Lazy initialization - will be created when first accessed
175
+ config = DSPy.config.instrumentation
176
+
177
+ # Return early if instrumentation is disabled
178
+ return unless config.enabled
179
+
180
+ # Validate configuration first
181
+ DSPy.validate_instrumentation!
182
+
183
+ # Setup each configured subscriber
184
+ config.subscribers.each do |subscriber_type|
185
+ setup_subscriber(subscriber_type)
186
+ end
187
+ end
188
+
189
+ def self.setup_subscriber(subscriber_type)
190
+ case subscriber_type
191
+ when :logger
192
+ setup_logger_subscriber
193
+ when :otel
194
+ setup_otel_subscriber if otel_available?
195
+ when :newrelic
196
+ setup_newrelic_subscriber if newrelic_available?
197
+ when :langfuse
198
+ setup_langfuse_subscriber if langfuse_available?
199
+ else
200
+ raise ArgumentError, "Unknown subscriber type: #{subscriber_type}"
201
+ end
202
+ rescue LoadError => e
203
+ DSPy.logger.warn "Failed to setup #{subscriber_type} subscriber: #{e.message}"
204
+ end
205
+
206
+ def self.setup_logger_subscriber
207
+ # Create subscriber - it will read configuration when handling events
208
+ logger_subscriber
209
+ end
210
+
211
+ def self.setup_otel_subscriber
212
+ # Create subscriber - it will read configuration when handling events
213
+ otel_subscriber
214
+ end
215
+
216
+ def self.setup_newrelic_subscriber
217
+ # Create subscriber - it will read configuration when handling events
218
+ newrelic_subscriber
219
+ end
220
+
221
+ def self.setup_langfuse_subscriber
222
+ # Create subscriber - it will read configuration when handling events
223
+ langfuse_subscriber
224
+ end
225
+
226
+ # Dependency checking methods
227
+ def self.otel_available?
228
+ begin
229
+ require 'opentelemetry/sdk'
230
+ true
231
+ rescue LoadError
232
+ false
233
+ end
234
+ end
235
+
236
+ def self.newrelic_available?
237
+ begin
238
+ require 'newrelic_rpm'
239
+ true
240
+ rescue LoadError
241
+ false
242
+ end
243
+ end
244
+
245
+ def self.langfuse_available?
246
+ begin
247
+ require 'langfuse'
248
+ true
249
+ rescue LoadError
250
+ false
251
+ end
252
+ end
253
+
254
+ # Generate timestamp in the configured format
255
+ def self.generate_timestamp
256
+ case DSPy.config.instrumentation.timestamp_format
257
+ when DSPy::TimestampFormat::ISO8601
258
+ { timestamp: Time.now.iso8601 }
259
+ when DSPy::TimestampFormat::RFC3339_NANO
260
+ { timestamp: Time.now.strftime('%Y-%m-%dT%H:%M:%S.%9N%z') }
261
+ when DSPy::TimestampFormat::UNIX_NANO
262
+ { timestamp_ns: (Time.now.to_f * 1_000_000_000).to_i }
263
+ else
264
+ { timestamp: Time.now.iso8601 } # Fallback to iso8601
265
+ end
266
+ end
267
+
268
+ # Legacy setup method for backward compatibility
269
+ def self.setup_subscribers_legacy
270
+ # Legacy initialization - will be created when first accessed
179
271
  # Force initialization of enabled subscribers
180
272
  logger_subscriber
181
273
 
data/lib/dspy/lm.rb CHANGED
@@ -39,38 +39,52 @@ module DSPy
39
39
  input_text = messages.map { |m| m[:content] }.join(' ')
40
40
  input_size = input_text.length
41
41
 
42
- # Instrument LM request
43
- response = Instrumentation.instrument('dspy.lm.request', {
44
- gen_ai_operation_name: 'chat',
45
- gen_ai_system: provider,
46
- gen_ai_request_model: model,
47
- signature_class: signature_class.name,
48
- provider: provider,
49
- adapter_class: adapter.class.name,
50
- input_size: input_size
51
- }) do
52
- adapter.chat(messages: messages, &block)
53
- end
42
+ # Check trace level to decide instrumentation strategy
43
+ trace_level = DSPy.config.instrumentation.trace_level
54
44
 
55
- # Extract actual token usage from response (more accurate than estimation)
56
- token_usage = Instrumentation::TokenTracker.extract_token_usage(response, provider)
45
+ # Extract token usage and prepare consolidated payload
46
+ response = nil
47
+ token_usage = {}
57
48
 
58
- # Emit token usage event if available
59
- if token_usage.any?
60
- Instrumentation.emit('dspy.lm.tokens', token_usage.merge({
49
+ if should_emit_lm_events?(trace_level)
50
+ # Detailed mode: emit all LM events as before
51
+ response = Instrumentation.instrument('dspy.lm.request', {
52
+ gen_ai_operation_name: 'chat',
61
53
  gen_ai_system: provider,
62
54
  gen_ai_request_model: model,
63
- signature_class: signature_class.name
64
- }))
65
- end
66
-
67
- # Instrument response parsing
68
- parsed_result = Instrumentation.instrument('dspy.lm.response.parsed', {
69
- signature_class: signature_class.name,
70
- provider: provider,
71
- response_length: response.content&.length || 0
72
- }) do
73
- parse_response(response, input_values, signature_class)
55
+ signature_class: signature_class.name,
56
+ provider: provider,
57
+ adapter_class: adapter.class.name,
58
+ input_size: input_size
59
+ }) do
60
+ adapter.chat(messages: messages, &block)
61
+ end
62
+
63
+ # Extract actual token usage from response (more accurate than estimation)
64
+ token_usage = Instrumentation::TokenTracker.extract_token_usage(response, provider)
65
+
66
+ # Emit token usage event if available
67
+ if token_usage.any?
68
+ Instrumentation.emit('dspy.lm.tokens', token_usage.merge({
69
+ gen_ai_system: provider,
70
+ gen_ai_request_model: model,
71
+ signature_class: signature_class.name
72
+ }))
73
+ end
74
+
75
+ # Instrument response parsing
76
+ parsed_result = Instrumentation.instrument('dspy.lm.response.parsed', {
77
+ signature_class: signature_class.name,
78
+ provider: provider,
79
+ response_length: response.content&.length || 0
80
+ }) do
81
+ parse_response(response, input_values, signature_class)
82
+ end
83
+ else
84
+ # Consolidated mode: execute without nested instrumentation
85
+ response = adapter.chat(messages: messages, &block)
86
+ token_usage = Instrumentation::TokenTracker.extract_token_usage(response, provider)
87
+ parsed_result = parse_response(response, input_values, signature_class)
74
88
  end
75
89
 
76
90
  parsed_result
@@ -78,6 +92,38 @@ module DSPy
78
92
 
79
93
  private
80
94
 
95
+ # Determines if LM-level events should be emitted based on trace level
96
+ def should_emit_lm_events?(trace_level)
97
+ case trace_level
98
+ when :minimal
99
+ false # Never emit LM events in minimal mode
100
+ when :standard
101
+ # In standard mode, emit LM events only if we're not in a nested context
102
+ !is_nested_context?
103
+ when :detailed
104
+ true # Always emit LM events in detailed mode
105
+ else
106
+ true
107
+ end
108
+ end
109
+
110
+ # Determines if we're in a nested context where higher-level events are being emitted
111
+ def is_nested_context?
112
+ caller_locations = caller_locations(1, 30)
113
+ return false if caller_locations.nil?
114
+
115
+ # Look for higher-level DSPy modules in the call stack
116
+ # We consider ChainOfThought and ReAct as higher-level modules
117
+ higher_level_modules = caller_locations.select do |loc|
118
+ loc.path.include?('chain_of_thought') ||
119
+ loc.path.include?('re_act') ||
120
+ loc.path.include?('react')
121
+ end
122
+
123
+ # If we have higher-level modules in the call stack, we're in a nested context
124
+ higher_level_modules.any?
125
+ end
126
+
81
127
  def parse_model_id(model_id)
82
128
  unless model_id.include?('/')
83
129
  raise ArgumentError, "model_id must include provider (e.g., 'openai/gpt-4', 'anthropic/claude-3'). Legacy format without provider is no longer supported."