langextract 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0aeb0a461910ee77f31dade0f57e4007a6525e8c4a8a8f8791f7a58a1b472926
4
- data.tar.gz: c96a48d1f86d5257f5107a67b03a57b30e051ce877fd52b9d4b352fe54b57ed4
3
+ metadata.gz: 18f4df67c5283fcde932a3a75aa0d4c0a02978b90a56d68f44219dc89a8081fe
4
+ data.tar.gz: d0cc9f1c38f50535e365adbabfa91ee31d69713716a3ec0fd13f9f05b98cb606
5
5
  SHA512:
6
- metadata.gz: 6c86ace0907fd9260cc93aad98c540aa269b5d6944d17d694775bc362d88865b6892b07fd1bc7a02794c25448ae724cd76b404c84274e427953ed6b092788565
7
- data.tar.gz: af118cf15965bb7ef4d03fd8a2a5a07ba1b7631c84d346be42217e0d44bf0cafdc006dbceda56b255387eed87461c257680a197cdeb8662083fd2c2472043b85
6
+ metadata.gz: deb42f645fabcea65211bc2bc12348f6ba278fa3e1213c8d54c548b1006515e1d442c6aea9a09bb88431401812acaf1400a01aa97f23403af8977f9f59f370c1
7
+ data.tar.gz: d3d9f43beef6ace15dd65c24250bb991066d5b6f0d7dbf3a073e36f087f157e02633f1273b7476d738b8ea9ea1294b5b12718eedd3dba0d27d0d99a750ca4a06
data/CHANGELOG.md CHANGED
@@ -2,6 +2,40 @@
2
2
 
3
3
  All notable changes to this project will be documented in this file.
4
4
 
5
+ ## [0.3.0] - 2026-07-11
6
+
7
+ ### Fixed
8
+ - Resolver: the case-insensitive alignment fallback derived offsets from a downcased copy of the source text, producing invalid intervals for characters whose lowercase form changes length (e.g. "İ") — an extraction could be marked `exact` while pointing at the wrong span. Matching now runs case-insensitively against the original text, so returned offsets are always valid.
9
+ - Resolver: the case-insensitive fallback found only the first occurrence; repeated extractions (e.g. two `alpha` against `ALPHA ALPHA`) now align to distinct spans instead of colliding on the first one.
10
+ - A bare `Hash` passed as `documents:` to `LangExtract.extract` (or to `LangExtract::IO.save_annotated_documents`) was silently split into key-value pairs and corrupted into multiple bogus documents. It is now treated as a single document.
11
+ - Chunking: a whitespace boundary exactly at the buffer limit produced chunks of `max_char_buffer + 1` characters. Split chunks now never exceed `max_char_buffer`.
12
+ - Extraction merging deduplicated by span/text/class only, discarding extractions that differed solely in `attributes` or `group_id`. Both are now part of the merge key; genuinely identical extractions still deduplicate (first wins).
13
+ - Value objects are now deeply immutable: nested hashes in `Extraction#attributes`, `Document#metadata`, and example data are frozen recursively (previously only the outer hash and arrays were frozen, so nested data could be mutated after construction).
14
+
15
+ ### Changed
16
+ - **Breaking:** `LangExtract.extract` and `Extractor#initialize` now validate configuration at construction and raise `ArgumentError` for invalid values: `extraction_passes` and `max_char_buffer` must be Integers >= 1, `context_window_chars` an Integer >= 0, `fuzzy_threshold` Numeric within 0.0..1.0, `prompt_description` a non-empty String, `examples` enumerable (nil coerces to `[]`), and `format`/`prompt_validation` must be valid modes. Previously e.g. `extraction_passes: 0` silently made zero provider calls and returned empty results.
17
+ - **Breaking:** The minimum Ruby version is now 4.0.5 (`required_ruby_version = ">= 4.0.5"`), matching what CI actually tests. Previously the gem claimed `>= 4.0` without verifying older 4.0.x releases.
18
+ - Gem packaging now includes only git-tracked files (`git ls-files --cached`); untracked working-directory files can no longer leak into locally built gems. A packaging test guards the manifest.
19
+ - JSONL loading streams the file with `File.foreach` instead of reading it fully into memory.
20
+ - README: the source-grounding feature description no longer claims every extraction carries offsets — grounded extractions include character and token offsets, while ungrounded results retain an explicit alignment status.
21
+
22
+ Note: 0.2.0 was tagged but never published to RubyGems.org; 0.3.0 is the first published release containing both changelogs' changes since 0.1.0.
23
+
24
+ ## [0.2.0] - 2026-07-11
25
+
26
+ ### Added
27
+ - Provider error taxonomy under `LangExtract::Core::ProviderError`: `ProviderAuthError`, `ProviderRateLimitError`, `ProviderTimeoutError`, `ProviderResponseError`, and `ProviderConfigError`.
28
+ - Configured logging is now used: suppressed parse errors are logged at `warn` and per-document extraction summaries at `debug`. Set `logger` to `nil` to disable logging. `Rails.logger` is picked up only when actually present.
29
+
30
+ ### Changed
31
+ - **Breaking:** The minimum Ruby version is now 4.0. CI tests Ruby 4.0.5.
32
+ - **Breaking:** Previously all provider errors were wrapped in `ProviderConfigError`. Now only recognized RubyLLM and timeout errors are mapped into the `ProviderError` taxonomy, with the original exception preserved as `#cause`; unknown errors such as `SocketError` and `Faraday::ConnectionFailed` propagate unchanged. `rescue LangExtract::ProviderConfigError` is no longer a catch-all; rescue `LangExtract::ProviderError` plus `StandardError` as needed.
33
+ - Improved resolver performance on large documents by reusing a resolver instance per document, pruning fuzzy-alignment candidates with similarity upper bounds, and exiting early on acceptable perfect matches. Fuzzy alignment now prefers matches inside the chunk's preferred interval before considering the rest of the document, which can select a chunk-local match over a marginally higher-scoring match elsewhere. This matches upstream LangExtract's chunk-local alignment semantics; previously all candidates were pooled and ranked globally by score.
34
+
35
+ ### Fixed
36
+ - JSONL IO and fixture reads now force UTF-8 encoding, so the gem works under C/POSIX locales. Encoding failures raise `LangExtract::Core::IOFailure`.
37
+ - Lazy initialization of `LangExtract.config` and `Factory.router` is now thread-safe.
38
+
5
39
  ## [0.1.0] - 2026-04-19
6
40
 
7
41
  ### Added
data/README.md CHANGED
@@ -11,7 +11,7 @@ Use it when a Ruby or Rails app needs structured LLM output that can be traced b
11
11
 
12
12
  ## Features
13
13
 
14
- - **Source grounding** — every extraction includes character and token offsets back to the original text
14
+ - **Source grounding** — grounded extractions include character and token offsets back to the original text, while ungrounded results retain an explicit alignment status
15
15
  - **Structured outputs** — deterministic, serializable result objects with alignment status
16
16
  - **Long-document chunking** — sentence-aware chunking with sequential multi-pass extraction
17
17
  - **Interactive visualization** — self-contained HTML highlighting of extraction spans
@@ -20,8 +20,8 @@ Use it when a Ruby or Rails app needs structured LLM output that can be traced b
20
20
 
21
21
  ## Requirements
22
22
 
23
- - Ruby >= 3.4.5
24
- - Tested on Ruby 3.4.5 and 4.0.2
23
+ - Ruby >= 4.0.5
24
+ - Tested on Ruby 4.0.5
25
25
  - Optional live inference adapter: `ruby_llm` >= 1.0 when using `LangExtract::Factory.create_model`
26
26
 
27
27
  ## Installation
@@ -172,12 +172,20 @@ LangExtract::Core::FormatHandler.new.parse(model_output, schema: schema)
172
172
 
173
173
  ## Error handling
174
174
 
175
+ Rescue `LangExtract::ProviderError` for all recognized provider failures, or rescue a specific subclass:
176
+
177
+ - `LangExtract::ProviderAuthError` for authentication and authorization failures
178
+ - `LangExtract::ProviderRateLimitError` for provider throttling
179
+ - `LangExtract::ProviderTimeoutError` for request timeouts
180
+ - `LangExtract::ProviderResponseError` for invalid or unsuccessful provider responses
181
+ - `LangExtract::ProviderConfigError` for invalid provider configuration
182
+
175
183
  ```ruby
176
184
  begin
177
185
  LangExtract.extract(...)
178
186
  rescue LangExtract::InvalidModelConfigError => e
179
187
  warn "Invalid model configuration: #{e.message}"
180
- rescue LangExtract::ProviderConfigError => e
188
+ rescue LangExtract::ProviderError => e
181
189
  warn "Provider failed: #{e.message}"
182
190
  rescue LangExtract::PromptValidationError, LangExtract::FormatParsingError => e
183
191
  warn e.message
@@ -188,6 +196,20 @@ rescue LangExtract::IOFailure => e
188
196
  end
189
197
  ```
190
198
 
199
+ Unknown errors propagate. Mapped provider errors preserve the original exception as `#cause`.
200
+
201
+ ## Logging
202
+
203
+ ```ruby
204
+ LangExtract.configure { |config| config.logger = Logger.new($stdout) }
205
+ ```
206
+
207
+ Set `logger` to `nil` to disable logging. `Rails.logger` is auto-detected when present.
208
+
209
+ ## Thread safety
210
+
211
+ Configuration and provider router initialization are mutex-guarded.
212
+
191
213
  ## API reference
192
214
 
193
215
  - Upstream project: [google/langextract](https://github.com/google/langextract)
@@ -14,7 +14,9 @@ module LangExtract
14
14
  private
15
15
 
16
16
  def default_logger
17
- defined?(Rails) ? Rails.logger : Logger.new($stderr)
17
+ return Rails.logger if defined?(Rails) && Rails.respond_to?(:logger) && Rails.logger
18
+
19
+ Logger.new($stderr, progname: "langextract").tap { |logger| logger.level = Logger::WARN }
18
20
  end
19
21
  end
20
22
  end
@@ -84,7 +84,7 @@ module LangExtract
84
84
 
85
85
  def split_position(text, start_pos)
86
86
  limit = [start_pos + max_char_buffer, text.length].min
87
- whitespace = text.rindex(/\s/, limit)
87
+ whitespace = text.rindex(/\s/, limit - 1)
88
88
  return limit unless whitespace && whitespace > start_pos
89
89
 
90
90
  whitespace + 1
@@ -12,7 +12,7 @@ module LangExtract
12
12
  def stringify_keys(hash)
13
13
  (hash || {}).each_with_object({}) do |(key, value), result|
14
14
  result[key.to_s] = coerce_value(value)
15
- end
15
+ end.freeze
16
16
  end
17
17
 
18
18
  def coerce_value(value)
@@ -4,7 +4,12 @@ module LangExtract
4
4
  module Core
5
5
  class Error < StandardError; end
6
6
  class InvalidModelConfigError < Error; end
7
- class ProviderConfigError < Error; end
7
+ class ProviderError < Error; end
8
+ class ProviderConfigError < ProviderError; end
9
+ class ProviderAuthError < ProviderError; end
10
+ class ProviderRateLimitError < ProviderError; end
11
+ class ProviderTimeoutError < ProviderError; end
12
+ class ProviderResponseError < ProviderError; end
8
13
  class FormatParsingError < Error; end
9
14
  class PromptValidationError < Error; end
10
15
  class AlignmentError < Error; end
@@ -5,6 +5,20 @@ require_relative "tokenizer"
5
5
 
6
6
  module LangExtract
7
7
  module Core
8
+ module SimilarityUpperBound
9
+ module_function
10
+
11
+ def passes?(target, candidate, target_counts, threshold)
12
+ denominator = target.length + candidate.length
13
+ length_matches = [target.length, candidate.length].min
14
+ return false if (2.0 * length_matches / denominator) < threshold
15
+
16
+ candidate_counts = candidate.each_char.tally
17
+ character_matches = target_counts.sum { |character, count| [count, candidate_counts[character].to_i].min }
18
+ (2.0 * character_matches / denominator) >= threshold
19
+ end
20
+ end
21
+
8
22
  class Resolver
9
23
  DEFAULT_FUZZY_THRESHOLD = 0.78
10
24
  MAX_FUZZY_CANDIDATE_STARTS = 4_000
@@ -61,40 +75,36 @@ module LangExtract
61
75
  end
62
76
 
63
77
  def find_exact(extraction_text, preferred_interval, occupied)
64
- intervals = candidate_search_ranges(preferred_interval).flat_map do |range|
65
- exact_intervals_in_range(extraction_text, range)
66
- end
67
- intervals = intervals.uniq
68
- return nil if intervals.empty?
78
+ overlap_fallback = nil
69
79
 
70
- first_non_overlap = intervals.find { |interval| allow_overlaps || !overlaps_any?(interval, occupied) }
71
- return [first_non_overlap, AlignmentStatus::EXACT] if first_non_overlap
80
+ candidate_search_ranges(preferred_interval).each do |range|
81
+ intervals = exact_intervals_in_range(extraction_text, range).uniq
82
+ overlap_fallback ||= intervals.first
83
+ non_overlap = intervals.find { |interval| allow_overlaps || !overlaps_any?(interval, occupied) }
84
+ return [non_overlap, AlignmentStatus::EXACT] if non_overlap
85
+ end
72
86
 
73
- [intervals.first, AlignmentStatus::OVERLAP]
87
+ overlap_fallback && [overlap_fallback, AlignmentStatus::OVERLAP]
74
88
  end
75
89
 
76
90
  def exact_intervals_in_range(extraction_text, range)
77
91
  intervals = []
78
92
  cursor = range.begin
79
-
80
- while cursor < range.end
93
+ range_end = range.end
94
+ while cursor < range_end
81
95
  match_pos = text.index(extraction_text, cursor)
82
- break unless match_pos && match_pos < range.end
96
+ break unless match_pos && match_pos < range_end
83
97
 
84
98
  end_pos = match_pos + extraction_text.length
85
- intervals << CharInterval.new(start_pos: match_pos, end_pos: end_pos) if end_pos <= range.end
99
+ intervals << CharInterval.new(start_pos: match_pos, end_pos: end_pos) if end_pos <= range_end
86
100
  cursor = match_pos + 1
87
101
  end
88
-
89
102
  if intervals.empty?
90
- downcase_text = text[range].downcase
91
- downcase_target = extraction_text.downcase
92
- local_pos = downcase_text.index(downcase_target)
93
- if local_pos
94
- intervals << CharInterval.new(
95
- start_pos: range.begin + local_pos,
96
- end_pos: range.begin + local_pos + extraction_text.length
97
- )
103
+ pattern = Regexp.new(Regexp.escape(extraction_text), Regexp::IGNORECASE)
104
+ cursor = range.begin
105
+ while (match = pattern.match(text, cursor)) && match.begin(0) < range_end
106
+ intervals << CharInterval.new(start_pos: match.begin(0), end_pos: match.end(0)) if match.end(0) <= range_end
107
+ cursor = match.begin(0) + 1
98
108
  end
99
109
  end
100
110
 
@@ -105,19 +115,21 @@ module LangExtract
105
115
  target = normalize_for_match(extraction_text)
106
116
  return nil if target.empty?
107
117
 
108
- candidates = candidate_search_ranges(preferred_interval).flat_map do |range|
109
- fuzzy_candidates_in_range(extraction_text, target, range)
110
- end
111
- candidates = ranked_unique_candidates(candidates)
112
- return nil if candidates.empty?
118
+ target_counts = target.each_char.tally
119
+ overlap_fallback = nil
113
120
 
114
- non_overlap = candidates.find { |interval, _score| allow_overlaps || !overlaps_any?(interval, occupied) }
115
- return [non_overlap.first, AlignmentStatus::FUZZY] if non_overlap
121
+ candidate_search_ranges(preferred_interval).each do |range|
122
+ candidates = fuzzy_candidates_in_range(extraction_text, target, target_counts, range, occupied)
123
+ candidates = ranked_unique_candidates(candidates)
124
+ overlap_fallback ||= candidates.first&.first
125
+ non_overlap = candidates.find { |interval, _score| allow_overlaps || !overlaps_any?(interval, occupied) }
126
+ return [non_overlap.first, AlignmentStatus::FUZZY] if non_overlap
127
+ end
116
128
 
117
- [candidates.first.first, AlignmentStatus::OVERLAP]
129
+ overlap_fallback && [overlap_fallback, AlignmentStatus::OVERLAP]
118
130
  end
119
131
 
120
- def fuzzy_candidates_in_range(extraction_text, normalized_target, range)
132
+ def fuzzy_candidates_in_range(extraction_text, normalized_target, target_counts, range, occupied)
121
133
  target_length = extraction_text.length
122
134
  min_length = [1, (target_length * 0.65).floor].max
123
135
  max_length = [(target_length * 1.35).ceil, min_length].max
@@ -129,13 +141,18 @@ module LangExtract
129
141
  next if end_pos > range.end
130
142
 
131
143
  candidate = text[start_pos...end_pos]
132
- score = similarity(normalized_target, normalize_for_match(candidate))
144
+ normalized_candidate = normalize_for_match(candidate)
145
+ next unless SimilarityUpperBound.passes?(normalized_target, normalized_candidate, target_counts,
146
+ fuzzy_threshold)
147
+
148
+ score = similarity(normalized_target, normalized_candidate)
133
149
  next if score < fuzzy_threshold
134
150
 
135
- candidates << [
136
- CharInterval.new(start_pos: start_pos, end_pos: end_pos),
137
- score
138
- ]
151
+ match = [CharInterval.new(start_pos: start_pos, end_pos: end_pos), score]
152
+ interval = match.first
153
+ return [match] if score >= 1.0 && (allow_overlaps || !overlaps_any?(interval, occupied))
154
+
155
+ candidates << match
139
156
  end
140
157
  end
141
158
 
@@ -5,7 +5,12 @@ require_relative "core/errors"
5
5
  module LangExtract
6
6
  Error = Core::Error
7
7
  InvalidModelConfigError = Core::InvalidModelConfigError
8
+ ProviderError = Core::ProviderError
8
9
  ProviderConfigError = Core::ProviderConfigError
10
+ ProviderAuthError = Core::ProviderAuthError
11
+ ProviderRateLimitError = Core::ProviderRateLimitError
12
+ ProviderTimeoutError = Core::ProviderTimeoutError
13
+ ProviderResponseError = Core::ProviderResponseError
9
14
  FormatParsingError = Core::FormatParsingError
10
15
  PromptValidationError = Core::PromptValidationError
11
16
  AlignmentError = Core::AlignmentError
@@ -8,6 +8,7 @@ require_relative "core/prompting"
8
8
  module LangExtract
9
9
  class Extractor
10
10
  DEFAULT_MAX_CHAR_BUFFER = Core::SentenceAwareChunker::DEFAULT_MAX_CHAR_BUFFER
11
+ FORMATS = %i[auto json yaml].freeze
11
12
 
12
13
  def initialize(model:, prompt_description:, examples: [], additional_context: nil,
13
14
  max_char_buffer: DEFAULT_MAX_CHAR_BUFFER, context_window_chars: 0,
@@ -18,7 +19,7 @@ module LangExtract
18
19
  tokenizer: Core::UnicodeTokenizer.new)
19
20
  @model = model
20
21
  @prompt_description = prompt_description
21
- @examples = examples
22
+ @examples = examples.nil? ? [] : examples
22
23
  @additional_context = additional_context
23
24
  @max_char_buffer = max_char_buffer
24
25
  @context_window_chars = context_window_chars
@@ -31,6 +32,7 @@ module LangExtract
31
32
  @allow_overlaps = allow_overlaps
32
33
  @fuzzy_threshold = fuzzy_threshold
33
34
  @tokenizer = tokenizer
35
+ validate_configuration!
34
36
  validate_model!
35
37
  end
36
38
 
@@ -52,26 +54,28 @@ module LangExtract
52
54
  chunker = Core::SentenceAwareChunker.new(max_char_buffer: max_char_buffer, tokenizer: tokenizer)
53
55
  prompt_builder = Core::PromptBuilder.new(validation_mode: prompt_validation)
54
56
  format_handler = Core::FormatHandler.new
57
+ resolver = Core::Resolver.new(
58
+ text: document.text,
59
+ tokenizer: tokenizer,
60
+ fuzzy_threshold: fuzzy_threshold,
61
+ allow_overlaps: allow_overlaps,
62
+ suppress_alignment_errors: suppress_alignment_errors
63
+ )
55
64
  extractions = []
56
65
 
57
66
  chunker.chunks(document).each do |chunk|
58
67
  extraction_passes.times do |pass_index|
59
68
  raw_output = infer(prompt_builder, document, chunk, pass_index)
60
- parsed = parse(format_handler, raw_output)
61
- resolver = Core::Resolver.new(
62
- text: document.text,
63
- tokenizer: tokenizer,
64
- fuzzy_threshold: fuzzy_threshold,
65
- allow_overlaps: allow_overlaps,
66
- suppress_alignment_errors: suppress_alignment_errors
67
- )
69
+ parsed = parse(format_handler, raw_output, document: document, chunk: chunk)
68
70
  extractions.concat(
69
71
  resolver.resolve(parsed, document_id: document.id, preferred_interval: chunk.char_interval)
70
72
  )
71
73
  end
72
74
  end
73
75
 
74
- Core::AnnotatedDocument.new(document: document, extractions: merge_extractions(extractions))
76
+ annotated = Core::AnnotatedDocument.new(document: document, extractions: merge_extractions(extractions))
77
+ log_document_summary(annotated)
78
+ annotated
75
79
  end
76
80
 
77
81
  def infer(prompt_builder, document, chunk, pass_index)
@@ -94,14 +98,25 @@ module LangExtract
94
98
  result.respond_to?(:text) ? result.text.to_s : result.to_s
95
99
  end
96
100
 
97
- def parse(format_handler, raw_output)
101
+ def parse(format_handler, raw_output, document:, chunk:)
98
102
  format_handler.parse(raw_output, format: format, strict: strict)
99
- rescue Core::FormatParsingError
103
+ rescue Core::FormatParsingError => e
100
104
  raise unless suppress_parse_errors
101
105
 
106
+ LangExtract.config.logger&.warn(
107
+ "suppressed parse error document_id=#{document.id} chunk_index=#{chunk.index}: #{e.message}"
108
+ )
102
109
  []
103
110
  end
104
111
 
112
+ def log_document_summary(annotated)
113
+ counts = annotated.extractions.group_by(&:alignment_status).transform_values(&:length)
114
+ statuses = counts.sort.map { |status, count| "#{status}=#{count}" }.join(",")
115
+ LangExtract.config.logger&.debug(
116
+ "document_id=#{annotated.document.id} extractions=#{annotated.extractions.length} alignment_status=#{statuses}"
117
+ )
118
+ end
119
+
105
120
  def merge_extractions(extractions)
106
121
  occupied = []
107
122
  seen = {}
@@ -128,13 +143,17 @@ module LangExtract
128
143
  interval&.end_pos,
129
144
  extraction.text,
130
145
  extraction.description,
131
- extraction.extraction_class
146
+ extraction.extraction_class,
147
+ extraction.attributes,
148
+ extraction.group_id
132
149
  ]
133
150
  end
134
151
 
135
152
  def coerce_documents(text, documents)
136
153
  if documents
137
- Array(documents).map.with_index { |document, index| coerce_document(document, index) }
154
+ (documents.is_a?(Hash) ? [documents] : Array(documents)).map.with_index do |document, index|
155
+ coerce_document(document, index)
156
+ end
138
157
  elsif text
139
158
  [Core::Document.new(text: text, id: "document_0")]
140
159
  else
@@ -154,5 +173,45 @@ module LangExtract
154
173
 
155
174
  raise Core::InvalidModelConfigError, "model must respond to #infer(prompt:)"
156
175
  end
176
+
177
+ def validate_configuration!
178
+ validate_integer!(extraction_passes, "extraction_passes", 1)
179
+ validate_integer!(max_char_buffer, "max_char_buffer", 1)
180
+ validate_integer!(context_window_chars, "context_window_chars", 0)
181
+ validate_fuzzy_threshold!
182
+ validate_prompt_description!
183
+ raise ArgumentError, "examples must respond to #each" unless examples.respond_to?(:each)
184
+
185
+ validate_symbol_value!(format, FORMATS, "format must be one of: auto, json, yaml")
186
+ validate_symbol_value!(
187
+ prompt_validation,
188
+ Core::PromptValidation::MODES,
189
+ "prompt_validation must be one of: off, warning, error"
190
+ )
191
+ end
192
+
193
+ def validate_integer!(value, name, minimum)
194
+ return if value.is_a?(Integer) && value >= minimum
195
+
196
+ raise ArgumentError, "#{name} must be an Integer >= #{minimum}"
197
+ end
198
+
199
+ def validate_fuzzy_threshold!
200
+ return if fuzzy_threshold.is_a?(Numeric) && (0.0..1.0).cover?(fuzzy_threshold)
201
+
202
+ raise ArgumentError, "fuzzy_threshold must be Numeric within 0.0..1.0"
203
+ end
204
+
205
+ def validate_prompt_description!
206
+ return if prompt_description.is_a?(String) && !prompt_description.empty?
207
+
208
+ raise ArgumentError, "prompt_description must be a non-empty String"
209
+ end
210
+
211
+ def validate_symbol_value!(value, valid_values, error_message)
212
+ return if value.respond_to?(:to_sym) && valid_values.include?(value.to_sym)
213
+
214
+ raise ArgumentError, error_message
215
+ end
157
216
  end
158
217
  end
@@ -38,6 +38,8 @@ module LangExtract
38
38
  end
39
39
 
40
40
  module Factory
41
+ ROUTER_MUTEX = Mutex.new
42
+
41
43
  module_function
42
44
 
43
45
  def create_model(config = ModelConfig.new)
@@ -45,11 +47,11 @@ module LangExtract
45
47
  end
46
48
 
47
49
  def router
48
- @router ||= default_router
50
+ ROUTER_MUTEX.synchronize { @router ||= default_router }
49
51
  end
50
52
 
51
53
  def reset_router!
52
- @router = default_router
54
+ ROUTER_MUTEX.synchronize { @router = default_router }
53
55
  end
54
56
 
55
57
  def default_router
@@ -10,25 +10,27 @@ module LangExtract
10
10
  module_function
11
11
 
12
12
  def save_annotated_documents(path, documents)
13
- normalized = Array(documents)
14
- ::File.open(path, "w") do |file|
13
+ normalized = documents.is_a?(Hash) ? [documents] : Array(documents)
14
+ ::File.open(path, "w:UTF-8") do |file|
15
15
  normalized.each do |document|
16
16
  annotated = document.is_a?(Core::AnnotatedDocument) ? document : Core::AnnotatedDocument.from_h(document)
17
17
  file.puts(JSON.generate(annotated.to_h))
18
18
  end
19
19
  end
20
20
  path
21
- rescue SystemCallError, JSON::GeneratorError => e
21
+ rescue SystemCallError, JSON::GeneratorError, EncodingError => e
22
22
  raise Core::IOFailure, "failed to save annotated documents: #{e.message}"
23
23
  end
24
24
 
25
25
  def load_annotated_documents_jsonl(path)
26
- ::File.readlines(path, chomp: true).filter_map do |line|
26
+ documents = []
27
+ ::File.foreach(path, encoding: "UTF-8") do |line|
27
28
  next if line.strip.empty?
28
29
 
29
- Core::AnnotatedDocument.from_h(JSON.parse(line))
30
+ documents << Core::AnnotatedDocument.from_h(JSON.parse(line))
30
31
  end
31
- rescue SystemCallError, JSON::ParserError => e
32
+ documents
33
+ rescue SystemCallError, JSON::ParserError, EncodingError => e
32
34
  raise Core::IOFailure, "failed to load annotated documents: #{e.message}"
33
35
  end
34
36
  end
@@ -2,6 +2,7 @@
2
2
 
3
3
  require_relative "base"
4
4
  require_relative "../core/types"
5
+ require "timeout"
5
6
 
6
7
  module LangExtract
7
8
  module Providers
@@ -14,11 +15,40 @@ module LangExtract
14
15
  rescue LoadError => e
15
16
  raise Core::ProviderConfigError, "ruby_llm is required for live provider inference: #{e.message}"
16
17
  rescue StandardError => e
17
- raise Core::ProviderConfigError, "provider inference failed: #{e.message}"
18
+ error_class = provider_error_class(e)
19
+ raise unless error_class
20
+
21
+ raise error_class, "provider inference failed: #{e.message}"
18
22
  end
19
23
 
20
24
  private
21
25
 
26
+ def provider_error_class(error)
27
+ return Core::ProviderTimeoutError if timeout_error?(error)
28
+ return Core::ProviderAuthError if ruby_llm_error?(error, :UnauthorizedError, :ForbiddenError,
29
+ :PaymentRequiredError)
30
+ return Core::ProviderConfigError if ruby_llm_error?(error, :ConfigurationError)
31
+ return Core::ProviderRateLimitError if ruby_llm_error?(error, :RateLimitError, :OverloadedError,
32
+ :ServiceUnavailableError)
33
+ return Core::ProviderResponseError if defined?(RubyLLM::Error) && error.is_a?(RubyLLM::Error)
34
+
35
+ nil
36
+ end
37
+
38
+ def timeout_error?(error)
39
+ error.is_a?(Timeout::Error) || error.is_a?(Errno::ETIMEDOUT) ||
40
+ (defined?(Faraday::TimeoutError) && error.is_a?(Faraday::TimeoutError)) ||
41
+ error.class.name.to_s.match?(/timeout/i)
42
+ end
43
+
44
+ def ruby_llm_error?(error, *constant_names)
45
+ return false unless defined?(RubyLLM)
46
+
47
+ constant_names.any? do |constant_name|
48
+ RubyLLM.const_defined?(constant_name, false) && error.is_a?(RubyLLM.const_get(constant_name))
49
+ end
50
+ end
51
+
22
52
  def chat_options
23
53
  options = config.options.dup
24
54
  options[:model] = config.model if config.model
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module LangExtract
4
- VERSION = "0.1.0"
4
+ VERSION = "0.3.0"
5
5
  end
data/lib/langextract.rb CHANGED
@@ -18,6 +18,8 @@ require_relative "langextract/visualization"
18
18
  require_relative "langextract/factory"
19
19
 
20
20
  module LangExtract
21
+ CONFIG_MUTEX = Mutex.new
22
+
21
23
  CharInterval = Core::CharInterval
22
24
  TokenInterval = Core::TokenInterval
23
25
  Extraction = Core::Extraction
@@ -28,7 +30,7 @@ module LangExtract
28
30
 
29
31
  class << self
30
32
  def config
31
- @config ||= Config.new
33
+ CONFIG_MUTEX.synchronize { @config ||= Config.new }
32
34
  end
33
35
 
34
36
  def configure
@@ -36,7 +38,7 @@ module LangExtract
36
38
  end
37
39
 
38
40
  def reset_configuration!
39
- @config = nil
41
+ CONFIG_MUTEX.synchronize { @config = nil }
40
42
  Factory.reset_router!
41
43
  end
42
44
 
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: langextract
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - David Paluy
@@ -51,7 +51,7 @@ metadata:
51
51
  homepage_uri: https://github.com/dpaluy/langextract
52
52
  documentation_uri: https://rubydoc.info/gems/langextract
53
53
  source_code_uri: https://github.com/dpaluy/langextract
54
- changelog_uri: https://github.com/dpaluy/langextract/blob/main/CHANGELOG.md
54
+ changelog_uri: https://github.com/dpaluy/langextract/blob/master/CHANGELOG.md
55
55
  bug_tracker_uri: https://github.com/dpaluy/langextract/issues
56
56
  rubygems_mfa_required: 'true'
57
57
  rdoc_options: []
@@ -61,14 +61,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
61
61
  requirements:
62
62
  - - ">="
63
63
  - !ruby/object:Gem::Version
64
- version: 3.4.5
64
+ version: 4.0.5
65
65
  required_rubygems_version: !ruby/object:Gem::Requirement
66
66
  requirements:
67
67
  - - ">="
68
68
  - !ruby/object:Gem::Version
69
69
  version: '0'
70
70
  requirements: []
71
- rubygems_version: 3.6.9
71
+ rubygems_version: 4.0.10
72
72
  specification_version: 4
73
73
  summary: Ruby port of LangExtract for source-grounded structured extraction.
74
74
  test_files: []