julewire-core 1.0.0 → 1.1.0

data/docs/internals.md

@@ -11,12 +11,14 @@ The runtime stores immutable snapshots of:
 - pipeline
 - whether the active pipeline has been closed
 
-Writes are serialized where state transitions need serialization. Main-ractor
-hot-path reads use the current frozen runtime state without taking a global
-runtime lock; state transitions swap that reference under the runtime state
-mutex. Health and counters use their own synchronization outside that mutex.
-Non-main ractors use ractor-local runtime storage because they cannot touch the
-main runtime object directly.
+Main-ractor hot-path reads use the current frozen runtime state without taking
+a global runtime lock. State transitions replace that snapshot atomically;
+configure and reset are additionally serialized with the configure mutex while
+close may overlap them. The successful atomic transition owns the previous
+pipeline lifecycle, so concurrent close/reset operations cannot close the same
+pipeline twice. Health and counters use their own synchronization. Non-main
+ractors use ractor-local runtime storage because they cannot touch the main
+runtime object directly.
 
 Configuration, reconfigure, reset, and close are state transitions. They should
 stay boring and explicit.
@@ -67,10 +69,15 @@ extensions do not pay repeated equivalent-key scans.
 
 `Records::RawInput` reads app-facing emit input before record construction.
 `Integration::Values::Read` extracts best-effort values from framework objects.
-`Integration::Values::Shape` normalizes adapter-built hashes. `Fields::FieldSet`
-owns trusted field-bag copying, merging, and key normalization after values have
-entered core. `Fields::Lookup` is for display reads that tolerate symbol/string
-keys and return `nil` for unreadable inputs.
+`Integration::Values::Shape` normalizes adapter-built hashes at framework
+ingress. `Fields::FieldSet` owns public field-bag copying, merging, and key
+normalization at application or framework ingress. Once data is owned,
+`Integration::Protocol` and strict deep validation require recursively Symbol-
+keyed hashes; cross-gem code does not repair String keys. `Fields::Lookup` is
+for best-effort display reads with the exact requested key and returns `nil` for
+unreadable inputs; it does not bridge Symbol/String key forms inside normalized
+records. Presentation code that reads decoded wire payloads uses its separate
+`wire_value` helper at that restore boundary.
 
 ## Context Storage
 
@@ -90,24 +97,18 @@ custom destination objects.
 
 ## Scheduling
 
-`Scheduling::DeadlineScheduler` is the small stdlib timer heap. Main-process
-diagnostics and framework integrations use `Scheduling::SharedScheduler` so
-they do not each keep a background timer thread. Ractor integrations keep their
-own schedulers because worker ractors cannot share the main scheduler object.
+Main-process diagnostics and framework integrations use the process-owned
+`Scheduling::SharedScheduler` so they do not each keep a background timer
+thread. Ractor integrations own their stdlib timeout threads because a worker
+Ractor cannot access the main scheduler's concurrent-ruby objects.
 
 ## Remote Envelope Hook
 
-Core keeps `Runtime#emit_envelope(input:, context:, attributes:, carry:, neutral:, scope:, enforce_level:)`
+Core keeps `Runtime#emit_envelope(input:, context:, attributes:, carry:, neutral:, scope:, enforce_level:, owned:)`
 for bridge code. The bridge reconstructs the scope snapshot, and core rebuilds
 a normal record from input, context, attributes, carry, neutral, and that
-snapshot before emitting through the active pipeline. It is not a public
-application API.
-
-## Test Seams
-
-Some private methods are intentionally reachable through `Julewire::Testing`.
-They reset process-global registries or storage that normal applications should
-not touch directly.
+snapshot before emitting through the active pipeline. Bridges pass `owned: true`
+only for Julewire-owned wire data. It is not a public application API.
 
 ## Emit Entrypoints
 

data/docs/outputs-and-lifecycle.md

@@ -40,7 +40,10 @@ strings.
 
 Processors may intentionally return `:drop`. Core counts those records in
 `health[:pipeline][:counts][:processor_dropped]`; processor failures are counted
-separately as `processor_error`.
+separately as `processor_error`. Unsupported non-`nil`/non-draft return values
+keep the current draft and increment `processor_invalid`. A processor that
+leaves an invalid draft also increments `processor_invalid`, but that record is
+dropped before the next processor and also increments `processor_dropped`.
 
 Async queues, files, fanout appenders, batching, retries, acknowledgements,
 reopen, rotation, shutdown hooks, and delivery policy belong in custom
@@ -74,8 +77,11 @@ for `$stdout`, caller-owned loggers, and shared objects.
 
 ## Synchronous Output
 
-Core wraps direct outputs with one mutex. Concurrent emitters do not interleave
-individual encoded records, but a slow sink blocks the emitting thread.
+Core wraps direct output writes with one mutex. Concurrent emitters do not
+interleave individual encoded records, but a slow sink blocks the emitting
+thread. `flush` is lifecycle-only and may overlap writes so slow flushes do not
+stall the emit path; direct outputs should be IO-like or internally safe for
+flush/write overlap. `close` is terminal and serialized with writes.
 
 A synchronous output failure is contained. The destination records output
 failure counters and calls `on_failure`. Core does not retry, back off, or
@@ -146,11 +152,12 @@ out, `Julewire.close` returns `false`, but later emits still drop as
 `runtime_closed` until the next `configure` or `reset!`.
 
 `Julewire.after_fork!` resets process-local counters, failure snapshots,
-current context, warning state, and mutexes inherited from the parent process.
-It also forwards `after_fork!` to destinations and outputs that implement it,
-then runs integration after-fork hooks registered through core. File, socket,
-queue, and async transports should reopen worker-local resources from their
-destination or output `after_fork!` method.
+current context, warning state, schedulers, and registries that cannot be
+shared with the child process. It also forwards `after_fork!` to destinations
+and outputs that implement it, then runs integration after-fork hooks
+registered through core. File, socket, queue, and async transports should
+reopen worker-local resources from their destination or output `after_fork!`
+method.
 
 Core does not install an `at_exit` hook. Small scripts should call close from
 their own shutdown path:

data/docs/records-and-data-policy.md

@@ -27,10 +27,11 @@ string or symbol keys defensively. Processors receive a mutable
 ```
 
 Raw input is normalized through `Julewire::RecordDraft.build`, then processors
-mutate the draft owned by the current emit. After processors finish, core
-validates the draft and freezes it through `Julewire::RecordDraft#to_record`
-before destinations and formatters see it. Immutable `Record` objects are not
-the public raw-input construction API.
+mutate the draft owned by the current emit. Core validates the draft after
+every processor, then freezes it through `Julewire::RecordDraft#to_record`
+before destinations and formatters see it. A malformed processor result is
+attributed to that processor and never reaches the next one. Immutable
+`Record` objects are not the public raw-input construction API.
 
 `Julewire::RecordFormatter` turns that normalized record into a public log
 projection. It omits internal keys such as `:carry` and execution lineage
@@ -59,6 +60,11 @@ source/event metadata when available. Level filtering then applies normally, so
 an invalid explicit severity can still be dropped when `config.level` is above
 `:info`. Configuration severities remain strict.
 
+Severity Symbols are the canonical lowercase values `:debug`, `:info`, `:warn`,
+`:error`, `:fatal`, and `:unknown`. String ingress is case-insensitive and
+standard-library Logger severity integers are accepted. Non-canonical Symbols
+are invalid rather than silently rewritten at an internal or cross-gem boundary.
+
 ## Structured Sections
 
 Structured sections prefer hashes:
@@ -87,13 +93,38 @@ Julewire.emit(payload: "raw")
 ```
 
 Runtime mutation helpers are forgiving for the same reason: logging should not
-crash the app.
+crash the app when a positional field value is not a hash.
 
 String keys inside structured sections are normalized to symbols before
 processors and destinations run. Encoders and custom destinations may convert
 the record to string-key payloads, but the core Ruby record contract stays
 symbol-keyed.
 
+## Key Boundaries
+
+Public application input and decoded propagation envelopes may use String or
+Symbol keys. Core converts String keys to Symbols once as those values enter
+core. Other key types are invalid; they are never stringified or silently
+accepted.
+
+Integration-owned, bridge-owned, processor-owned, and normalized-record sections
+are different: each section must be a Hash and every recursive hash key must
+already be a Symbol. They are strict contracts, not another normalization
+boundary. Core does not convert their String keys. JSON carrier extraction is
+the explicit exception because JSON decode is wire ingress and necessarily
+produces String keys.
+
+Direct constructors and integration field APIs raise `TypeError` for a non-Hash
+section or invalid key. Strict owned validation visits every recursive
+container iteratively, including data deeper than the later copy bound. A
+normal `emit` contains such a failure: it drops the invalid
+record, records the failure in health and `on_failure`, and does not raise into
+application code. Integration input is validated recursively before draft
+construction or any configured processor. Unknown integration top-level fields
+are errors instead of being ignored; integrations put custom data in
+`:payload`, `:attributes`, or another documented section. An unconfigured
+pipeline drops before record construction, as it does for every emit.
+
 ## Optional Metadata
 
 `logger` is the logical logger entrypoint, for example `"app"` or
@@ -181,8 +212,8 @@ Put redaction in processors or a separate policy gem before formatting.
 
 Truncated containers, containers pruned by `max_depth`, and circular container
 references get `_julewire_truncation` metadata. Serializer keys beginning with
-`_julewire_` are reserved by convention for core metadata; user payloads must
-not use that namespace. Public Julewire record contracts use symbol keys
+`_julewire_` are reserved for core metadata; public field ingress rejects the
+truncation marker as a user key. Public Julewire record contracts use symbol keys
 internally. Payloads should also use one JSON field name per value; mixed key
 types that stringify to the same JSON field are outside the serializer contract.
 Long strings use a `...[Truncated]` suffix. User data that already contains that

data/lib/julewire/core/cli/doctor.rb

@@ -23,21 +23,22 @@ module Julewire
         def call
           options = doctor_options
           report = Julewire.doctor
-          options.fetch(:format) == :json ? write_json(report) : write_text(report, options)
+          if options[:format] == :text
+            write_text(report, options)
+          else
+            write_json(report)
+          end
           0
         end
 
         private
 
         def doctor_options
-          { color: color_output?, format: :json, theme: :plain }.tap do |options|
-            until @argv.empty?
-              value = @argv.shift
-              if (assignment = FLAGS[value])
-                apply_option(options, assignment)
-              else
-                raise ArgumentError, "unknown option #{value}"
-              end
+          @argv.each_with_object(color: color_output?) do |value, options|
+            if (assignment = FLAGS[value])
+              apply_option(options, assignment)
+            else
+              raise ArgumentError, "unknown option #{value}"
             end
           end
         end
@@ -46,7 +47,7 @@ module Julewire
           key = assignment.fetch(0)
           value = assignment.fetch(1)
           options[key] = value
-          options[:format] = :text if key == :theme && value == :punk
+          options[:format] = :text if value == :punk
         end
 
         def color_output?
@@ -59,18 +60,19 @@ module Julewire
         end
 
         def write_text(report, options)
-          theme = options.fetch(:theme)
+          theme = options[:theme]
           color = options.fetch(:color)
+          pipeline = report.fetch(:pipeline)
           lines = [
             title(theme),
             status_line(report, theme: theme, color: color),
             runtime_line(report.fetch(:runtime)),
-            pipeline_line(report.fetch(:pipeline)),
-            component_line("destinations", report.dig(:pipeline, :destinations) || {}),
+            pipeline_line(pipeline),
+            component_line("destinations", pipeline.fetch(:destinations)),
             component_line("runtime_integrations", report.fetch(:integrations)),
             component_line("process_integrations", report.fetch(:process_integrations)),
             warning_lines(report.fetch(:warnings), theme: theme)
-          ].flatten.compact
+          ]
           @stdout.write("#{lines.join("\n")}\n")
         end
 
@@ -108,7 +110,9 @@ module Julewire
           return "warnings=none" if warnings.empty?
 
           header = theme == :punk ? "!! warnings=#{warnings.length}" : "warnings=#{warnings.length}"
-          [header, *warnings.map { warning_line(it, theme: theme) }]
+          lines = [header]
+          warnings.each { lines << warning_line(it, theme: theme) }
+          lines
         end
 
         def warning_line(warning, theme:)
@@ -125,7 +129,7 @@ module Julewire
           return value unless color
 
           styles = severity_styles(theme)
-          "\e[#{styles.fetch(severity.to_s, styles.fetch("unknown"))}m#{value}\e[0m"
+          "\e[#{styles.fetch(severity)}m#{value}\e[0m"
         end
 
         def severity_for_status(status)

data/lib/julewire/core/cli/line_helpers.rb

@@ -7,8 +7,12 @@ module Julewire
         private
 
         def parse_command_options(options, command:)
-          yield(options, @argv.shift) until @argv.empty?
-          raise ArgumentError, "#{command} log path is required" unless options[:path]
+          @argv.length.times do
+            break if @argv.empty?
+
+            yield(options, @argv.shift)
+          end
+          raise ArgumentError, "#{command} log path is required" unless options.fetch(:path)
 
           options
         end
@@ -24,26 +28,33 @@ module Julewire
           value = @argv.shift
           raise ArgumentError, "#{name} value is required" unless value
 
-          Validation.validate_integer_limit!(Integer(value, 10), name: name.delete_prefix("--"), positive: true)
-        rescue ArgumentError
-          raise ArgumentError, "#{name} must be a positive integer"
+          begin
+            integer = Integer(value, 10)
+            raise ArgumentError unless integer.positive?
+
+            integer
+          rescue ArgumentError
+            raise ArgumentError, "#{name} must be a positive integer"
+          end
         end
 
         def apply_path_option(options, value, command:)
-          raise ArgumentError, "unknown option #{value}" if value.start_with?("-") && value != "-"
-          raise ArgumentError, "#{command} accepts one log path" if options[:path]
+          raise ArgumentError, "unknown option #{value}" if value.start_with?("-") && !value.eql?("-")
+          raise ArgumentError, "#{command} accepts one log path" if options.fetch(:path)
 
           options[:path] = value
         end
 
-        def handle_invalid_line(line, error, mode)
+        def handle_invalid_line(line, mode)
           case mode
           when :skip
             nil
           when :raw
             @stdout.write(raw_line(line))
+          when :fail
+            raise
           else
-            raise error
+            raise ArgumentError, "invalid line policy must be fail, raw, or skip"
           end
         end
 
@@ -53,7 +64,7 @@ module Julewire
 
         def indexed_lines(lines)
           lines.each_with_index.filter_map do |line, index|
-            [index + 1, line] unless line.strip.empty?
+            [index + 1, line] if line.match?(/\S/)
           end
         end
 
@@ -68,8 +79,8 @@ module Julewire
         def write_encoded_record_line(line, line_number, input_format:, invalid:, encoder:)
           record = LogFormats.record_from_json_line(line, line_number: line_number, format: input_format)
           @stdout.write(encoder.call(record))
-        rescue ArgumentError => e
-          handle_invalid_line(line, e, invalid)
+        rescue ArgumentError
+          handle_invalid_line(line, invalid)
         end
       end
     end

data/lib/julewire/core/cli/log_formats/core_json_decoder.rb

@@ -25,9 +25,9 @@ module Julewire
 
             def record_base(source)
               {
-                timestamp: source["timestamp"],
-                severity: Records::Severity.normalize(source["severity"] || :info),
-                kind: RecordDecoder.kind(source["kind"] || :point),
+                timestamp: source.fetch("timestamp"),
+                severity: Records::Severity.normalize(source.fetch("severity")),
+                kind: RecordDecoder.kind(source.fetch("kind")),
                 event: source["event"],
                 message: source["message"],
                 logger: source["logger"],

data/lib/julewire/core/cli/log_formats/record_decoder.rb

@@ -17,7 +17,7 @@ module Julewire
             def section(value)
               return {} unless value.is_a?(Hash)
 
-              Fields::FieldSet.deep_symbolize_keys(value)
+              Fields::FieldSet.deep_symbolize_owned_keys(value)
             end
 
             def sections(source, sections: Fields::Bags.record_hash_sections)

data/lib/julewire/core/cli/log_formats.rb

@@ -11,17 +11,17 @@ module Julewire
         @entries = []
 
         class << self
-          def register(name, decoder: nil, encoder: nil, priority: 0)
+          def register(name, decoder: nil, encoder: nil, priority: nil)
             name = normalize(name)
             validate_component(decoder, :decoder) if decoder
             validate_component(encoder, :encoder) if encoder
             existing = @entries.find { it.name == name }
-            priority = priority.to_i
+            priority = priority.nil? ? existing&.priority || 0 : Integer(priority)
             entry = Entry.new(
               name: name,
               decoder: decoder || existing&.decoder,
               encoder: encoder || existing&.encoder,
-              priority: priority.zero? && existing ? existing.priority : priority
+              priority: priority
             )
             @entries = @entries.reject { it.name == name } + [entry]
             entry
@@ -43,21 +43,21 @@ module Julewire
 
           def record_from_json_line(line, line_number:, format: AUTO_FORMAT)
             payload = JSON.parse(line)
-            Records::Record.from_normalized_hash(decode(payload, format: format))
+            Record.from_normalized_hash(decode(payload, format: format))
           rescue JSON::ParserError => e
-            raise ArgumentError, "line #{line_number}: invalid JSON: #{e.message}"
+            raise ArgumentError, "line #{line_number}: invalid JSON: #{e}"
           rescue TypeError, ArgumentError => e
-            raise ArgumentError, "line #{line_number}: #{e.message}"
+            raise ArgumentError, "line #{line_number}: #{e}"
           end
 
           def normalize(value)
             name = Core.normalize_name(value, name: "log format")
-            return name if name.to_s.match?(FORMAT_NAME_PATTERN)
+            return name if name.match?(FORMAT_NAME_PATTERN)
 
             raise ArgumentError, "log format must contain lowercase letters, digits, or underscores"
           end
 
-          def load(name)
+          def load_gem_format(name)
             path = "julewire/#{name}"
             require path
           rescue LoadError => e
@@ -99,7 +99,7 @@ module Julewire
           def load_format(name)
             return if @entries.any? { it.name == name }
 
-            load(name)
+            load_gem_format(name)
           end
 
           def decode_entries

data/lib/julewire/core/cli/tail.rb

@@ -55,17 +55,19 @@ module Julewire
         end
 
         def apply_tail_option(options, value)
-          if (assignment = FLAGS[value])
+          case value
+          when *FLAGS.keys
+            assignment = FLAGS.fetch(value)
             options[assignment.fetch(0)] = assignment.fetch(1)
-          elsif value.start_with?("--format=")
-            options[:format] = value.delete_prefix("--format=").to_sym
-          elsif value == "--format"
+          when /\A--format=(.+)\z/
+            options[:format] = Regexp.last_match(1)
+          when "--format"
             options[:format] = next_symbol_option("--format")
-          elsif value == "--theme"
+          when "--theme"
             options[:theme] = next_symbol_option("--theme")
-          elsif value == "--limit"
+          when "--limit"
             options[:limit] = positive_integer_option("--limit")
-          elsif value == "--max-value-bytes"
+          when "--max-value-bytes"
             options[:max_value_bytes] = positive_integer_option("--max-value-bytes")
           else
             apply_path_option(options, value, command: "tail")
@@ -74,7 +76,7 @@ module Julewire
 
         def tail_renderer(options)
           encoder = console_text_encoder(options)
-          proc do |line, line_number|
+          lambda do |line, line_number|
             write_encoded_record_line(
               line,
               line_number,
@@ -87,13 +89,16 @@ module Julewire
 
         def tail_stdin(options, renderer)
           limit = options.fetch(:limit)
-          return render_limited_stdin(limit, renderer) if limit
-
-          render_stream(@stdin.each_line, renderer)
+          if limit
+            render_limited_stdin(limit, renderer)
+          else
+            render_stream(@stdin.each_line, renderer)
+          end
         end
 
         def tail_file(options, renderer)
-          File.open(options.fetch(:path), "r") do |file|
+          descriptor = IO.sysopen(options.fetch(:path))
+          IO.open(descriptor) do |file|
             line_number = render_file_snapshot(file, options, renderer)
             follow_file(file, line_number, options, renderer) if options.fetch(:follow)
           end
@@ -113,7 +118,7 @@ module Julewire
               render_entries([[line_number, line]], renderer)
             else
               line_number = reset_follow_position(file) if file.stat.size < file.pos
-              sleep(options.fetch(:poll_interval))
+              Kernel.sleep(options.fetch(:poll_interval))
             end
           end
         end
@@ -132,7 +137,7 @@ module Julewire
           line_number = 0
           lines.each do |line|
             line_number += 1
-            next if line.strip.empty?
+            next unless line.match?(/\S/)
 
             renderer.call(line, line_number)
           end

data/lib/julewire/core/cli/transcode.rb

@@ -69,7 +69,7 @@ module Julewire
 
         def apply_named_option(options, value)
           if value.start_with?("--from=")
-            options[:from] = value.delete_prefix("--from=").to_sym
+            options[:from] = value.delete_prefix("--from=")
           elsif value.start_with?("--to=")
             options[:to] = value.delete_prefix("--to=").to_sym
           else
@@ -84,7 +84,7 @@ module Julewire
           when "--theme" then options[:theme] = next_symbol_option("--theme")
           when "--max-value-bytes" then options[:max_value_bytes] = positive_integer_option("--max-value-bytes")
           else
-            apply_path_option(options, value, command: "transcode")
+            raise ArgumentError, "unknown option #{value}"
           end
         end
 
@@ -95,9 +95,8 @@ module Julewire
         end
 
         def each_entry(path, &)
-          return indexed_lines(@stdin.each_line).each(&) if path == "-"
-
-          File.open(path, "r") { |file| indexed_lines(file.each_line).each(&) }
+          lines = path == "-" ? @stdin.each_line : File.foreach(path)
+          indexed_lines(lines).each(&)
         end
       end
     end

data/lib/julewire/core/cli.rb

@@ -31,7 +31,7 @@ module Julewire
       rescue Interrupt
         INTERRUPTED_STATUS
       rescue ArgumentError, Errno::ENOENT => e
-        fail_with(e.message)
+        fail_with(e)
       end
 
       private

data/lib/julewire/core/configuration.rb

@@ -24,7 +24,7 @@ module Julewire
         reject_unknown_options!(options)
         @destinations = options.fetch(:destinations) { Destinations::Registry.new }
         @emit_non_standard_exception_summaries = options.fetch(:emit_non_standard_exception_summaries, false)
-        @error_backtrace_lines = options.fetch(:error_backtrace_lines, Core::MAX_BACKTRACE_LINES)
+        @error_backtrace_lines = options.fetch(:error_backtrace_lines, MAX_BACKTRACE_LINES)
         @labels = options.fetch(:labels) { Fields::StaticLabels.new }
         @level = options.fetch(:level, :debug)
         @on_drop = options.fetch(:on_drop, nil)