julewire-core 1.0.0

data/docs/instrumentation-cheatsheet.md

@@ -0,0 +1,29 @@
+# Instrumentation Cheatsheet
+
+Use the smallest entrypoint that matches the work:
+
+| Need | API |
+| --- | --- |
+| Add ambient fields | `Julewire.context.add(...)`, `Julewire.attributes.add(...)`, `Julewire.carry.add(...)` |
+| Emit a point record | `Julewire.info("message", attributes: {...})` |
+| Wrap work with a summary | `Julewire.with_execution(type: :job, id: id) { ... }` |
+| Measure block-shaped work | `Julewire.measure(:db) { query }` |
+| Measure split start/finish work | `handle = Julewire.measure_start(:db); handle.finish` |
+| Install a destination | `config.destinations.use(:default, output: $stdout)` |
+| Tail local records | `julewire tail log/julewire.jsonl --format=auto --follow` |
+
+Field placement:
+
+| Field bag | Use for |
+| --- | --- |
+| `context` | Request/job/message identity useful on every record in scope. |
+| `carry` | Propagation data that may cross process boundaries. |
+| `neutral` | Provider-neutral semantic fields such as HTTP, job, code, or messaging keys. |
+| `attributes` | Framework/app-specific structured data. |
+| `payload` | User event payload for point records and summary counters. |
+| `metrics` | Numeric summary measurements. |
+| `labels` | Operator routing labels. |
+
+`logger` names the entrypoint that produced the record, such as
+`"framework.error"` or `"job.event"`. `source` names the integration or
+runtime producer, such as `"web"`, `"job"`, or `"julewire"`.

data/docs/internals.md

@@ -0,0 +1,135 @@
+# Internals
+
+This file is for contributors and extension authors who need the shape of the
+machine without reading every class first.
+
+## Runtime State
+
+The runtime stores immutable snapshots of:
+
+- configuration
+- pipeline
+- whether the active pipeline has been closed
+
+Writes are serialized where state transitions need serialization. Main-ractor
+hot-path reads use the current frozen runtime state without taking a global
+runtime lock; state transitions swap that reference under the runtime state
+mutex. Health and counters use their own synchronization outside that mutex.
+Non-main ractors use ractor-local runtime storage because they cannot touch the
+main runtime object directly.
+
+Configuration, reconfigure, reset, and close are state transitions. They should
+stay boring and explicit.
+
+Configure calls are serialized with a dedicated configure mutex. The staged
+pipeline is built before runtime state is swapped. If runtime state changes
+before a staged configure can swap in, the staged pipeline is closed and
+configure fails.
+
+Close is terminal for the active runtime state. Post-close emits are counted and
+dropped before pipeline work. Reconfigure installs a fresh open pipeline.
+
+## Pipeline
+
+The normal emit path is:
+
+```text
+threshold precheck
+record normalization
+static labels
+processors
+destinations
+formatter/encoder/output write per destination
+```
+
+Below-threshold records with eager severity are dropped before context lookup
+and record building. Lazy `emit` blocks without eager severity are evaluated
+first so block-provided severity can be checked correctly.
+
+Pipeline counters are operational counters, not delivery guarantees.
+Destination `output_accepted` means the configured output accepted the encoded
+string according to core's output contract.
+
+## Fields::FieldSet
+
+`Fields::FieldSet` owns three invariants:
+
+- defensive copying of hashes, arrays, and strings at trust boundaries
+- cycle-safe duplication of those container graphs
+- symbol-key normalization
+
+Public ingress accepts string or symbol keys because JSON-style hashes and
+Ruby-style hashes both happen in app code. Core normalizes those keys once at
+the boundary and uses symbol keys internally so processors, destinations, and
+extensions do not pay repeated equivalent-key scans.
+
+## Value Readers
+
+`Records::RawInput` reads app-facing emit input before record construction.
+`Integration::Values::Read` extracts best-effort values from framework objects.
+`Integration::Values::Shape` normalizes adapter-built hashes. `Fields::FieldSet`
+owns trusted field-bag copying, merging, and key normalization after values have
+entered core. `Fields::Lookup` is for display reads that tolerate symbol/string
+keys and return `nil` for unreadable inputs.
+
+## Context Storage
+
+Main runtime context uses storage on the current Ruby fiber.
+
+Raw fibers do not inherit parent context. Julewire wrappers capture and restore
+propagation envelopes explicitly.
+
+Non-main ractors use thread-local storage because concurrent-ruby local-variable
+helpers are not Ractor-shareable.
+
+## Destination Boundary
+
+Core writes synchronously to configured outputs. Async queues, files, fanout,
+batching, retries, acknowledgements, and delivery health live in
+custom destination objects.
+
+## Scheduling
+
+`Scheduling::DeadlineScheduler` is the small stdlib timer heap. Main-process
+diagnostics and framework integrations use `Scheduling::SharedScheduler` so
+they do not each keep a background timer thread. Ractor integrations keep their
+own schedulers because worker ractors cannot share the main scheduler object.
+
+## Remote Envelope Hook
+
+Core keeps `Runtime#emit_envelope(input:, context:, attributes:, carry:, neutral:, scope:, enforce_level:)`
+for bridge code. The bridge reconstructs the scope snapshot, and core rebuilds
+a normal record from input, context, attributes, carry, neutral, and that
+snapshot before emitting through the active pipeline. It is not a public
+application API.
+
+## Test Seams
+
+Some private methods are intentionally reachable through `Julewire::Testing`.
+They reset process-global registries or storage that normal applications should
+not touch directly.
+
+## Emit Entrypoints
+
+| Entrypoint | Caller | Level gate | Input ownership |
+| --- | --- | --- | --- |
+| `Runtime#emit` | application facade | yes | normalized by core |
+| `Runtime#emit_without_level` | host-process integrations | no | normalized by core |
+| `Integration::Facade.emit` | integration SPI | configurable | adapter-owned record hash |
+| `Runtime#emit_envelope` | bridge SPI | configurable | detached, adapter-owned envelope |
+
+## Field Stack Layers
+
+Field stacks keep immutable layers plus versioned read caches. A layer without
+a parent builds a direct snapshot from its own fields. A layer whose parent
+already has a snapshot merges on top of that cached parent snapshot. Otherwise
+it walks the unsnapshotted source chain back to the nearest cached ancestor and
+then replays fields plus delete paths forward.
+
+## Best-Effort Rule
+
+Core may contain `StandardError` from its own logging path. It must not swallow
+application exceptions from user code.
+
+That rule is the line between "logging must not crash the app" and "debugging
+must not hide the app's real exception".

data/docs/outputs-and-lifecycle.md

@@ -0,0 +1,206 @@
+# Outputs and Lifecycle
+
+Outputs are stdout-ish:
+
+```ruby
+output.write(string)
+```
+
+Optional methods:
+
+```ruby
+output.flush
+output.close
+```
+
+Lifecycle hooks are no-argument output hooks:
+
+```ruby
+flush
+close
+```
+
+Runtime timeouts provide one carried deadline while core walks destinations.
+Core cannot interrupt arbitrary blocking output code, and plain outputs do not
+receive the timeout value. After the first attempted resource, core skips later
+resources when the deadline is already exhausted. Custom destination objects own
+timeout-aware drain, retry, reopen, and rotation policy.
+
+## Boundary
+
+Core is synchronous output only:
+
+```text
+emit -> normalize -> processors -> destination(formatter -> encoder -> output)
+```
+
+Destinations format an immutable `Julewire::Record` into a payload object.
+Encoders turn direct destination payload objects into strings. Outputs receive
+strings.
+
+Processors may intentionally return `:drop`. Core counts those records in
+`health[:pipeline][:counts][:processor_dropped]`; processor failures are counted
+separately as `processor_error`.
+
+Async queues, files, fanout appenders, batching, retries, acknowledgements,
+reopen, rotation, shutdown hooks, and delivery policy belong in custom
+destination objects. Core keeps enough output lifecycle to be useful for
+`$stdout`, tests, and small scripts.
+
+## No Output
+
+An empty destination registry means an intentional no-output pipeline. Records
+are dropped before normalization, processors, and formatting. Core increments
+`health[:pipeline][:counts][:no_output_dropped]` so accidental no-output mode is
+visible in health.
+
+Application and integration code should usually make no-output mode
+explicit instead of inheriting it silently. Core keeps it available for tests,
+scripts, and deliberate dry runs.
+
+## Ownership
+
+Outputs are caller-owned by default.
+
+```ruby
+Julewire.configure do |config|
+  file = File.open("log/julewire.log", "a")
+  config.destinations.use(:default, output: file, close_output: true)
+end
+```
+
+Use `close_output = true` for files or sockets core should close. Keep it false
+for `$stdout`, caller-owned loggers, and shared objects.
+
+## Synchronous Output
+
+Core wraps direct outputs with one mutex. Concurrent emitters do not interleave
+individual encoded records, but a slow sink blocks the emitting thread.
+
+A synchronous output failure is contained. The destination records output
+failure counters and calls `on_failure`. Core does not retry, back off, or
+short-circuit broken outputs; those are destination policies.
+
+Plain `write` outputs and custom destination `emit` calls are optimistic: any
+non-raising result except `false` is accepted. A plain `false` is treated as a
+rejected write or destination rejection and counted as destination loss. Custom
+destination objects still own richer backpressure, retry, and partial acceptance
+semantics.
+
+## Destinations
+
+Direct core output is always configured through destinations:
+
+```text
+emit -> normalize -> processors -> :default(formatter -> encoder -> output)
+```
+
+Multiple destinations are explicit:
+
+```ruby
+Julewire.configure do |config|
+  config.destinations.use(
+    :json_stdout,
+    formatter: Julewire::RecordFormatter.new,
+    encoder: Julewire::JsonEncoder.new,
+    output: $stdout
+  )
+
+  config.destinations.use(
+    :debug,
+    formatter: Julewire::RecordFormatter.new,
+    encoder: Julewire::JsonEncoder.new,
+    output: File.open("log/debug.json", "a"),
+    close_output: true
+  )
+end
+```
+
+Custom destinations may add a destination object directly with
+`config.destinations.add(destination)`. That object receives the same immutable
+processed `Julewire::Record` and owns its own formatter, serialization,
+and delivery contract.
+
+Multiple destinations receive the same immutable processed record object.
+Hashes, arrays, and copied strings inside the record are frozen; arbitrary app
+objects inside fields are not deep-frozen. Formatters are read-only mappers.
+Mutation belongs in processors.
+
+Destinations cannot share the same raw output object. Core sync output keeps one
+mutex per destination and rejects shared sinks instead of guessing fanout
+semantics. Use a custom destination when same-format multi-sink writes need
+shared locking, async, buffering, rotation, batching, or fanout policy.
+
+## Lifecycle
+
+`Julewire.flush` and `Julewire.close` call the active pipeline's destinations
+in order.
+
+`flush` and `close` use `config.pipeline_close_timeout` when no timeout is
+provided. A caller-provided `nil` timeout means unbounded wait. This timeout
+does not make direct output calls interruptible; it only bounds whether core
+attempts later destinations after a previous lifecycle call returns.
+
+`close` is terminal for the active runtime state. If output close fails or times
+out, `Julewire.close` returns `false`, but later emits still drop as
+`runtime_closed` until the next `configure` or `reset!`.
+
+`Julewire.after_fork!` resets process-local counters, failure snapshots,
+current context, warning state, and mutexes inherited from the parent process.
+It also forwards `after_fork!` to destinations and outputs that implement it,
+then runs integration after-fork hooks registered through core. File, socket,
+queue, and async transports should reopen worker-local resources from their
+destination or output `after_fork!` method.
+
+Core does not install an `at_exit` hook. Small scripts should call close from
+their own shutdown path:
+
+```ruby
+begin
+  # work
+ensure
+  Julewire.close(timeout: 1)
+end
+```
+
+Applications and integration code should install their own shutdown hooks with
+finite timeouts.
+
+## Health
+
+Core health is a pipeline snapshot, not a delivery receipt.
+
+Preferred alert fields:
+
+- top-level `status`, `closed`, `generation`, `counts`, and `last_failure`
+- `pipeline.configured`, `pipeline.counts`, and `pipeline.last_failure`
+- `destinations.*.counts`
+- `destinations.*.last_failure`
+- `destinations.*.last_loss`
+
+Destination `counts` includes direct-output loss counters such as formatter
+failures, encoding failures, record-size drops, output exceptions, and output
+rejections. Custom destination objects own their own queue, file, fanout,
+retry, delivery, and lifecycle health fields.
+
+Destination loss belongs to the active pipeline generation. Any destination
+with non-zero active-generation loss is degraded until reconfigure or reset.
+`destinations.*.last_loss` gives the latest safe loss reason and record
+coordinates without exposing raw payloads or exception messages.
+
+Integrations own concrete metric names. Core keeps health paths explicit for
+contract tests, but does not prescribe external metrics mapping.
+
+## Loss Model
+
+Core is best effort:
+
+- no output drops records intentionally
+- level filtering drops records before normalization when eager severity is known
+- oversized encoded records are dropped
+- formatter and encoder failures drop the affected record
+- output exceptions and `false` writes drop the affected record
+- post-close emits are dropped and counted at runtime level
+
+Core does not provide durable storage. Custom destinations that need delivery
+semantics own batching, retry, acknowledgement, and shutdown drain behavior.

data/docs/quickstart.md

@@ -0,0 +1,133 @@
+# Quickstart
+
+Start with one configured output:
+
+```ruby
+Julewire.configure do |config|
+  config.destinations.use(:default, output: $stdout)
+end
+
+Julewire.emit(message: "hello")
+```
+
+By default, records are encoded as newline-delimited JSON.
+
+## Point Records
+
+`Julewire.emit` writes an immediate point record:
+
+```ruby
+Julewire.emit(
+  severity: :info,
+  event: "customer.charged",
+  message: "charged customer",
+  payload: { amount_cents: 1200 }
+)
+```
+
+The shorthand form is useful for scripts:
+
+```ruby
+Julewire.emit("starting import")
+Julewire.warn("retrying import", attempt: 3)
+```
+
+Known record keywords stay at the record top level: `timestamp`, `severity`,
+`kind`, `event`, `message`, `logger`, `source`, `execution`, `context`,
+`carry`, `neutral`, `attributes`, `labels`, `payload`, `metrics`, and `error`.
+Other fields are folded into `payload`. Explicit `payload` keys win over folded
+fields when they collide.
+
+For expensive low-level payloads, put the severity in the eager fields and
+build the rest lazily:
+
+```ruby
+Julewire.emit(severity: :debug) do
+  { message: "expanded import state", payload: expensive_snapshot }
+end
+```
+
+The block is not called when the eager severity is below the configured level.
+If eager severity is present, it is authoritative; a block cannot upgrade or
+downgrade it. If eager severity is absent, the block may provide severity, but
+the block must run before core can apply the level check. Severity helpers use
+the eager-severity rule:
+
+```ruby
+Julewire.debug { { message: "expanded import state", payload: expensive_snapshot } }
+Julewire.warn("retrying")
+```
+
+`emit` returns `nil`. Use configured outputs, callbacks, and `Julewire.health`
+for observation.
+
+## Executions and Summaries
+
+Use `with_execution` around work that has a beginning and an end:
+
+```ruby
+Julewire.with_execution(type: :operation, id: "op-1") do
+  Julewire.context.add(tenant_id: "tenant-1")
+  Julewire.summary.add(plan: "pro")
+
+  Julewire.emit(message: "doing work")
+end
+```
+
+Context fields appear on point records and summaries. Summary fields appear only
+on the final summary. `with_execution` emits one summary record by default on
+completion or failure.
+
+Nested executions are allowed. Each level gets its own summary, and point
+records include cheap parent/root/depth metadata. Processors can use the
+explicit lineage accessor when they need the bounded ancestor chain.
+
+## No Output Means No-Op
+
+This is valid:
+
+```ruby
+Julewire.configure do |config|
+  config.destinations.clear
+end
+```
+
+With no destinations, core does not build, process, format, or write records. It
+only increments `health[:pipeline][:counts][:no_output_dropped]`. This is useful
+in tests and in deployments where custom code owns output setup.
+
+## The Mental Model
+
+Think of core as:
+
+```text
+emit -> normalize -> processors -> destination(formatter -> encoder -> output)
+```
+
+Processors own neutral policy and enrichment. A destination owns the final
+formatter/output path. Its formatter sees an immutable processed
+`Julewire::Record` and returns a payload object; its encoder turns that
+payload into the output string.
+
+For field placement, use this rule:
+
+| Field bag | Best for |
+| --- | --- |
+| `context` | queryable facts copied onto point records and summaries |
+| `carry` | propagation metadata for integrations and custom formatters |
+| `neutral` | provider-neutral facts read by formatters |
+| `attributes` | integration namespaces and application metadata |
+| `summary` | final-only counters, timings, and completion facts |
+| `labels` | operator-safe routing/grouping metadata |
+| `metrics` | numeric measurements such as duration |
+
+Custom destination objects own async output, files, fanout, batching, retries,
+and delivery policy:
+
+```text
+emit -> normalize -> processors -> custom destination
+```
+
+Core stays synchronous. Processors and direct destination formatters run on the
+emitting thread. Custom destinations own their own serialization and transport
+work.

data/docs/record-sources.md

@@ -0,0 +1,17 @@
+# Record Sources
+
+Core records come from several broad source classes. Integrations can map
+their own runtime surfaces into these shapes, but core does not know those
+runtime APIs.
+
+| Source | Julewire shape | Rule |
+| --- | --- | --- |
+| External event | Point record | Map runtime event data into `event`, `source`, `context`, and `attributes`; leave `payload` for caller data. |
+| Logger call | Point record, usually `event: "log"` | Treat it as a real log item. Do not parse or deduplicate text. |
+| Execution boundary | Summary record | Use `with_execution` and emit one final summary when enabled. |
+| Error report | Point record | Map explicit error reports as point records with current context. |
+| Rendered/runtime exception | Point record | Map structured exception surfaces. Do not parse human exception text. |
+
+Core only normalizes, processes, formats, and writes records. Integrations own
+subscription, suppression of duplicate upstream emitters, and any
+runtime-specific policy.