logstruct 0.0.2 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 22a9091af167d668b3470dfc55ea0d02de4809b66dd51db5531fe60e138c5452
-  data.tar.gz: 75f1e277bc6dafa3666ebd1be7e1c64c5e783065520fe5261eb734c3011ade3f
+  metadata.gz: 0b926657fc690c72031cf72973642c3654264b3ea348f2212a625788b62d1ade
+  data.tar.gz: 650b11e3b8b3e975feee474994ae9420b68da2eb933942f28cc495649d71f225
 SHA512:
-  metadata.gz: 6202a87818cb4ba803df9b5a516291fd7513ad58c0d94e2a42bd5f341f9bc3f5e04e777bdaa82f843d09b78a4ddbd823f87e64777ff91f9d4b07606deb554291
-  data.tar.gz: 886387655858e495eb9cb36f0d4d727f17719161771b7bdd52b67328cea78c975af3e40d8689e66e9b1cfacc8e2339867286303245770ed2680d37890d5d8e08
+  metadata.gz: 2e0339abeafbea5bf239799dcf1ca843f57aae004e356e407a18909d9c0b6e4d1302c897f358be7bf539b5e0534efd032319a0b3055ff29e169c1d3f22557193
+  data.tar.gz: de385bc25a32f189044fbf1749e4511c1f9411d37a9af8628fc655e9706daba1a6b94b9482de756ef12557cc497deffa6f49a39ab9b434b3be6e25c7cb402214

data/CHANGELOG.md

@@ -5,28 +5,10 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [0.0.1] - 2025-03-04
+## [0.1.1] - 2025-09-29
 
-### Added
+Added dotenv-rails integration. Many other fixes and improvements.
 
-- Pushed empty gem to RubyGems to secure the name
+## [0.1.0] - 2025-09-07
 
-## [0.0.2-rc1] - 2025-09-05
-
-### Added
-
-- Unified GitHub Actions workflow to release RubyGem and sync/tag the Terraform provider; dry-run support for safe validation
-- Provider catalog export script and automated provider CI (build/vet/test)
-- Terraform docs page with quickstart, recipes, and provider README link
-- API design doc (typed vs. untyped) planned; initial philosophy captured
-- Coverage threshold gate (>= 80%) in CI; additional tests to lift coverage
-- Release helper: scripts/create_release_tag.sh (creates annotated tag from version.rb)
-
-### Changed
-
-- ActionMailer callbacks patched for Rails 7.0; event logging + metadata collection tested
-- Provider CloudWatch filter builder refactored for testability; key lookups aligned with exported catalog (event/source)
-
-### Fixed
-
-- Pre-commit hooks reporting; cspell dictionary updated; TypeScript import fixes in site
+Initial beta release.

data/README.md

@@ -48,9 +48,9 @@ Please see the [documentation](https://logstruct.com/docs/configuration/) for co
 
 ### Important Notes on Integration
 
-Once initialized, the gem automatically includes its modules into the appropriate base classes:
+Once initialized (and enabled), the gem automatically includes its modules into the appropriate base classes:
 
-- `ActiveSupport::TaggedLogging` is patched to support both Hashes and Strings
+- `ActiveSupport::TaggedLogging` is patched to support both Hashes and Strings (only when LogStruct is enabled)
 - `ActionMailer::Base` includes error handling and event logging modules
 - We configure `Lograge` for request logging
 - A Rack middleware is inserted to catch and log errors, including security violations (IP spoofing, CSRF, blocked hosts, etc.)
@@ -58,10 +58,33 @@ Once initialized, the gem automatically includes its modules into the appropriat
 - Rails `config.filter_parameters` are merged into LogStruct's filters and then cleared (to avoid double filtering). Configure sensitive keys via `LogStruct.config.filters`.
 - When `RAILS_LOG_TO_STDOUT` is set, we log to STDOUT only. Otherwise, we log to STDOUT by default without adding a file appender to avoid duplicate logs.
 
+### Development behavior
+
+- Disabled by default in development. Enable explicitly via `LOGSTRUCT_ENABLED=true` or `LogStruct.configure { |c| c.enabled = true }`.
+- When enabled in development, LogStruct now defaults to production‑style JSON output so you can preview exactly what will be shipped in prod.
+- You can opt back into the colorful human formatter with:
+
+```ruby
+LogStruct.configure do |c|
+  c.prefer_json_in_development = false
+  c.enable_color_output = true
+end
+```
+
 ## Documentation
 
 Please see the [documentation](https://logstruct.com/docs) for more details. (All code examples are type-checked and tested, and it's harder to keep a README up to date.)
 
+### Custom Typed Logs
+
+In addition to the built‑in, strictly typed log structures (Request, Error, SQL, etc.), you can define your own app‑specific typed logs while still using the public `LogStruct.info/error/...` methods.
+
+- Compose the public interfaces: include `LogStruct::Log::Interfaces::PublicCommonFields` and the helpers `SerializeCommonPublic` + `MergeAdditionalDataFields` in your `T::Struct`.
+- Fix your `source` to a constant (e.g., return the string `"payments"`), and restrict `event` with a `T::Enum` (e.g., `processed|failed|refunded`).
+- The `LogStruct.info` signature accepts either the internal `CommonFields` (for built‑ins) or your public custom type, so you keep type safety at the call site.
+
+See the docs page for a complete example: [Sorbet Types → Custom Typed Logs](https://logstruct.com/docs/sorbet-types#custom-log-classes).
+
 ## License
 
 This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.

data/lib/log_struct/boot_buffer.rb

@@ -0,0 +1,28 @@
+# typed: strict
+# frozen_string_literal: true
+
+module LogStruct
+  # Collects structured logs during very early boot before the logger is ready.
+  module BootBuffer
+    extend T::Sig
+
+    @@logs = T.let([], T::Array[LogStruct::Log::Interfaces::CommonFields])
+
+    sig { params(log: LogStruct::Log::Interfaces::CommonFields).void }
+    def self.add(log)
+      @@logs << log
+    end
+
+    sig { void }
+    def self.flush
+      return if @@logs.empty?
+      @@logs.each { |l| LogStruct.info(l) }
+      @@logs.clear
+    end
+
+    sig { void }
+    def self.clear
+      @@logs.clear
+    end
+  end
+end

data/lib/log_struct/builders/active_job.rb

@@ -0,0 +1,84 @@
+# typed: strict
+# frozen_string_literal: true
+
+require_relative "../log/active_job/enqueue"
+require_relative "../log/active_job/schedule"
+require_relative "../log/active_job/start"
+require_relative "../log/active_job/finish"
+
+module LogStruct
+  module Builders
+    module ActiveJob
+      extend T::Sig
+
+      sig { params(job: T.untyped).returns(T.nilable(String)) }
+      def self.safe_provider_job_id(job)
+        job.respond_to?(:provider_job_id) ? job.provider_job_id : nil
+      end
+
+      sig { params(job: T.untyped).returns(T.nilable(Integer)) }
+      def self.safe_executions(job)
+        job.respond_to?(:executions) ? job.executions : nil
+      end
+
+      sig { params(job: T.untyped).returns(T.nilable(T::Array[T.untyped])) }
+      def self.safe_arguments(job)
+        return nil unless job.class.respond_to?(:log_arguments?)
+        job.class.log_arguments? ? job.arguments : nil
+      end
+
+      sig { params(job: T.untyped).returns(Log::ActiveJob::Enqueue) }
+      def self.enqueue(job)
+        Log::ActiveJob::Enqueue.new(
+          job_id: job.job_id,
+          job_class: job.class.to_s,
+          queue_name: job.queue_name,
+          arguments: safe_arguments(job),
+          executions: safe_executions(job),
+          provider_job_id: safe_provider_job_id(job)
+        )
+      end
+
+      sig { params(job: T.untyped, scheduled_at: Time).returns(Log::ActiveJob::Schedule) }
+      def self.schedule(job, scheduled_at:)
+        Log::ActiveJob::Schedule.new(
+          job_id: job.job_id,
+          job_class: job.class.to_s,
+          queue_name: job.queue_name,
+          arguments: safe_arguments(job),
+          executions: safe_executions(job),
+          provider_job_id: safe_provider_job_id(job),
+          scheduled_at: scheduled_at
+        )
+      end
+
+      sig { params(job: T.untyped, started_at: Time, attempt: T.nilable(Integer)).returns(Log::ActiveJob::Start) }
+      def self.start(job, started_at:, attempt:)
+        Log::ActiveJob::Start.new(
+          job_id: job.job_id,
+          job_class: job.class.to_s,
+          queue_name: job.queue_name,
+          arguments: safe_arguments(job),
+          executions: safe_executions(job),
+          provider_job_id: safe_provider_job_id(job),
+          started_at: started_at,
+          attempt: attempt
+        )
+      end
+
+      sig { params(job: T.untyped, duration_ms: Float, finished_at: Time).returns(Log::ActiveJob::Finish) }
+      def self.finish(job, duration_ms:, finished_at:)
+        Log::ActiveJob::Finish.new(
+          job_id: job.job_id,
+          job_class: job.class.to_s,
+          queue_name: job.queue_name,
+          arguments: safe_arguments(job),
+          executions: safe_executions(job),
+          provider_job_id: safe_provider_job_id(job),
+          duration_ms: duration_ms,
+          finished_at: finished_at
+        )
+      end
+    end
+  end
+end

data/lib/log_struct/concerns/configuration.rb

@@ -41,16 +41,16 @@ module LogStruct
         def set_enabled_from_rails_env!
           # Set enabled based on current Rails environment and the LOGSTRUCT_ENABLED env var.
           # Precedence:
-          # 1. Check if LOGSTRUCT_ENABLED env var is defined
-          #    - Sets enabled=true only when value is "true"
-          #    - Sets enabled=false when value is "false" (or any non-"true")
+          # 1. Check if LOGSTRUCT_ENABLED env var is defined (not an empty string)
+          #    - Sets enabled=true only when value is "true", "yes", "1", etc.
+          #    - Sets enabled=false when value is any other value
           # 2. Otherwise, check if current Rails environment is in enabled_environments
           # 3. Otherwise, leave as config.enabled (defaults to true)
 
           # Then check if LOGSTRUCT_ENABLED env var is set
           config.enabled = if ENV["LOGSTRUCT_ENABLED"]
             # Override to true only if env var is "true"
-            ENV["LOGSTRUCT_ENABLED"] == "true"
+            %w[true t yes y 1].include?(ENV["LOGSTRUCT_ENABLED"]&.strip&.downcase)
           else
             config.enabled_environments.include?(::Rails.env.to_sym)
           end
@@ -72,20 +72,133 @@ module LogStruct
 
           rails_filter_params = ::Rails.application.config.filter_parameters
           return unless rails_filter_params.is_a?(Array)
+          return if rails_filter_params.empty?
+
+          symbol_filters = T.let([], T::Array[Symbol])
+          matchers = T.let([], T::Array[ConfigStruct::FilterMatcher])
+          leftovers = T.let([], T::Array[T.untyped])
+
+          rails_filter_params.each do |entry|
+            matcher = build_filter_matcher(entry)
+
+            if matcher
+              matchers << matcher
+              next
+            end
+
+            normalized_symbol = normalize_filter_symbol(entry)
+            if normalized_symbol
+              symbol_filters << normalized_symbol
+            else
+              leftovers << entry
+            end
+          end
+
+          if symbol_filters.any?
+            config.filters.filter_keys |= symbol_filters
+          end
 
-          # Convert all Rails filter parameters to symbols and merge with our filter keys
-          converted_params = rails_filter_params.map do |param|
-            param.respond_to?(:to_sym) ? param.to_sym : param
+          if matchers.any?
+            matchers.each do |matcher|
+              existing = config.filters.filter_matchers.any? do |registered|
+                registered.label == matcher.label
+              end
+              config.filters.filter_matchers << matcher unless existing
+            end
           end
 
-          # Add Rails filter parameters to our filter keys
-          config.filters.filter_keys += converted_params
+          replace_filter_parameters(rails_filter_params, leftovers)
+        end
+
+        private
+
+        sig { params(filter: T.untyped).returns(T.nilable(Symbol)) }
+        def normalize_filter_symbol(filter)
+          return filter if filter.is_a?(Symbol)
+          return filter.downcase.to_sym if filter.is_a?(String)
 
-          # Ensure no duplicates
-          config.filters.filter_keys.uniq!
+          return nil unless filter.respond_to?(:to_sym)
+
+          begin
+            sym = filter.to_sym
+            sym.is_a?(Symbol) ? sym : nil
+          rescue
+            nil
+          end
+        end
+
+        sig { params(filter: T.untyped).returns(T.nilable(ConfigStruct::FilterMatcher)) }
+        def build_filter_matcher(filter)
+          case filter
+          when ::Regexp
+            callable = Kernel.lambda do |key, _value|
+              filter.match?(key)
+            end
+            return ConfigStruct::FilterMatcher.new(callable: callable, label: filter.inspect)
+          else
+            return build_callable_filter_matcher(filter) if callable_filter?(filter)
+          end
+
+          nil
+        end
+
+        sig { params(filter: T.untyped).returns(T::Boolean) }
+        def callable_filter?(filter)
+          filter.respond_to?(:call)
+        end
+
+        sig { params(filter: T.untyped).returns(T.nilable(ConfigStruct::FilterMatcher)) }
+        def build_callable_filter_matcher(filter)
+          callable = Kernel.lambda do |key, value|
+            call_args = case arity_for_filter(filter)
+            when 0
+              []
+            when 1
+              [key]
+            else
+              [key, value]
+            end
+
+            result = filter.call(*call_args)
+            !!result
+          rescue ArgumentError
+            begin
+              !!filter.call(key)
+            rescue => e
+              handle_filter_error(e, filter, key)
+              false
+            end
+          rescue => e
+            handle_filter_error(e, filter, key)
+            false
+          end
+          ConfigStruct::FilterMatcher.new(callable: callable, label: filter.inspect)
+        end
+
+        sig { params(filter: T.untyped).returns(Integer) }
+        def arity_for_filter(filter)
+          filter.respond_to?(:arity) ? filter.arity : 2
+        end
+
+        sig { params(filter_params: T::Array[T.untyped], leftovers: T::Array[T.untyped]).void }
+        def replace_filter_parameters(filter_params, leftovers)
+          filter_params.clear
+          filter_params.concat(leftovers)
+        end
 
-          # Clear Rails filter parameters since we've incorporated them
-          ::Rails.application.config.filter_parameters.clear
+        sig { params(error: StandardError, filter: T.untyped, key: String).void }
+        def handle_filter_error(error, filter, key)
+          context = {
+            filter: filter.class.name,
+            key: key,
+            filter_label: begin
+              filter.inspect
+            rescue
+              "unknown"
+            end
+          }
+
+          LogStruct.handle_exception(error, source: Source::Internal, context: context)
         end
       end
     end

data/lib/log_struct/concerns/error_handling.rb

@@ -21,12 +21,12 @@ module LogStruct
           case source
           when Source::TypeChecking
             config.error_handling_modes.type_checking_errors
-          when Source::LogStruct
+          when Source::Internal
             config.error_handling_modes.logstruct_errors
           when Source::Security
             config.error_handling_modes.security_errors
           when Source::Rails, Source::App, Source::Job, Source::Storage, Source::Mailer,
-               Source::Shrine, Source::CarrierWave, Source::Sidekiq
+               Source::Shrine, Source::CarrierWave, Source::Sidekiq, Source::Dotenv, Source::Puma
             config.error_handling_modes.standard_errors
           else
             # Ensures the case statement is exhaustive
@@ -38,11 +38,7 @@ module LogStruct
         sig { params(error: StandardError, source: Source, context: T.nilable(T::Hash[Symbol, T.untyped])).void }
         def log_error(error, source:, context: nil)
           # Create structured log entry
-          error_log = Log::Error.from_exception(
-            source,
-            error,
-            context || {}
-          )
+          error_log = Log.from_exception(source, error, context || {})
           LogStruct.error(error_log)
         end
 

data/lib/log_struct/concerns/logging.rb

@@ -11,31 +11,31 @@ module LogStruct
         extend T::Sig
 
         # Log a log struct at debug level
-        sig { params(log: Log::Interfaces::CommonFields).void }
+        sig { params(log: T.any(Log::Interfaces::CommonFields, Log::Interfaces::PublicCommonFields)).void }
         def debug(log)
           Rails.logger.debug(log)
         end
 
         # Log a log struct at info level
-        sig { params(log: Log::Interfaces::CommonFields).void }
+        sig { params(log: T.any(Log::Interfaces::CommonFields, Log::Interfaces::PublicCommonFields)).void }
         def info(log)
           Rails.logger.info(log)
         end
 
         # Log a log struct at warn level
-        sig { params(log: Log::Interfaces::CommonFields).void }
+        sig { params(log: T.any(Log::Interfaces::CommonFields, Log::Interfaces::PublicCommonFields)).void }
         def warn(log)
           Rails.logger.warn(log)
         end
 
         # Log a log struct at error level
-        sig { params(log: Log::Interfaces::CommonFields).void }
+        sig { params(log: T.any(Log::Interfaces::CommonFields, Log::Interfaces::PublicCommonFields)).void }
         def error(log)
           Rails.logger.error(log)
         end
 
         # Log a log struct at fatal level
-        sig { params(log: Log::Interfaces::CommonFields).void }
+        sig { params(log: T.any(Log::Interfaces::CommonFields, Log::Interfaces::PublicCommonFields)).void }
         def fatal(log)
           Rails.logger.fatal(log)
         end

data/lib/log_struct/config_struct/filters.rb

@@ -3,6 +3,18 @@
 
 module LogStruct
   module ConfigStruct
+    class FilterMatcher < T::Struct
+      extend T::Sig
+
+      const :callable, T.proc.params(key: String, value: T.untyped).returns(T::Boolean)
+      const :label, String
+
+      sig { params(key: String, value: T.untyped).returns(T::Boolean) }
+      def matches?(key, value)
+        callable.call(key, value)
+      end
+    end
+
     class Filters < T::Struct
       include Sorbet::SerializeSymbolKeys
 
@@ -75,6 +87,12 @@ module LogStruct
       # Filter MAC addresses
       # Default: false
       prop :mac_addresses, T::Boolean, default: false
+
+      # Additional filter matchers built from Rails filter_parameters entries that aren't simple symbols.
+      # Each matcher receives the key (String) and optional value, returning true when the pair should be filtered.
+      prop :filter_matchers,
+        T::Array[FilterMatcher],
+        factory: -> { [] }
     end
   end
 end

data/lib/log_struct/config_struct/integrations.rb

@@ -60,18 +60,6 @@ module LogStruct
       # Default: true
       prop :enable_semantic_logger, T::Boolean, default: true
 
-      # Enable colored JSON output in development
-      # Default: true
-      prop :enable_color_output, T::Boolean, default: true
-
-      # Color configuration for JSON output
-      # Default: nil (uses SemanticLogger defaults)
-      prop :color_map, T.nilable(T::Hash[Symbol, Symbol]), default: nil
-
-      # Filter noisy loggers (ActionView, etc.)
-      # Default: false
-      prop :filter_noisy_loggers, T::Boolean, default: false
-
       # Enable SQL query logging through ActiveRecord instrumentation
       # Default: false (can be resource intensive)
       prop :enable_sql_logging, T::Boolean, default: false
@@ -84,6 +72,22 @@ module LogStruct
       # Include bind parameters in SQL logs (disable in production for security)
       # Default: true in development/test, false in production
       prop :sql_log_bind_params, T::Boolean, factory: -> { !defined?(::Rails) || !::Rails.respond_to?(:env) || !::Rails.env.production? }
+
+      # Enable Ahoy (analytics events) integration
+      # Default: true (safe no-op unless Ahoy is defined)
+      prop :enable_ahoy, T::Boolean, default: true
+
+      # Enable ActiveModelSerializers integration
+      # Default: true (safe no-op unless ActiveModelSerializers is defined)
+      prop :enable_active_model_serializers, T::Boolean, default: true
+
+      # Enable dotenv-rails integration (convert to structured logs)
+      # Default: true
+      prop :enable_dotenv, T::Boolean, default: true
+
+      # Enable Puma integration (convert server lifecycle logs)
+      # Default: true
+      prop :enable_puma, T::Boolean, default: true
     end
   end
 end

data/lib/log_struct/configuration.rb

@@ -20,6 +20,19 @@ module LogStruct
     prop :enabled, T::Boolean, default: true
     prop :enabled_environments, T::Array[Symbol], factory: -> { [:test, :production] }
     prop :local_environments, T::Array[Symbol], factory: -> { [:development, :test] }
+
+    # Prefer production-style JSON in development when LogStruct is enabled
+    prop :prefer_json_in_development, T::Boolean, default: true
+
+    # Enable colorful human formatter in development
+    prop :enable_color_output, T::Boolean, default: true
+
+    # Custom color map for the color formatter
+    prop :color_map, T.nilable(T::Hash[Symbol, Symbol]), default: nil
+
+    # Filter noisy loggers (ActionView, etc.)
+    prop :filter_noisy_loggers, T::Boolean, default: false
+
     const :integrations, ConfigStruct::Integrations, factory: -> { ConfigStruct::Integrations.new }
     const :filters, ConfigStruct::Filters, factory: -> { ConfigStruct::Filters.new }
 

data/lib/log_struct/enums/event.rb

@@ -26,10 +26,23 @@ module LogStruct
       Stream = new(:stream)
       Url = new(:url)
 
+      # Data generation events
+      Generate = new(:generate)
+
       # Email events
       Delivery = new(:delivery)
       Delivered = new(:delivered)
 
+      # Configuration / boot events
+      Load = new(:load)
+      Update = new(:update)
+      Save = new(:save)
+      Restore = new(:restore)
+
+      # Server lifecycle (e.g., Puma)
+      # Start already defined above
+      Shutdown = new(:shutdown)
+
       # Security events
       IPSpoof = new(:ip_spoof)
       CSRFViolation = new(:csrf_violation)