sumologic-query 1.3.4 → 1.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 72c4c0c9c57655df15506b9be19a21de47398ba2a2aedaae8e8d22d3efbd0873
-  data.tar.gz: 9cd38107b915187cb699e32305c8af8e6a970c2ea24c2b2340a443cc0a0e2637
+  metadata.gz: 5822fc268d1979e97d6993b7f233b720049f832c4d5d921a75c32ecccc16ce68
+  data.tar.gz: f9e1cde6a138f45d70ec4ed2e06b22f87faf9500a6cb4fb7274fe8df787bcba1
 SHA512:
-  metadata.gz: a073e490f4714e8f11c8c495775937dd42bb420535c38d82137a71a940b4b9db2b9fe33d4e6ec4b7745385a7661ec657db5472b9761dbecc110fb473efca757f
-  data.tar.gz: 3a9aa83222b99b34fbcdceb45f2e1cdd126c29d28ba4a93db818ea34438b366f7a7b5eb0290f7986e0a9ef9d5deaed95465cca45a72468ec432ec5d6d97a29f4
+  metadata.gz: 661862b1dfbc17a5729fbffc54c459e9b2e52a996346b0c028179002c606689096c0a191a26d7bb06d737c197f50324eb86e574289a093c4a392b9248a3e2647
+  data.tar.gz: fda3e751c1dc953fa10d878288193ffe060dcd9366e9d18e8cad9d8eeaf8a194e0dcaa1f50ca245fdc5ab20a95c7f1b67809add3dbed2aa28341f68130188ffd

data/CHANGELOG.md

@@ -6,6 +6,54 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and release notes are automatically generated from commit messages.
+## [1.4.0](https://github.com/patrick204nqh/sumologic-query/compare/v1.3.5...v1.4.0) (2026-02-11)
+
+### 🎉 New Features
+
+- add comprehensive skills documentation for Sumo Logic CLI commands
+- add export-content command with async job handling
+- add get-content command for path-based content lookup
+- add get-lookup and list-apps commands
+- add list-health-events and list-fields commands
+- migrate monitors to search API with status/query filters
+- add support for retrieving monitors and dashboards in the CLI, refactor monitor collection logic for improved readability
+- Enhance Sumo Logic CLI with monitors, folders, and dashboards commands
+- implement aggregation queries and enhance error handling for rate limits
+
+### 🐛 Bug Fixes
+
+- migrate dashboards from v1 to v2 API
+
+### 🔧 Refactoring
+
+- clean design for AI agent readiness (v1.4.0)
+
+### 📚 Documentation
+
+- update naming conventions to match actual codebase
+- reorganize into SDLC structure
+- clarify discover-sources as search-based technique
+- revise architecture overview with enhanced clarity on design philosophy, component structure, and key features
+
+
+
+## [1.3.5](https://github.com/patrick204nqh/sumologic-query/compare/v1.3.4...v1.3.5) (2025-11-19)
+
+### 🎉 New Features
+
+- add discover-sources command for dynamic source discovery from logs
+- implement rate limiting configuration and enhance documentation for querying options
+
+### 🔧 Refactoring
+
+- improve source discovery logic and enhance debugging output
+
+### 📚 Documentation
+
+- update tldr.md with enhanced search options and new commands for querying logs
+
+
+
 ## [1.3.4](https://github.com/patrick204nqh/sumologic-query/compare/v1.3.3...v1.3.4) (2025-11-19)
 
 ### 🎉 New Features

data/README.md

@@ -6,351 +6,166 @@
 [![Downloads](https://img.shields.io/gem/dt/sumologic-query.svg)](https://rubygems.org/gems/sumologic-query)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-## Why This Tool?
+## Features
 
-- **Intuitive time parsing**: Use relative times like `-1h`, `-30m`, or `now` - no more calculating timestamps!
-- **Flexible timezone support**: US, Australian, and IANA timezone formats supported
-- **Minimal dependencies**: Uses only Ruby stdlib + Thor for CLI
-- **Fast queries**: Efficient polling and automatic pagination
-- **Interactive mode**: Explore logs with FZF-powered fuzzy search and preview
-- **Simple interface**: Just query, get results, done
-- **Read-only**: No write operations, perfect for safe log access
-- **Modular architecture**: Clean separation of concerns (HTTP, Search, Metadata)
-- **Metadata support**: List collectors and sources alongside log queries
-
-All existing Ruby Sumo Logic gems are unmaintained (2-9 years dormant). This tool provides a fresh, minimal approach focused on querying logs and metadata.
+- **Simple time parsing** - Use `-1h`, `-30m`, `now` instead of timestamps
+- **Dynamic source discovery** - Find CloudWatch/ECS/Lambda sources from logs
+- **Interactive mode** - Explore logs with FZF fuzzy search
+- **Timezone support** - US, Australian, and IANA formats
+- **Fast & efficient** - Smart polling and pagination
+- **Read-only** - Safe log access with no write operations
 
 ## Installation
 
-### Via RubyGems
-
 ```bash
+# Via RubyGems
 gem install sumologic-query
-```
 
-### Via Homebrew
-
-```bash
+# Via Homebrew
 brew tap patrick204nqh/tap
 brew install sumologic-query
 ```
 
-### From Source
-
-```bash
-git clone https://github.com/patrick204nqh/sumologic-query.git
-cd sumologic-query
-bundle install
-bundle exec rake install
-```
-
 ## Quick Start
 
-### 1. Set Up Credentials
-
-Export your Sumo Logic API credentials:
+### 1. Set Credentials
 
 ```bash
 export SUMO_ACCESS_ID="your_access_id"
 export SUMO_ACCESS_KEY="your_access_key"
-export SUMO_DEPLOYMENT="us2"  # Optional: us1, us2 (default), eu, au, etc.
+export SUMO_DEPLOYMENT="us2"  # Optional: us1, us2 (default), eu, au
 ```
 
-**Getting credentials:**
-1. Log in to Sumo Logic
-2. Go to **Administration → Security → Access Keys**
-3. Create a new access key or use existing
-4. Copy the Access ID and Access Key
+Get credentials: Sumo Logic → **Administration → Security → Access Keys**
 
-### 2. Run Your First Query
+### 2. Run Queries
 
 ```bash
-# Search logs from last hour (easy!)
-sumo-query search --query 'error' --from '-1h' --to 'now' --limit 10
-
-# Search logs from last 30 minutes
-sumo-query search --query 'error' --from '-30m' --to 'now'
+# Search logs
+sumo-query search -q 'error' -f '-1h' -t 'now' --limit 100
 
-# Or use ISO 8601 format
-sumo-query search --query 'error' \
-  --from '2025-11-13T14:00:00' \
-  --to '2025-11-13T15:00:00' \
-  --limit 10
+# Discover dynamic sources (CloudWatch/ECS/Lambda)
+sumo-query discover-sources
 
-# List collectors
+# List collectors and sources
 sumo-query collectors
-
-# List sources
 sumo-query sources
 ```
 
-## Usage
+## Commands
 
-The CLI provides three main commands:
-
-### Search Logs
+### 1. Search Logs
 
 ```bash
-sumo-query search --query "YOUR_QUERY" \
-  --from "START_TIME" \
-  --to "END_TIME" \
-  [--output FILE] \
-  [--limit N] \
-  [--time-zone TZ] \
-  [--interactive]
+sumo-query search -q "YOUR_QUERY" -f "START" -t "END" [OPTIONS]
 ```
 
-**Required options:**
-- `-q, --query QUERY` - Sumo Logic query string
-- `-f, --from TIME` - Start time (ISO 8601 format)
-- `-t, --to TIME` - End time (ISO 8601 format)
-
-**Optional options:**
-- `-i, --interactive` - Launch interactive browser with FZF
-- `-z, --time-zone TZ` - Time zone (default: UTC)
-- `-l, --limit N` - Limit number of messages
-- `-o, --output FILE` - Save to file (default: stdout)
-- `-d, --debug` - Enable debug output
+**Options:**
+- `-q, --query` - Query string (required)
+- `-f, --from` - Start time (required, e.g., `-1h`, `2025-11-19T14:00:00`)
+- `-t, --to` - End time (required, e.g., `now`)
+- `-z, --time-zone` - Timezone (default: UTC)
+- `-l, --limit` - Max messages to return
+- `-o, --output` - Save to file
+- `-i, --interactive` - Launch FZF browser
+- `-d, --debug` - Debug output
 
-### Interactive Mode 🚀
+**Interactive Mode** (`-i`): FZF-based browser with fuzzy search, preview, and multi-select. Requires `fzf` ([install](https://github.com/junegunn/fzf#installation)).
 
-Explore your logs interactively with a powerful FZF-based interface:
+### 2. Discover Dynamic Sources
 
 ```bash
-# Launch interactive mode - last hour
-sumo-query search --query 'error' --from '-1h' --to 'now' --interactive
+sumo-query discover-sources [OPTIONS]
+```
 
-# Last 30 minutes with shorthand
-sumo-query search -q 'error' -f '-30m' -t 'now' -i
+Discovers source names from log data using search aggregation (`* | count by _sourceName, _sourceCategory`). This is not an official Sumo Logic API — it complements `list-sources` by finding runtime sources (CloudWatch, ECS, Lambda streams) that use dynamic `_sourceName` values.
 
-# Or use ISO 8601 format
-sumo-query search -q 'error' -f '2025-11-13T14:00:00' -t '2025-11-13T15:00:00' -i
-```
+**Options:**
+- `-f, --from` - Start time (default: `-24h`)
+- `-t, --to` - End time (default: `now`)
+- `--filter` - Filter query (e.g., `_sourceCategory=*ecs*`)
+- `-z, --time-zone` - Timezone (default: UTC)
+- `-o, --output` - Save to file
 
-**Features:**
-- 🔍 Fuzzy search across all message fields
-- 👁️ Live preview with full JSON details
-- 🎨 Color-coded log levels (ERROR, WARN, INFO)
-- ⌨️ Keyboard shortcuts for quick actions
-- 📋 Multi-select and batch operations
-- 💾 Export selected messages
-
-**Keybindings:**
-- `Enter` - Toggle selection (mark/unmark message)
-- `Tab` - Open current message in pager (copyable view)
-- `Ctrl-S` - Save selected messages to `sumo-selected.txt` and exit
-- `Ctrl-Y` - Copy selected messages to clipboard and exit
-- `Ctrl-E` - Export selected messages to `sumo-export.jsonl` and exit
-- `Ctrl-A` - Select all messages
-- `Ctrl-D` - Deselect all messages
-- `Ctrl-/` - Toggle preview pane
-- `Ctrl-Q` - Quit without saving
-
-**Requirements:**
-- Install FZF: `brew install fzf` (macOS) or `apt-get install fzf` (Linux)
-- See: https://github.com/junegunn/fzf#installation
-
-### Time Format Examples
-
-Combine relative times with timezones for powerful queries:
+**Examples:**
 
 ```bash
-# Last hour in Sydney time
-sumo-query search -q 'error' -f '-1h' -t 'now' -z AEST
+# Discover all sources from last 24 hours
+sumo-query discover-sources
 
-# Last 30 minutes in US Eastern time
-sumo-query search -q 'error' -f '-30m' -t 'now' -z EST
+# Filter to ECS only
+sumo-query discover-sources --filter '_sourceCategory=*ecs*'
 
-# Last 7 days with output to file (directories auto-created)
-sumo-query search -q 'error' -f '-7d' -t 'now' -o logs/weekly/errors.json
+# Last 7 days, save to file
+sumo-query discover-sources -f '-7d' -o sources.json
+```
 
-# Mix relative and ISO 8601 formats
-sumo-query search -q 'error' -f '-24h' -t '2025-11-19T12:00:00'
+### 3. List Collectors & Sources
 
-# Unix timestamps from last hour to now
-sumo-query search -q 'error' -f '1700000000' -t 'now'
+```bash
+# List collectors
+sumo-query collectors [-o FILE]
+
+# List static sources
+sumo-query sources [-o FILE]
 ```
 
-### List Collectors
+## Time Formats
 
 ```bash
-sumo-query collectors [--output FILE]
-```
+# Relative (recommended)
+-1h, -30m, -7d, now
 
-Lists all collectors in your account with status and metadata.
+# ISO 8601
+2025-11-19T14:00:00
 
-### List Sources
+# Unix timestamp
+1700000000
 
-```bash
-sumo-query sources [--output FILE]
+# Timezones
+UTC, AEST, EST, America/New_York, Australia/Sydney, +10:00
 ```
 
-Lists all sources from active collectors.
-
+See [examples/queries.md](examples/queries.md) for comprehensive query patterns.
 
-## Ruby Library Usage
+## Ruby Library
 
 ```ruby
 require 'sumologic'
 
-# Initialize client
 client = Sumologic::Client.new(
   access_id: ENV['SUMO_ACCESS_ID'],
-  access_key: ENV['SUMO_ACCESS_KEY'],
-  deployment: 'us2'
-)
-
-# Search logs
-results = client.search(
-  query: 'error',
-  from_time: '2025-11-13T14:00:00',
-  to_time: '2025-11-13T15:00:00',
-  time_zone: 'UTC',
-  limit: 1000
+  access_key: ENV['SUMO_ACCESS_KEY']
 )
 
-# List collectors and sources
-collectors = client.list_collectors
-sources = client.list_all_sources
-```
+# Search
+client.search(query: 'error', from_time: '-1h', to_time: 'now')
 
-**Time parsing utilities:**
+# Discover sources
+client.discover_dynamic_sources(from_time: '-24h', to_time: 'now')
 
-```ruby
-require 'sumologic/utils/time_parser'
-
-# Parse relative times and timezones
-from_time = Sumologic::Utils::TimeParser.parse('-1h')
-timezone = Sumologic::Utils::TimeParser.parse_timezone('AEST')
+# Metadata
+client.list_collectors
+client.list_all_sources
 ```
 
-
-## Time Formats
-
-Multiple time formats are supported:
-
-```bash
-# Relative time (easiest!)
-sumo-query search -q 'error' -f '-1h' -t 'now'
-sumo-query search -q 'error' -f '-30m' -t 'now'
-
-# ISO 8601
-sumo-query search -q 'error' -f '2025-11-13T14:00:00' -t '2025-11-13T15:00:00'
-
-# Unix timestamps
-sumo-query search -q 'error' -f '1700000000' -t 'now'
-
-# With timezones
-sumo-query search -q 'error' -f '-1h' -t 'now' -z 'AEST'
-sumo-query search -q 'error' -f '-1h' -t 'now' -z 'America/New_York'
-```
-
-**Supported time units:** `s`, `m`, `h`, `d`, `w`, `M`, `now`
-
-**Supported timezones:** IANA names (`UTC`, `America/New_York`, `Australia/Sydney`), US abbreviations (`EST`, `PST`), Australian abbreviations (`AEST`, `ACST`, `AWST`), UTC offsets (`+10:00`)
-
-See [examples/time-formats.md](examples/time-formats.md) for comprehensive examples.
-
-## Output Format
-
-Results are returned as JSON:
-
-```json
-{
-  "query": "error",
-  "from": "2025-11-13T14:00:00",
-  "to": "2025-11-13T15:00:00",
-  "time_zone": "UTC",
-  "message_count": 42,
-  "messages": [
-    {
-      "map": {
-        "_messagetime": "1731506400123",
-        "_sourceCategory": "prod/api",
-        "_sourceName": "api-server-01",
-        "message": "Error processing request: timeout"
-      }
-    }
-  ]
-}
-```
-
-## Performance
-
-Query execution time depends on data volume:
-
-| Messages | Typical Time |
-|----------|--------------|
-| < 10K | 30-60 seconds |
-| 10K-100K | 1-2 minutes |
-| 100K+ | 2-5 minutes |
-
-**Tips for faster queries:**
-- Narrow your time range
-- Add `_sourceCategory` filters
-- Use `--limit` to cap results
-- Use aggregation queries instead of raw messages
-
 ## Documentation
 
-- **[Quick Reference (tldr)](docs/tldr.md)** - Concise command examples
-- **[Query Examples](examples/queries.md)** - Common query patterns
-- **[Time Format Examples](examples/time-formats.md)** - Time parsing and timezone options
-- **[Architecture](docs/architecture/)** - Design and architecture decisions
-
-## Development
-
-See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, testing, and contribution guidelines.
-
-Quick start:
-
-```bash
-# Clone and install
-git clone https://github.com/patrick204nqh/sumologic-query.git
-cd sumologic-query
-bundle install
-
-# Run tests (73+ specs including time parser tests)
-bundle exec rspec
-
-# Run linter
-bundle exec rubocop
-
-# Test locally with new time formats
-bundle exec bin/sumo-query search --query "error" \
-  --from "-1h" --to "now"
-
-# Test with timezone support
-bundle exec bin/sumo-query search --query "error" \
-  --from "-30m" --to "now" --time-zone "AEST"
-```
+- [Query Examples](examples/queries.md) - Query patterns and examples
+- [Quick Reference](docs/tldr.md) - Command cheat sheet
+- [Rate Limiting](docs/rate-limiting.md) - Performance tuning
+- [Architecture](docs/architecture/) - Design decisions
 
 ## Contributing
 
-Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details.
-
-1. Fork the repository
-2. Create your feature branch (`git checkout -b feature/amazing-feature`)
-3. Commit your changes (`git commit -m 'Add amazing feature'`)
-4. Push to the branch (`git push origin feature/amazing-feature`)
-5. Open a Pull Request
+See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
 
 ## License
 
-This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
-
-## Support
-
-- **Issues**: [GitHub Issues](https://github.com/patrick204nqh/sumologic-query/issues)
-- **Discussions**: [GitHub Discussions](https://github.com/patrick204nqh/sumologic-query/discussions)
-- **Documentation**: [docs/](docs/)
-
-## Resources
-
-- **Sumo Logic API Docs**: https://help.sumologic.com/docs/api/search-job/
-- **Query Language**: https://help.sumologic.com/docs/search/
-- **Bug Reports**: https://github.com/patrick204nqh/sumologic-query/issues
+MIT License - see [LICENSE](LICENSE) file.
 
----
+## Links
 
-**Note**: This tool provides read-only access to Sumo Logic logs. It does not modify any data or configuration.
+- [Issues](https://github.com/patrick204nqh/sumologic-query/issues)
+- [Sumo Logic API Docs](https://help.sumologic.com/docs/api/search-job/)
+- [Query Language](https://help.sumologic.com/docs/search/)

data/lib/sumologic/cli/commands/base_command.rb

@@ -31,26 +31,6 @@ module Sumologic
             puts json_output
           end
         end
-
-        def format_collector(collector)
-          {
-            id: collector['id'],
-            name: collector['name'],
-            collectorType: collector['collectorType'],
-            alive: collector['alive'],
-            category: collector['category']
-          }
-        end
-
-        def format_source(source)
-          {
-            id: source['id'],
-            name: source['name'],
-            category: source['category'],
-            sourceType: source['sourceType'],
-            alive: source['alive']
-          }
-        end
       end
     end
   end

data/lib/sumologic/cli/commands/discover_source_metadata_command.rb

@@ -0,0 +1,59 @@
+# frozen_string_literal: true
+
+require_relative 'base_command'
+require_relative '../../utils/time_parser'
+
+module Sumologic
+  class CLI < Thor
+    module Commands
+      # Handles the discover-source-metadata command execution
+      class DiscoverSourceMetadataCommand < BaseCommand
+        def execute
+          parse_time_options
+          log_discovery_info
+          results = perform_discovery
+
+          output_json(results)
+        end
+
+        private
+
+        def parse_time_options
+          # Parse time formats and store both original and parsed values
+          @original_from = options[:from]
+          @original_to = options[:to]
+          @parsed_from = Utils::TimeParser.parse(options[:from])
+          @parsed_to = Utils::TimeParser.parse(options[:to])
+          @parsed_timezone = Utils::TimeParser.parse_timezone(options[:time_zone])
+        rescue Utils::TimeParser::ParseError => e
+          warn "Error parsing time: #{e.message}"
+          exit 1
+        end
+
+        def log_discovery_info
+          warn '=' * 60
+          warn 'Discovering Source Metadata'
+          warn '=' * 60
+          warn "Time Range: #{@original_from} to #{@original_to}"
+          if @original_from != @parsed_from || @original_to != @parsed_to
+            warn "  (Parsed: #{@parsed_from} to #{@parsed_to})"
+          end
+          warn "Time Zone: #{@parsed_timezone}"
+          warn "Filter: #{options[:filter] || 'none (all sources)'}"
+          warn '-' * 60
+          warn 'Running aggregation query to discover sources...'
+          $stderr.puts
+        end
+
+        def perform_discovery
+          client.discover_source_metadata(
+            from_time: @parsed_from,
+            to_time: @parsed_to,
+            time_zone: @parsed_timezone,
+            filter: options[:filter]
+          )
+        end
+      end
+    end
+  end
+end

data/lib/sumologic/cli/commands/export_content_command.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+require_relative 'base_command'
+
+module Sumologic
+  class CLI < Thor
+    module Commands
+      # Handles the export-content command execution
+      class ExportContentCommand < BaseCommand
+        def execute
+          content_id = options[:content_id]
+          warn "Exporting content #{content_id}..."
+          result = client.export_content(content_id: content_id)
+
+          output_json(result)
+        end
+      end
+    end
+  end
+end

data/lib/sumologic/cli/commands/get_content_command.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+require_relative 'base_command'
+
+module Sumologic
+  class CLI < Thor
+    module Commands
+      # Handles the get-content command execution
+      class GetContentCommand < BaseCommand
+        def execute
+          path = options[:path]
+          warn "Looking up content at path: #{path}..."
+          content = client.get_content(path: path)
+
+          output_json(content)
+        end
+      end
+    end
+  end
+end

data/lib/sumologic/cli/commands/get_dashboard_command.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+require_relative 'base_command'
+
+module Sumologic
+  class CLI < Thor
+    module Commands
+      # Handles the get-dashboard command execution
+      class GetDashboardCommand < BaseCommand
+        def execute
+          dashboard_id = options[:dashboard_id]
+          warn "Fetching dashboard #{dashboard_id}..."
+          dashboard = client.get_dashboard(dashboard_id: dashboard_id)
+
+          output_json(dashboard)
+        end
+      end
+    end
+  end
+end

data/lib/sumologic/cli/commands/get_lookup_command.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+require_relative 'base_command'
+
+module Sumologic
+  class CLI < Thor
+    module Commands
+      # Handles the get-lookup command execution
+      class GetLookupCommand < BaseCommand
+        def execute
+          lookup_id = options[:lookup_id]
+          warn "Fetching lookup table #{lookup_id}..."
+          lookup = client.get_lookup(lookup_id: lookup_id)
+
+          output_json(lookup)
+        end
+      end
+    end
+  end
+end