csvops 0.7.0.alpha → 0.9.0.alpha

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 803fa825ef1f50edcd7c0bc032a86926d356cb3ba6d943c460d59759a953fdcd
-  data.tar.gz: 2ba2afc9951aa96e777cbf3ea81dc77a41c88d2546505c885302607432461633
+  metadata.gz: 940c2492f5bea33d56ad0a47ebf6933cb2e43817530aa04d4e02950affe9d493
+  data.tar.gz: 9c32a162b4393f25e99b55e908df2b9dcb55099ce0d820aa3255c310fc09d983
 SHA512:
-  metadata.gz: 4f82dd7e9d3ac5ff53f8aaf40a0e5500e9b074aa052a031f6de4f5a2cc1ab711a5c375d5c203bdfaae802d36a02ecf14c4f73231a9f14e31d2f042ffeecd9a08
-  data.tar.gz: f9428d2ef29d257c99b484c7277dcff566dd5cf09ec06b78b4514c410b7858ffd6854f8aafd39a727c9c3d1e44e6940bc15456f3b11fdcac4a5b879bee9cc826
+  metadata.gz: 3babceb657e3e3c366c19daa305fcffe18fca1e145dafa8d8f66f022531e17fc21f0788da7b27a0d76e17e07c88fa098105d20116353cced5e4b207870a0f88e
+  data.tar.gz: a8efe4b0a86dd3e303ca8946a0b25b3dbcffb1bbeb76823910d43ab8ff1ba4fc7670f1a2c16e135941caf74a3a216f988010bd0178b892a60eb9bd82963c4596

data/README.md

@@ -39,30 +39,28 @@ CSV Tool Menu
 4. Dedupe using another CSV
 5. Validate parity
 6. Split CSV into chunks
-7. Exit
+7. CSV stats summary
+8. Exit
 >
 ```
 
-Select `1` for column extraction, `2` for row-range extraction, `3` for row randomization, `4` for cross-CSV dedupe, `5` for parity validation, or `6` for CSV splitting.
+Select `1` for column extraction, `2` for row-range extraction, `3` for row randomization, `4` for cross-CSV dedupe, `5` for parity validation, `6` for CSV splitting, or `7` for CSV stats.
 
 ### 3. Follow prompts
 
-Each menu action runs through a dedicated CLI workflow (`interface/cli/workflows/*`) that handles prompts/output and delegates execution to an interface-agnostic application use case.
+Each action asks only for what it needs (file path, separator, and any action-specific options), then prints results to the console or writes to a file when selected.
 
-Workflow internals are split into small composable parts:
+Typical prompt pattern:
 
-- `workflows/builders/*` for session construction
-- `workflows/support/*` for shared mapping/dispatch utilities
-- `workflows/presenters/*` for output formatting and summaries
+- choose source file(s)
+- choose separator/header options when relevant
+- choose action-specific options
+- choose output destination (console or file)
 
-Prompt flow by action:
+For architecture and internal design details, see:
 
-- `Extract column`: file path, separator, optional header filter + column select, skip blanks, preview/confirm, output destination.
-- `Extract rows (range)`: file path, separator, start row, end row, output destination.
-- `Randomize rows`: file path, separator, headers present, optional seed, output destination.
-- `Dedupe using another CSV`: source/reference files, separators, header modes, key selectors, match options, output destination.
-- `Validate parity`: left/right files, separator, header mode, parity summary, mismatch samples.
-- `Split CSV into chunks`: source file, separator, header mode, chunk size, output directory/prefix, overwrite policy, optional manifest.
+- [`docs/architecture.md`](docs/architecture.md)
+- [`docs/cli-output-conventions.md`](docs/cli-output-conventions.md)
 
 ### 4. Example interaction (console output)
 
@@ -123,6 +121,22 @@ With Bundler:
 bundle exec csvtool column /path/to/file.csv column_name
 ```
 
+Get CSV stats directly (default text output):
+
+```bash
+csvtool stats /path/to/file.csv
+```
+
+Optional output format and color mode:
+
+```bash
+csvtool stats /path/to/file.csv --format json
+csvtool stats /path/to/file.csv --format csv
+csvtool stats /path/to/file.csv --color auto
+csvtool stats /path/to/file.csv --color always
+csvtool stats /path/to/file.csv --color never
+```
+
 ### 7. Dedupe interaction example
 
 Legend: ` ` = prompt/menu, `+` = user input, `-` = tool output
@@ -134,8 +148,9 @@ Legend: ` ` = prompt/menu, `+` = user input, `-` = tool output
 3. Randomize rows
 4. Dedupe using another CSV
 5. Validate parity
- 6. Split CSV into chunks
- 7. Exit
+6. Split CSV into chunks
+ 7. CSV stats summary
+ 8. Exit
 +> 4
  CSV file path: /tmp/source.csv
  Source CSV separator:
@@ -184,7 +199,8 @@ Legend: ` ` = prompt/menu, `+` = user input, `-` = tool output
 4. Dedupe using another CSV
 5. Validate parity
  6. Split CSV into chunks
- 7. Exit
+ 7. CSV stats summary
+ 8. Exit
 +> 5
  Left CSV file path: /tmp/left.csv
  Right CSV file path: /tmp/right.csv
@@ -224,7 +240,8 @@ Legend: ` ` = prompt/menu, `+` = user input, `-` = tool output
  4. Dedupe using another CSV
  5. Validate parity
  6. Split CSV into chunks
- 7. Exit
+ 7. CSV stats summary
+ 8. Exit
 +> 6
  Source CSV file path: /tmp/people.csv
  Choose separator:
@@ -247,6 +264,48 @@ Legend: ` ` = prompt/menu, `+` = user input, `-` = tool output
 -/tmp/people_part_001.csv
 ```
 
+### 11. CSV stats interaction example
+
+Legend: ` ` = prompt/menu, `+` = user input, `-` = tool output
+
+```diff
+ CSV Tool Menu
+ 1. Extract column
+ 2. Extract rows (range)
+ 3. Randomize rows
+ 4. Dedupe using another CSV
+ 5. Validate parity
+ 6. Split CSV into chunks
+ 7. CSV stats summary
+ 8. Exit
++> 7
+ CSV file path: /tmp/people.csv
+ Choose separator:
+ 1. comma (,)
+ 2. tab (\t)
+ 3. semicolon (;)
+ 4. pipe (|)
+ 5. custom
++Separator choice [1]: 1
+ Headers present? [Y/n]:
+ Output destination:
+ 1. console
+ 2. file
++Output destination [1]: 1
+-CSV Stats Summary
+-Rows: 3
+-Columns: 2
+-Headers: name, city
+-Column completeness:
+-  name: non_blank=3 blank=0
+-  city: non_blank=3 blank=0
+```
+
+### 12. CSV stats large-file behavior
+
+- Stats scanning is streaming (`CSV.foreach`), processed in one pass.
+- Memory grows with per-column aggregates (`column_stats`), not with total row count.
+
 ## Testing
 
 Run tests:
@@ -263,7 +322,7 @@ bundle exec rake test
 
 ## Alpha release
 
-Current prerelease version: `0.7.0.alpha`
+Current prerelease version: `0.9.0.alpha`
 
 Install prerelease from RubyGems:
 
@@ -273,7 +332,7 @@ gem install csvops --pre
 
 Release runbook:
 
-- `docs/release-v0.7.0-alpha.md`
+- `docs/release-v0.9.0-alpha.md`
 
 
 ## Architecture
@@ -281,3 +340,4 @@ Release runbook:
 Full architecture and domain documentation lives in:
 
 - [`docs/architecture.md`](docs/architecture.md)
+- [`docs/cli-output-conventions.md`](docs/cli-output-conventions.md)

data/docs/architecture.md

@@ -2,15 +2,15 @@
 
 The codebase follows a DDD-lite layered structure:
 
-- `domain/`: core domain models and invariants (`ColumnSession`, `RowSession`, `RandomizationSession`, `CrossCsvDedupeSession`, and `CsvSplitSession` aggregates + supporting entities/value objects).
-- `application/`: use-case orchestration (`RunExtraction`, `RunRowExtraction`, `RunRowRandomization`, `RunCrossCsvDedupe`, `RunCsvParity`, `RunCsvSplit`).
+- `domain/`: core domain models and invariants (`ColumnSession`, `RowSession`, `RandomizationSession`, `CrossCsvDedupeSession`, `ParitySession`, `SplitSession`, and `CsvStatsSession` aggregates + supporting entities/value objects).
+- `application/`: use-case orchestration (`RunExtraction`, `RunRowExtraction`, `RunRowRandomization`, `RunCrossCsvDedupe`, `RunCsvParity`, `RunCsvSplit`, `RunCsvStats`).
 - `infrastructure/`: CSV reading/streaming/comparison and output adapters (console/file).
 - `interface/cli/`: menu, prompts, workflows, and user-facing error presentation.
 - `Csvtool::CLI`: entrypoint wiring from command args to interface/application flow.
 
 ## Workflow boundary (standardized)
 
-For all interactive domains (`Column Extraction`, `Row Extraction`, `Row Randomization`, `Cross-CSV Dedupe`, `CSV Parity`, `CSV Split`), the boundary is:
+For all interactive domains (`Column Extraction`, `Row Extraction`, `Row Randomization`, `Cross-CSV Dedupe`, `CSV Parity`, `CSV Split`, `CSV Stats`), the boundary is:
 
 - `interface/cli/workflows/*`: owns prompts, stdout rendering, and user-facing error presentation.
 - `interface/cli/workflows/builders/*`: builds domain sessions/aggregates from prompt results.
@@ -21,6 +21,9 @@ For all interactive domains (`Column Extraction`, `Row Extraction`, `Row Randomi
 - `domain/*`: invariants and domain policies.
 - `infrastructure/*`: CSV mechanics and output adapters.
 
+Output UI rules:
+- See [`docs/cli-output-conventions.md`](cli-output-conventions.md) for stream, format, color, and table rendering contracts used across workflows.
+
 Write-boundary rule:
 - Use cases coordinate write paths but do not perform direct file writes.
 - Direct write APIs (`CSV.open`, writable `File.open`, `File.write`, `IO.write`) are infrastructure-only.
@@ -34,6 +37,7 @@ Current usage:
 - `RunCrossCsvDedupeWorkflow` uses `WorkflowStepPipeline` + `Steps::CrossCsvDedupe::*`.
 - `RunCsvParityWorkflow` uses `WorkflowStepPipeline` + `Steps::Parity::*`.
 - `RunCsvSplitWorkflow` uses `WorkflowStepPipeline` + `Steps::CsvSplit::*`.
+- `RunCsvStatsWorkflow` uses `WorkflowStepPipeline` + `Steps::CsvStats::*`.
 
 ## Adding New Concepts
 
@@ -109,7 +113,7 @@ For a new function type, prefer one of these patterns:
 
 ## Domain model
 
-Bounded contexts: `Column Extraction`, `Row Extraction`, `Row Randomization`, `Cross-CSV Dedupe`, `CSV Parity`, and `CSV Split`.
+Bounded contexts: `Column Extraction`, `Row Extraction`, `Row Randomization`, `Cross-CSV Dedupe`, `CSV Parity`, `CSV Split`, and `CSV Stats`.
 
 ### Cross-CSV Dedupe (Large-file behavior)
 
@@ -476,6 +480,63 @@ classDiagram
   RunCsvSplit --> CsvSplitManifestWriter
 ```
 
+### CSV Stats
+
+Core DDD structure:
+
+- Aggregate root: `StatsSession`
+  - Captures one stats summary request.
+  - Holds source profile and output destination.
+- Entity:
+  - `StatsSource` (path + separator + header mode)
+- Value objects:
+  - `StatsOptions` (currently lightweight; keeps option growth explicit)
+  - Shared `OutputDestination` (`console` or `file(path)`)
+- Application service:
+  - `Application::UseCases::RunCsvStats` orchestrates stats scanning and output routing.
+- Infrastructure adapters:
+  - `Infrastructure::CSV::CsvStatsScanner` (streaming one-pass row aggregation)
+  - `Infrastructure::Output::CsvStatsFileWriter` (metric/value artifact writer)
+- Interface adapters:
+  - `Interface::CLI::MenuLoop`
+  - `Interface::CLI::Workflows::RunCsvStatsWorkflow`
+  - `Interface::CLI::Workflows::Builders::CsvStatsSessionBuilder`
+  - `Interface::CLI::Workflows::Steps::WorkflowStepPipeline`
+  - `Interface::CLI::Workflows::Steps::CsvStats::*`
+  - `Interface::CLI::Workflows::Presenters::CsvStatsPresenter`
+  - `Interface::CLI::Workflows::Support::{OutputDestinationMapper,ResultErrorHandler}`
+  - `Interface::CLI::Prompts::*`
+  - `Interface::CLI::Errors::Presenter`
+
+```mermaid
+classDiagram
+  direction LR
+  class MenuLoop
+  class RunCsvStatsWorkflow
+  class Prompts
+  class Errors
+  class RunCsvStats
+  class StatsSession
+  class StatsSource
+  class StatsOptions
+  class OutputDestination
+  class CsvStatsScanner
+  class CsvStatsFileWriter
+  class CsvStatsPresenter
+
+  MenuLoop --> RunCsvStatsWorkflow : invokes
+  RunCsvStatsWorkflow --> Prompts : uses
+  RunCsvStatsWorkflow --> Errors : reports failures
+  RunCsvStatsWorkflow --> CsvStatsPresenter : renders
+  RunCsvStatsWorkflow --> RunCsvStats : calls
+  RunCsvStats --> StatsSession : orchestrates
+  StatsSession o-- StatsSource
+  StatsSession o-- StatsOptions
+  StatsSession o-- OutputDestination
+  RunCsvStats --> CsvStatsScanner
+  RunCsvStats --> CsvStatsFileWriter
+```
+
 ## Project layout
 
 ```text
@@ -487,6 +548,7 @@ lib/csvtool/domain/row_randomization_session/*
 lib/csvtool/domain/cross_csv_dedupe_session/*
 lib/csvtool/domain/csv_parity_session/*
 lib/csvtool/domain/csv_split_session/*
+lib/csvtool/domain/csv_stats_session/*
 lib/csvtool/domain/shared/output_destination.rb
 lib/csvtool/application/use_cases/run_extraction.rb
 lib/csvtool/application/use_cases/run_row_extraction.rb
@@ -494,6 +556,7 @@ lib/csvtool/application/use_cases/run_row_randomization.rb
 lib/csvtool/application/use_cases/run_cross_csv_dedupe.rb
 lib/csvtool/application/use_cases/run_csv_parity.rb
 lib/csvtool/application/use_cases/run_csv_split.rb
+lib/csvtool/application/use_cases/run_csv_stats.rb
 lib/csvtool/infrastructure/csv/*
 lib/csvtool/infrastructure/output/*
 lib/csvtool/interface/cli/menu_loop.rb

data/docs/cli-output-conventions.md

@@ -0,0 +1,49 @@
+# CLI Output Conventions
+
+This document defines the output contract for all CLI workflows.
+
+## 1. Stream contract
+
+- `stdout` is for data output only.
+- `stderr` is for prompts, menu UI, status, and errors.
+- Commands should be pipe-safe: redirecting `stdout` must not capture prompts/errors.
+
+## 2. Format contract
+
+- Supported formats: `text`, `json`, `csv`.
+- `text` is human-readable and may include tables/colors.
+- `json` and `csv` are machine-readable and should remain stable over time.
+- Structured formats must avoid decorative output.
+
+## 3. Color policy
+
+- Supported modes: `auto`, `always`, `never`.
+- `auto` colors only when output target is a TTY.
+- `NO_COLOR` disables color in `auto` mode.
+- `always` overrides `NO_COLOR`.
+- Structured formats (`json`, `csv`) are not colorized.
+
+## 4. Table rendering rules
+
+- Use shared table renderer for summary-style text output.
+- Render within terminal width constraints.
+- Truncate long cells with ellipsis when necessary.
+- Avoid broken/overlapping columns in narrow terminals.
+
+## 5. Shared services usage
+
+All workflows should use shared output services under:
+
+- `lib/csvtool/interface/cli/output/streams.rb`
+- `lib/csvtool/interface/cli/output/formatters/*`
+- `lib/csvtool/interface/cli/output/color_policy.rb`
+- `lib/csvtool/interface/cli/output/colorizer.rb`
+- `lib/csvtool/interface/cli/output/table_renderer.rb`
+
+Prefer these services over ad-hoc formatting in presenters/workflows.
+
+## 6. Testing expectations
+
+- Add focused unit tests for each output service.
+- Add workflow/CLI tests for stream separation and representative formatting behavior.
+- Keep acceptance assertions centered on contract semantics rather than fragile spacing where possible.

data/docs/release-v0.8.0-alpha.md

@@ -0,0 +1,88 @@
+# Release Checklist: v0.8.0-alpha
+
+## 1. Verify environment
+
+```bash
+ruby -v
+bundle -v
+```
+
+Expected:
+- Ruby `3.3.x`
+
+## 2. Install dependencies
+
+```bash
+bundle install
+```
+
+## 3. Run quality checks
+
+```bash
+bundle exec rake test
+```
+
+## 4. Smoke test CLI commands
+
+```bash
+bundle exec csvtool menu
+bundle exec csvtool column test/fixtures/sample_people.csv name
+```
+
+## 5. Smoke test workflows
+
+### CSV stats workflow (new in this release)
+
+Use menu option `7` (`CSV stats summary`) and verify:
+- happy path summary prints rows/columns/headers
+- separator and header mode options work (CSV/TSV/headerless/custom)
+- column completeness output is correct for blanks
+- output destination supports console and file
+- invalid output path returns friendly error
+
+### Existing workflows regression pass
+
+Use menu options `1-6` and verify:
+- column extraction still works
+- row-range extraction still works
+- row randomization still works
+- cross-CSV dedupe still works
+- parity validation still works
+- CSV split still works
+
+## 6. Build and validate gem package
+
+```bash
+gem build csvops.gemspec
+gem install ./csvops-0.8.0.alpha.gem
+csvtool menu
+```
+
+## 7. Commit release prep
+
+```bash
+git add -A
+git commit -m "chore(release): prepare v0.8.0-alpha"
+```
+
+## 8. Tag release
+
+```bash
+git tag -a v0.8.0-alpha -m "v0.8.0-alpha"
+git push origin main --tags
+```
+
+## 9. Publish gem
+
+```bash
+gem push csvops-0.8.0.alpha.gem
+```
+
+## 10. Create GitHub release
+
+Create release `v0.8.0-alpha` with:
+- New `CSV stats summary` workflow
+- Stats-domain architecture (workflow steps, builder, presenter, use case, infrastructure adapters)
+- Console/file output destination support for stats summary artifacts
+- Streaming stats scanner coverage for large files
+- Stats documentation updates in README + architecture guide

data/docs/release-v0.9.0-alpha.md

@@ -0,0 +1,80 @@
+# Release Checklist: v0.9.0-alpha
+
+## 1. Verify environment
+
+```bash
+ruby -v
+bundle -v
+```
+
+Expected:
+- Ruby `3.3.x`
+
+## 2. Install dependencies
+
+```bash
+bundle install
+```
+
+## 3. Run quality checks
+
+```bash
+bundle exec rake test
+```
+
+## 4. Smoke test CLI commands
+
+```bash
+bundle exec csvtool menu
+bundle exec csvtool stats test/fixtures/sample_people.csv --format text
+bundle exec csvtool stats test/fixtures/sample_people.csv --format json
+bundle exec csvtool stats test/fixtures/sample_people.csv --format csv
+```
+
+## 5. Smoke test output conventions across workflows
+
+Verify in menu-driven workflows:
+- prompts/menu/errors are on `stderr`
+- data output is on `stdout`
+
+Verify shared output behavior:
+- formatter consistency (`text|json|csv`)
+- color policy (`auto|always|never`, `NO_COLOR`)
+- width-aware summary tables in stats/parity/split/dedupe
+
+## 6. Build and validate gem package
+
+```bash
+gem build csvops.gemspec
+gem install ./csvops-0.9.0.alpha.gem
+csvtool menu
+```
+
+## 7. Commit release prep
+
+```bash
+git add -A
+git commit -m "chore(release): prepare v0.9.0-alpha"
+```
+
+## 8. Tag release
+
+```bash
+git tag -a v0.9.0-alpha -m "v0.9.0-alpha"
+git push origin main --tags
+```
+
+## 9. Publish gem
+
+```bash
+gem push csvops-0.9.0.alpha.gem
+```
+
+## 10. Create GitHub release
+
+Create release `v0.9.0-alpha` with:
+- Shared output stream services across workflows (`stdout` data, `stderr` UI/errors)
+- Shared formatter services and migrated presenters
+- Shared color policy + colorizer across workflows
+- Shared width-aware table rendering across summary presenters
+- New output conventions documentation (`docs/cli-output-conventions.md`)

data/lib/csvtool/application/use_cases/run_csv_stats.rb

@@ -0,0 +1,64 @@
+# frozen_string_literal: true
+
+require "csv"
+require "csvtool/infrastructure/csv/csv_stats_scanner"
+require "csvtool/infrastructure/output/csv_stats_file_writer"
+
+module Csvtool
+  module Application
+    module UseCases
+      class RunCsvStats
+        Result = Struct.new(:ok, :error, :data, keyword_init: true) do
+          def ok?
+            ok
+          end
+        end
+
+        def initialize(
+          scanner: Infrastructure::CSV::CsvStatsScanner.new,
+          csv_stats_file_writer: Infrastructure::Output::CsvStatsFileWriter.new
+        )
+          @scanner = scanner
+          @csv_stats_file_writer = csv_stats_file_writer
+        end
+
+        def call(session:)
+          path = session.source.path
+          return failure(:file_not_found, path: path) unless File.file?(path)
+
+          stats = @scanner.call(
+            file_path: path,
+            col_sep: session.source.separator,
+            headers_present: session.source.headers_present
+          )
+          if session.output_destination&.file?
+            @csv_stats_file_writer.call(path: session.output_destination.path, data: stats)
+            return success(stats.merge(output_path: session.output_destination.path))
+          end
+          success(stats)
+        rescue CSV::MalformedCSVError
+          failure(:could_not_parse_csv)
+        rescue Errno::EACCES => e
+          if session.output_destination&.file?
+            return failure(:cannot_write_output_file, path: session.output_destination.path, error_class: e.class)
+          end
+          failure(:cannot_read_file, path: path)
+        rescue Errno::ENOENT => e
+          return failure(:cannot_write_output_file, path: session.output_destination.path, error_class: e.class) if session.output_destination&.file?
+
+          failure(:cannot_read_file, path: path)
+        end
+
+        private
+
+        def success(data)
+          Result.new(ok: true, error: nil, data: data)
+        end
+
+        def failure(code, data = {})
+          Result.new(ok: false, error: code, data: data)
+        end
+      end
+    end
+  end
+end