sumd 0.3.45__tar.gz → 0.3.48__tar.gz

{sumd-0.3.45 → sumd-0.3.48}/.gitignore

@@ -1,3 +1,16 @@
+.code2llm_cache/CHANGELOG_1779369542938918821_23281.pkl
+.code2llm_cache/README_1779369515747267462_22533.pkl
+.code2llm_cache/README_1779369542476943815_21853.pkl
+.code2llm_cache/__init___1779369542477957807_771.pkl
+.code2llm_cache/__init___1779369542733974491_65.pkl
+.code2llm_cache/pyproject_1779369542473943784_2355.pkl
+.testql
+!.testql/
+!.testql/autoloop-state.json
+!.testql/schemas/
+!.testql/schemas/llm-decision.schema.json
+!.testql/reports/
+!.testql/reports/llm-decision.json
 # Byte-compiled / optimized / DLL files
 .env.example
 __pycache__/
@@ -106,7 +119,7 @@ ipython_config.py
 #   Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
 #   This is especially recommended for binary packages to ensure reproducibility, and is more
 #   commonly ignored for libraries.
-#uv.lock
+uv.lock
 
 # poetry
 #   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.

{sumd-0.3.45 → sumd-0.3.48}/CHANGELOG.md

@@ -7,6 +7,98 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+- **CQRS ES Architecture**: Implemented Command Query Responsibility Segregation with Event Sourcing
+  - Command and Query buses for separate read/write operations
+  - Event store for persistent audit trail and state reconstruction
+  - Aggregate roots for business logic consistency
+  - SUMD-specific aggregates, commands, queries, and events
+- **Domain Specific Language (DSL)**: Powerful scripting and interactive shell
+  - Complete lexer, parser, and engine for DSL expressions
+  - Arithmetic, logical, comparison, and pipeline operations
+  - Built-in commands for file operations, SUMD operations, search, and utilities
+  - Interactive shell with tab completion and history
+  - Script execution support with error handling
+- **Enhanced MCP Server**: Extended with CQRS ES and DSL capabilities
+  - New MCP tools: execute_command, execute_query, get_events, get_aggregate
+  - New MCP tools: execute_dsl, dsl_shell_info
+  - Full integration with CQRS ES architecture and DSL engine
+- **New CLI Commands**:
+  - `sumd dsl` - Interactive DSL shell and command execution
+  - `sumd cqrs` - Execute CQRS commands on SUMD aggregates
+- **Comprehensive Testing**: Full test suite for CQRS ES and DSL functionality
+  - Event store, command bus, query bus, and aggregate tests
+  - DSL parser, engine, and shell tests
+  - MCP server integration tests
+- **Documentation**: Updated README with CQRS ES and DSL architecture details
+
+### Fixed
+- **File filtering**: sumd now properly respects .gitignore and .sumdignore files when analyzing source files
+- Added support for standard gitignore patterns including wildcards, directory patterns, and negation
+- Improved file collection performance by skipping ignored files and directories
+
+## [0.3.48] - 2026-05-21
+
+### Docs
+- Update README.md
+- Update SUMD.md
+- Update SUMR.md
+- Update docs/README.md
+- Update project/README.md
+- Update project/context.md
+
+### Other
+- Update .code2llm_cache/CHANGELOG_1779369542938918821_23281.pkl
+- Update .code2llm_cache/README_1779369515747267462_22533.pkl
+- Update .code2llm_cache/README_1779369542476943815_21853.pkl
+- Update .code2llm_cache/__init___1779369542477957807_771.pkl
+- Update .code2llm_cache/__init___1779369542733974491_65.pkl
+- Update .code2llm_cache/pyproject_1779369542473943784_2355.pkl
+- Update app.doql.less
+- Update project.sh
+- Update project/analysis.toon.yaml
+- Update project/calls.mmd
+- ... and 19 more files
+
+## [0.3.47] - 2026-05-21
+
+### Docs
+- Update CHANGELOG.md
+- Update README.md
+- Update SPEC.md
+- Update SUMD.md
+- Update TODO.md
+- Update docs/USAGE.md
+- Update project/README.md
+- Update project/context.md
+- Update sumd_logic_validator/README.md
+
+### Test
+- Update testql-scenarios/sumd-cli.testql.toon.yaml
+- Update tests/test_architectural_logic.py
+
+### Other
+- Update TODO.txt
+- Update print_errors.py
+- Update project.sh
+- Update project/analysis.toon.yaml
+- Update project/logic.logicml
+- Update project/logic.pl
+- Update project/map.toon.yaml
+- Update sumd/__main__.py
+- Update sumd/cli.py
+- Update sumd/extractor.py
+- ... and 19 more files
+
+## [0.3.46] - 2026-05-02
+
+### Docs
+- Update README.md
+
+### Other
+- Update .sumdignore
+- Update sumd/extractor.py
+
 ## [0.3.45] - 2026-04-25
 
 ### Docs
@@ -561,6 +653,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - 88 unit tests across `test_pipeline.py`, `test_sections.py` (30), `test_extractor.py` (37)
 - `--cov-branch` added to pytest `addopts` for accurate branch+statement coverage reporting
 - pyqual gates: `coverage_min: 35`, `vallm_pass_min: 60`
+- **PyPI Release**: Published to PyPI: https://pypi.org/project/sumd/0.2.0rc1/
 
 ### Fixed
 - Single `.venv`: removed dual-venv split (`.venv` vs `venv`), all tasks use `{{.VENV_PY}} = .venv/bin/python`
@@ -591,6 +684,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 - `generate_sumd_content` refactored to 4-line shim delegating to `RenderPipeline` (fan-out: 34→2)
+- Extracted core logic into separate modular files: `sumd/extractor.py`, `sumd/renderer.py`, and `sumd/toon_parser.py`
+- Replaced `sumd/generator.py` with an 18-line shim re-exporting `extractor.py` and `renderer.py`
 - `_render_code_analysis` accepts `skip_files` param to avoid duplicating call graph data
 - Parser `_VALID_MARKPACT_KINDS` whitelist extended with 6 new semantic kinds
 

{sumd-0.3.45 → sumd-0.3.48}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sumd
-Version: 0.3.45
+Version: 0.3.48
 Summary: SUMD - Structured Unified Markdown Descriptor for AI-aware project documentation
 Project-URL: Homepage, https://github.com/tom/statement
 Project-URL: Repository, https://github.com/tom/statement
@@ -18,6 +18,7 @@ Requires-Python: >=3.10
 Requires-Dist: click>=8.3.3
 Requires-Dist: costs>=0.1.50
 Requires-Dist: goal>=2.1.190
+Requires-Dist: mcp>=1.27.0
 Requires-Dist: pfix>=0.1.72
 Requires-Dist: pyyaml>=6.0.3
 Requires-Dist: toml>=0.10.2
@@ -28,6 +29,7 @@ Requires-Dist: goal>=2.1.190; extra == 'dev'
 Requires-Dist: mcp>=1.27.0; extra == 'dev'
 Requires-Dist: pfix>=0.1.72; extra == 'dev'
 Requires-Dist: pyqual>=0.1.143; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
 Requires-Dist: pytest-cov>=7.1.0; extra == 'dev'
 Requires-Dist: pytest>=9.0.3; extra == 'dev'
 Requires-Dist: ruff>=0.15.11; extra == 'dev'
@@ -38,21 +40,18 @@ Description-Content-Type: text/markdown
 
 # SUMD
 
-
 ## AI Cost Tracking
 
-![PyPI](https://img.shields.io/badge/pypi-costs-blue) ![Version](https://img.shields.io/badge/version-0.3.45-blue) ![Python](https://img.shields.io/badge/python-3.9+-blue) ![License](https://img.shields.io/badge/license-Apache--2.0-green)
-![AI Cost](https://img.shields.io/badge/AI%20Cost-$7.50-orange) ![Human Time](https://img.shields.io/badge/Human%20Time-27.6h-blue) ![Model](https://img.shields.io/badge/Model-openrouter%2Fqwen%2Fqwen3--coder--next-lightgrey)
+![PyPI](https://img.shields.io/badge/pypi-costs-blue) ![Version](https://img.shields.io/badge/version-0.3.48-blue) ![Python](https://img.shields.io/badge/python-3.9+-blue) ![License](https://img.shields.io/badge/license-Apache--2.0-green)
+![AI Cost](https://img.shields.io/badge/AI%20Cost-$3.41-orange) ![Human Time](https://img.shields.io/badge/Human%20Time-34.0h-blue) ![Model](https://img.shields.io/badge/Model-openrouter%2Fqwen%2Fqwen3--coder--next-lightgrey)
 
-- 🤖 **LLM usage:** $7.5000 (75 commits)
-- 👤 **Human dev:** ~$2765 (27.6h @ $100/h, 30min dedup)
+- 🤖 **LLM usage:** $3.4095 (87 commits)
+- 👤 **Human dev:** ~$3400 (34.0h @ $100/h, 30min dedup)
 
-Generated on 2026-04-25 using [openrouter/qwen/qwen3-coder-next](https://openrouter.ai/qwen/qwen3-coder-next)
+Generated on 2026-05-21 using [openrouter/qwen/qwen3-coder-next](https://openrouter.ai/qwen/qwen3-coder-next)
 
 ---
 
-![Version](https://img.shields.io/badge/version-0.3.45-blue) ![Python](https://img.shields.io/badge/python-3.10+-blue) ![License](https://img.shields.io/badge/license-Apache--2.0-green)
-
 **SUMD** (Structured Unified Markdown Descriptor) is a semantic project descriptor format in Markdown.  
 It defines intent, structure, execution entry points, and the mental model of a system for both humans and LLMs.
 
@@ -83,7 +82,7 @@ Think of it as a machine-readable README: a file an AI agent can parse, reason o
 
 ```bash
 pip install sumd                  # stable
-pip install sumd==0.2.0rc1        # latest release candidate
+pip install sumd==0.3.45           # specific version
 ```
 
 ## Developer Workflow
@@ -102,8 +101,6 @@ task pyqual
 task publish
 ```
 
-
-
 ## Usage
 
 ### CLI Commands
@@ -153,6 +150,17 @@ sumd scaffold ./my-project --force          # overwrite existing files
 sumd analyze ./my-project                    # run all tools
 sumd analyze ./my-project --tools code2llm   # only code2llm
 sumd analyze ./my-project --force            # reinstall tools
+
+# DSL (Domain Specific Language) operations
+sumd dsl                                     # start interactive DSL shell
+sumd dsl -c "scan('.')"                      # execute single DSL command
+sumd dsl -s script.dsl                       # execute DSL script file
+sumd dsl -d /path/to/project                 # set working directory
+
+# CQRS ES (Command Query) operations
+sumd cqrs create_sumd_document ./SUMD.md --data '{"project_name":"MyProject"}'
+sumd cqrs add_section ./SUMD.md --data '{"section_name":"Architecture","content":"..."}'
+sumd cqrs validate_sumd_document ./SUMD.md
 ```
 
 ### Section Profiles
@@ -186,6 +194,174 @@ if not result["ok"]:
         print(issue)
 ```
 
+## File Filtering
+
+SUMD respects standard ignore files to exclude unwanted files and directories from analysis:
+
+### Supported Ignore Files
+
+- **`.gitignore`** - Standard Git ignore patterns (automatically detected)
+- **`.sumdignore`** - SUMD-specific ignore patterns (overrides .gitignore)
+
+### Pattern Syntax
+
+Supports full gitignore pattern syntax:
+
+```gitignore
+# File patterns
+*.log
+*.tmp
+coverage.xml
+
+# Directory patterns  
+temp/
+build/
+__pycache__/
+
+# Negation (include despite other patterns)
+!important.log
+!src/temp/
+```
+
+### Behavior
+
+- Patterns are read from both `.gitignore` and `.sumdignore` files
+- `.sumdignore` patterns take precedence over `.gitignore` patterns
+- Improves performance by skipping ignored files during analysis
+- Works with all `sumd scan`, `sumd map`, and `sumd analyze` commands
+
+## CQRS ES Architecture
+
+SUMD now implements **Command Query Responsibility Segregation (CQRS)** with **Event Sourcing (ES)** for robust state management and audit trails:
+
+### Architecture Components
+
+- **Commands**: Write operations that modify system state
+- **Queries**: Read operations that retrieve system state
+- **Events**: Immutable records of state changes
+- **Aggregates**: Consistency boundaries for business logic
+- **Event Store**: Persistent storage for event history
+
+### Benefits
+
+- **Audit Trail**: Complete history of all changes
+- **Temporal Queries**: Reconstruct state at any point in time
+- **Scalability**: Separate read/write models
+- **Resilience**: Event replay for error recovery
+
+### CLI Integration
+
+```bash
+# Execute CQRS commands
+sumd cqrs create_sumd_document ./SUMD.md --data '{"project_name":"MyProject"}'
+sumd cqrs add_section ./SUMD.md --data '{"section_name":"Architecture","content":"..."}'
+
+# Query system state
+sumd cqrs get_aggregate ./SUMD.md
+sumd cqrs get_events ./SUMD.md
+```
+
+### MCP Integration
+
+The MCP server exposes CQRS ES tools:
+
+- `execute_command` - Execute write commands
+- `execute_query` - Execute read queries  
+- `get_events` - Retrieve event history
+- `get_aggregate` - Get current aggregate state
+
+## DSL (Domain Specific Language)
+
+SUMD provides a powerful DSL for interactive operations and scripting:
+
+### DSL Shell
+
+```bash
+# Start interactive shell
+sumd dsl
+
+# Execute single command
+sumd dsl -c "scan('.') | validate('.')"
+
+# Execute script file
+sumd dsl -s script.dsl
+```
+
+### DSL Features
+
+- **Arithmetic**: `1 + 2 * 3`
+- **Logic**: `x and y or not z`
+- **Comparison**: `x == 42`, `name contains "test"`
+- **Variables**: `x = 42`, `result = scan('.')`
+- **Functions**: `len("hello")`, `exists("file.txt")`
+- **Pipelines**: `scan('.') | validate('.') | export("json")`
+- **File Operations**: `cat("file.txt")`, `ls("*.md")`, `edit("file.txt", "content")`
+
+### Built-in Commands
+
+#### File Operations
+- `cat` - Display file contents
+- `ls` - List directory contents  
+- `edit` - Edit file content
+- `mkdir` - Create directory
+- `rm` - Remove file/directory
+
+#### SUMD Operations
+- `sumd_scan` - Scan and generate SUMD
+- `sumd_map` - Generate project map
+- `sumd_validate` - Validate SUMD document
+- `sumd_info` - Show document info
+
+#### Search Operations
+- `find` - Find files matching pattern
+- `grep` - Search text in files
+
+#### Utility Operations
+- `echo` - Display message
+- `pwd` - Print working directory
+- `cd` - Change directory
+- `help` - Show help
+
+#### Variables
+- `set` - Set variable
+- `get` - Get variable value
+- `unset` - Remove variable
+- `vars` - List all variables
+
+### DSL Examples
+
+```bash
+# Basic arithmetic
+result = 1 + 2 * 3
+
+# File operations
+if exists("SUMD.md"):
+    content = cat("SUMD.md")
+    print(len(content))
+
+# SUMD workflow
+scan(".") | validate(".") | export("json")
+
+# Variable usage
+project = "my-project"
+files = ls("*.py")
+print(f"Found {len(files)} Python files in {project}")
+
+# Conditional logic
+if exists("pyproject.toml"):
+    scan(".")
+    validate("SUMD.md")
+else:
+    print("No Python project found")
+```
+
+### MCP DSL Integration
+
+The MCP server provides DSL tools:
+
+- `execute_dsl` - Execute DSL expressions
+- `dsl_shell_info` - Get shell capabilities
+
 ## What is Embedded in SUMD.md?
 
 SUMD auto-embeds the following sources from a project (when present):
@@ -225,31 +401,38 @@ SUMD auto-embeds the following sources from a project (when present):
 
 ## Ecosystem Architecture
 
-SUMD is part of a three-layer system:
+SUMD is part of a four-layer system:
 
 ```
 ┌─────────────────────────────────────────────────────────────┐
-│                     SUMD (opis)                              │
-│              Structured Unified Markdown Descriptor          │
-│         Project description, intent, architecture            │
+│                     SUMD (opis)                             │
+│              Structured Unified Markdown Descriptor         │
+│         Project description, intent, architecture           │
 └─────────────────────────────────────────────────────────────┘
                               ↓
 ┌─────────────────────────────────────────────────────────────┐
-│                    DOQL (wykonanie)                          │
-│              Declarative Object Query Language               │
-│              Data manipulation and execution                 │
+│                    DOQL (wykonanie)                         │
+│              Declarative Object Query Language              │
+│              Data manipulation and execution                │
 └─────────────────────────────────────────────────────────────┘
                               ↓
 ┌─────────────────────────────────────────────────────────────┐
-│                  Taskfile (runtime)                          │
-│              Task runner and workflow execution              │
+│                  Taskfile (runtime)                         │
+│              Task runner and workflow execution             │
 │              Automation and orchestration                   │
+└─────────────────────────────────────────────────────────────┘
+                              ↓
+┌─────────────────────────────────────────────────────────────┐
+│                   testql (weryfikacja)                      │
+│              Test generation and execution                  │
+│              API, GUI, hardware, shell, WebSocket tests     │
 └─────────────────────────────────────────────────────────────┘
 ```
 
 - **SUMD → opis (description)**: Defines what the system is and how it should work
 - **DOQL → wykonanie (execution)**: Provides the language to manipulate and execute operations
 - **Taskfile → runtime**: Manages the actual execution of workflows and tasks
+- **testql → weryfikacja (verification)**: Generates and executes tests across multiple domains
 
 ## DOQL Integration
 
@@ -294,6 +477,114 @@ Flow:
 
 This gives you a single command that keeps both documentation and generated code in sync, without unnecessary rebuilds.
 
+## testql Integration
+
+testql is the testing framework in the SUMD ecosystem, providing automated test generation and execution for APIs, GUI, hardware, shell commands, and WebSockets.
+
+### testql Features
+
+#### Test Generation
+
+testql automatically generates test scenarios from various sources:
+
+- **API Tests**: From OpenAPI/Swagger specifications, FastAPI/Flask/Django routes, Express.js endpoints
+- **GUI Tests**: From Playwright/Selenium configurations
+- **Hardware Tests**: From hardware peripheral configurations
+- **Shell Tests**: From CLI command definitions
+- **WebSocket Tests**: From WebSocket endpoint definitions
+- **Pytest Conversion**: Converts existing Python pytest tests to testql format
+- **OQL/CQL Scenarios**: Converts OQL/CQL scenario files to testql format
+
+#### Test Execution
+
+testql executes test scenarios written in `.testql.toon.yaml` format with support for:
+
+- **API Commands**: `API[method, endpoint, expected_status]` with retry logic
+- **Assertions**: `ASSERT_STATUS`, `ASSERT_OK`, `ASSERT_CONTAINS`, `ASSERT_JSON`, `ASSERT_HEADERS`, `ASSERT_SCHEMA`, `ASSERT_COOKIES`
+- **GUI Commands**: `GUI_START`, `GUI_CLICK`, `GUI_INPUT`, `GUI_ASSERT_VISIBLE`, `GUI_ASSERT_TEXT`, `GUI_CAPTURE`, `GUI_STOP`
+- **Hardware Commands**: `ENCODER_ON`, `ENCODER_OFF`, `ENCODER_SCROLL`, `ENCODER_CLICK`, `ENCODER_FOCUS`, `ENCODER_STATUS`
+- **Shell Commands**: `SHELL`, `EXEC`, `RUN` with exit code and output assertions
+- **WebSocket Commands**: `WS_CONNECT`, `WS_SEND`, `WS_RECEIVE`, `WS_ASSERT_MSG`, `WS_CLOSE`
+- **Flow Control**: `WAIT`, `WAIT_FOR`, `LOG`, `PRINT`, `INCLUDE`
+
+#### Endpoint Detection
+
+testql includes automatic endpoint detection from:
+
+- **Docker Compose**: Service port mappings and configurations
+- **Kubernetes Configs**: Service and deployment configurations
+- **.env Files**: Environment variables with URL, HOST, PORT patterns
+- **config.py Files**: Python configuration files with host/port/url assignments
+- **Framework Detectors**: FastAPI, Flask, Django, Express.js route discovery
+- **Specification Files**: OpenAPI/Swagger, GraphQL schemas
+
+#### Configuration
+
+testql scenarios support configuration via `CONFIG` block:
+
+```yaml
+CONFIG[4]{key, value}:
+  base_url, ${api_url:-http://localhost:8100}
+  timeout_ms, 10000
+  retry_count, 3
+  retry_backoff_ms, 100
+```
+
+#### Reporting
+
+testql generates HTML reports from test results:
+
+```bash
+# Generate report from pytest JSON output
+pytest --json-report --json-report-file=test-results.json
+testql report test-results.json -o report.html
+
+# Generate example report
+testql report --example -o report.html
+```
+
+#### Integration with SUMD
+
+SUMD automatically embeds testql scenarios from `testql-scenarios/` directory into SUMD.md:
+
+```yaml markpact:testql path=testql-scenarios/generated-cli-tests.testql.toon.yaml
+# SCENARIO: CLI Command Tests
+# TYPE: cli
+# GENERATED: true
+
+CONFIG[2]{key, value}:
+  cli_command, python -msumd
+  timeout_ms, 10000
+
+LOG[3]{message}:
+  "Test CLI help command"
+  "Test CLI version command"
+  "Test CLI main workflow"
+```
+
+#### Workflow Integration
+
+testql is integrated into SUMD workflows:
+
+```bash
+# Run pytest with testql report generation
+task test:report
+
+# Generate testql scenario scaffolds from OpenAPI
+sumd scaffold ./my-project --type smoke
+sumd scaffold ./my-project --type crud
+```
+
+## Dokumentacja
+
+- **[CHANGELOG.md](./CHANGELOG.md)** — Historia zmian i wydania
+- **[TODO.md](./TODO.md)** — Aktualne zadania i planowane funkcje
+- **[docs/USAGE.md](./docs/USAGE.md)** — Szczegółowa dokumentacja użycia
+- **[docs/TESTQL_AUTOLOOP_ONBOARDING.md](./docs/TESTQL_AUTOLOOP_ONBOARDING.md)** — Onboarding dla TestQL + MCP + aider loops
+- **[SPEC.md](./SPEC.md)** — Specyfikacja formatu SUMD
+- **[SUMD.md](./SUMD.md)** — Dokumentacja projektu sumd (wygenerowana)
+- **[SUMR.md](./SUMR.md)** — Raport refaktoryzacji (wygenerowany)
+
 ## License
 
 Licensed under Apache-2.0.