markdown_exec 3.5.1 → 3.5.2

data/semantic-tokens.md

@@ -0,0 +1,132 @@
+# Semantic Tokens
+
+**STDD Methodology Version**: 1.0.0
+
+## Overview
+This document defines semantic tokens (keywords, identifiers, and concepts) that tie together all documentation, code, and tests. These tokens provide a consistent vocabulary for discussing the system.
+
+## Token Format
+
+```
+[TYPE:IDENTIFIER]
+```
+
+## Token Types
+
+- `[REQ:*]` - Requirements (functional/non-functional) - **The source of intent**
+- `[ARCH:*]` - Architecture decisions - **High-level design choices that preserve intent**
+- `[IMPL:*]` - Implementation decisions - **Low-level choices that preserve intent**
+- `[TEST:*]` - Test specifications - **Validation of intent**
+
+## Token Naming Convention
+
+- Use UPPER_SNAKE_CASE for identifiers
+- Be descriptive but concise
+- Example: `[REQ:DUPLICATE_PREVENTION]` not `[REQ:DP]`
+
+## Cross-Reference Format
+
+When referencing other tokens:
+
+```markdown
+[IMPL:EXAMPLE] Description [ARCH:DESIGN] [REQ:REQUIREMENT]
+```
+
+## Requirements Tokens
+
+### Core Functionality
+- `[REQ:SHEBANG_HIDING]` - CLI option to hide shebang lines in document output (default enabled)
+- `[REQ:EXAMPLE_FEATURE]` - Example feature requirement
+- Add your requirements tokens here
+
+### Non-Functional Requirements
+- `[REQ:PERFORMANCE]` - Performance requirements
+- `[REQ:RELIABILITY]` - Reliability requirements
+- `[REQ:MAINTAINABILITY]` - Maintainability requirements
+- `[REQ:USABILITY]` - Usability requirements
+
+## Architecture Tokens
+
+### Core Architecture Decisions
+- `[ARCH:LANGUAGE_SELECTION]` - Language and runtime selection
+- `[ARCH:PROJECT_STRUCTURE]` - Project structure decision
+- `[ARCH:SHEBANG_EXTRACTION]` - Extract shebang lines during cached nested read [REQ:SHEBANG_HIDING]
+- `[ARCH:CLI_OPTION_DESIGN]` - CLI option design for shebang hiding [REQ:SHEBANG_HIDING]
+- `[ARCH:EXAMPLE_DECISION]` - Example architecture decision [REQ:EXAMPLE_FEATURE]
+- Add your architecture tokens here
+
+## Implementation Tokens
+
+### Core Implementation Decisions
+- `[IMPL:CONFIG_STRUCT]` - Configuration structure implementation [ARCH:CONFIG_STRUCTURE] [REQ:CONFIGURATION]
+- `[IMPL:SHEBANG_DETECTION]` - Shebang line detection logic [ARCH:SHEBANG_EXTRACTION] [REQ:SHEBANG_HIDING]
+- `[IMPL:SHEBANG_FILTERING]` - Filter shebang lines from processed output [ARCH:SHEBANG_EXTRACTION] [REQ:SHEBANG_HIDING]
+- `[IMPL:CLI_OPTION_IMPLEMENTATION]` - CLI option implementation in menu system [ARCH:CLI_OPTION_DESIGN] [REQ:SHEBANG_HIDING]
+- `[IMPL:EXAMPLE_IMPLEMENTATION]` - Example implementation [ARCH:EXAMPLE_DECISION] [REQ:EXAMPLE_FEATURE]
+- Add your implementation tokens here
+
+## Test Tokens
+
+### Test Specifications
+- `[TEST:SHELL_COMMAND]` - Comprehensive test suite for ShellCommand class covering execution, memoization, outputs, metadata, and edge cases
+- Add your test tokens here
+
+## Token Relationships
+
+### Hierarchical Relationships
+- `[REQ:PARENT_FEATURE]` contains `[REQ:SUB_FEATURE_1]`, `[REQ:SUB_FEATURE_2]`
+- `[ARCH:FEATURE]` includes `[ARCH:COMPONENT_1]`, `[ARCH:COMPONENT_2]`
+
+### Flow Relationships
+- `[REQ:FEATURE]` → `[ARCH:DESIGN]` → `[IMPL:IMPLEMENTATION]` → Code + Tests
+
+### Dependency Relationships
+- `[IMPL:FEATURE]` depends on `[ARCH:DESIGN]` and `[REQ:FEATURE]`
+- `[ARCH:DESIGN]` depends on `[REQ:FEATURE]`
+
+## Usage Examples
+
+### In Code Comments
+```[your-language]
+// [REQ:EXAMPLE_FEATURE] Implementation of example feature
+// [IMPL:EXAMPLE_IMPLEMENTATION] [ARCH:EXAMPLE_DECISION] [REQ:EXAMPLE_FEATURE]
+function exampleFunction() {
+    // ...
+}
+```
+
+### In Tests
+```[your-language]
+// Test validates [REQ:EXAMPLE_FEATURE] is met
+func TestExampleFeature_REQ_EXAMPLE_FEATURE(t *testing.T) {
+    // ...
+}
+```
+
+### In Documentation
+```markdown
+The feature uses [ARCH:EXAMPLE_DECISION] to fulfill [REQ:EXAMPLE_FEATURE].
+Implementation details are documented in [IMPL:EXAMPLE_IMPLEMENTATION].
+```
+
+## Token Index by Category
+
+### Requirements
+- `[REQ:SHEBANG_HIDING]` - CLI option to hide shebang lines in document output
+- Add your requirement tokens here
+
+### Architecture
+- `[ARCH:SHEBANG_EXTRACTION]` - Extract shebang lines during cached nested read
+- `[ARCH:CLI_OPTION_DESIGN]` - CLI option design for shebang hiding
+- Add your architecture tokens here
+
+### Implementation
+- `[IMPL:SHEBANG_DETECTION]` - Shebang line detection logic
+- `[IMPL:SHEBANG_FILTERING]` - Filter shebang lines from processed output
+- `[IMPL:CLI_OPTION_IMPLEMENTATION]` - CLI option implementation in menu system
+- Add your implementation tokens here
+
+### Tests
+- `[TEST:SHELL_COMMAND]` - Comprehensive test suite for ExecutedShellCommand class
+- Add your test tokens here
+

data/tasks.md

@@ -0,0 +1,69 @@
+# Tasks and Incomplete Subtasks
+
+**STDD Methodology Version**: 1.0.0
+
+## Overview
+This document tracks all tasks and subtasks for implementing your project. Tasks are organized by priority and implementation phase.
+
+## Priority Levels
+
+- **P0 (Critical)**: Must have - Core functionality, blocks other work
+- **P1 (Important)**: Should have - Enhanced functionality, better error handling
+- **P2 (Nice-to-Have)**: Could have - UI/UX improvements, convenience features
+- **P3 (Future)**: Won't have now - Deferred features, experimental ideas
+
+## Task Format
+
+```markdown
+## P0: Task Name [REQ:IDENTIFIER] [ARCH:IDENTIFIER] [IMPL:IDENTIFIER]
+
+**Status**: 🟡 In Progress | ✅ Complete | ⏸️ Blocked | ⏳ Pending
+
+**Description**: Brief description of what this task accomplishes.
+
+**Dependencies**: List of other tasks/tokens this depends on.
+
+**Subtasks**:
+- [ ] Subtask 1 [REQ:X] [IMPL:Y]
+- [ ] Subtask 2 [REQ:X] [IMPL:Z]
+- [ ] Subtask 3 [TEST:X]
+
+**Completion Criteria**:
+- [ ] All subtasks complete
+- [ ] Code implements requirement
+- [ ] Tests pass with semantic token references
+- [ ] Documentation updated
+
+**Priority Rationale**: Why this is P0/P1/P2/P3
+```
+
+## P1: Implement CLI Option for Hiding Shebang Lines [REQ:SHEBANG_HIDING] [ARCH:CLI_OPTION_DESIGN] [ARCH:SHEBANG_EXTRACTION] [IMPL:CLI_OPTION_IMPLEMENTATION] [IMPL:SHEBANG_DETECTION] [IMPL:SHEBANG_FILTERING]
+
+**Status**: ✅ Complete
+
+**Description**: Implement CLI option `hide_shebang` (default: true) that extracts and hides shebang lines from document output during cached nested read process.
+
+**Dependencies**: None
+
+**Subtasks**:
+- [x] Add `hide_shebang` option to `lib/menu.src.yml` [IMPL:CLI_OPTION_IMPLEMENTATION] [ARCH:CLI_OPTION_DESIGN] [REQ:SHEBANG_HIDING]
+- [x] Implement shebang detection logic in `CachedNestedFileReader` [IMPL:SHEBANG_DETECTION] [ARCH:SHEBANG_EXTRACTION] [REQ:SHEBANG_HIDING]
+- [x] Implement shebang filtering logic in `CachedNestedFileReader#readlines` [IMPL:SHEBANG_FILTERING] [ARCH:SHEBANG_EXTRACTION] [REQ:SHEBANG_HIDING]
+- [x] Pass `hide_shebang` option from `MarkParse` to `CachedNestedFileReader` [IMPL:CLI_OPTION_IMPLEMENTATION] [ARCH:SHEBANG_EXTRACTION] [REQ:SHEBANG_HIDING]
+- [x] Write unit tests for shebang detection [REQ:SHEBANG_HIDING]
+- [x] Write unit tests for shebang filtering [REQ:SHEBANG_HIDING]
+- [x] Write integration tests for CLI option behavior [REQ:SHEBANG_HIDING]
+- [x] Write integration tests for nested imports with shebang lines [REQ:SHEBANG_HIDING]
+- [x] Update CLI documentation to include new option [REQ:SHEBANG_HIDING]
+
+**Completion Criteria**:
+- [x] All subtasks complete
+- [x] Code implements [REQ:SHEBANG_HIDING]
+- [x] Tests pass with semantic token references
+- [x] Documentation updated
+- [x] Option defaults to enabled (true)
+- [x] Option works with nested imports
+- [x] Option accessible via CLI flag and environment variable
+
+**Priority Rationale**: P1 (Important) - Enhances document processing functionality and improves user experience by hiding execution directives from document output. Not critical for core functionality but important for professional document presentation.
+

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: markdown_exec
 version: !ruby/object:Gem::Version
-  version: 3.5.1
+  version: 3.5.2
 platform: ruby
 authors:
 - Fareed Stevenson
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2025-11-14 00:00:00.000000000 Z
+date: 2025-12-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: clipboard
@@ -91,8 +91,11 @@ executables:
 extensions: []
 extra_rdoc_files: []
 files:
+- ".ai-agent-instructions"
+- ".cursorrules"
 - ".pryrc"
 - ".reek"
+- ".rubocop.wide.yml"
 - ".rubocop.yml"
 - CHANGELOG.md
 - CODE_OF_CONDUCT.md
@@ -102,6 +105,8 @@ files:
 - LICENSE.txt
 - README.md
 - Rakefile
+- ai-principles.md
+- architecture-decisions.md
 - assets/approve_code.png
 - assets/example_blocks.png
 - assets/output_of_execution.png
@@ -177,8 +182,14 @@ files:
 - bin/setup
 - bin/tab_completion.sh
 - bin/tab_completion.sh.erb
+- conversation-template.md
 - demo/trap.demo1.gif
 - demo/trap.demo1.mp4
+- docs/block-execution-modes.md
+- docs/block-filtering.md
+- docs/block-naming-patterns.md
+- docs/block-scanning-patterns.md
+- docs/cli-reference.md
 - docs/dev/bats-document-configuration.md
 - docs/dev/block-hide.md
 - docs/dev/block-type-bash.md
@@ -258,8 +269,10 @@ files:
 - docs/dev/option-expansion.md
 - docs/dev/options-collapse.md
 - docs/dev/pass-through-arguments.md
+- docs/dev/print_bytes.md
 - docs/dev/requiring-blocks.md
 - docs/dev/screen-width.md
+- docs/dev/shebang.md
 - docs/dev/specs-import.md
 - docs/dev/specs.md
 - docs/dev/table-column-truncate.md
@@ -269,9 +282,11 @@ files:
 - docs/dev/variable-expansion-multiline.md
 - docs/dev/variable-expansion.md
 - docs/docker-testing.md
+- docs/execution-control.md
+- docs/getting-started.md
+- docs/import-options.md
 - docs/tab-completion.md
-- docs/ux-blocks-examples.md
-- docs/ux-blocks-init-act.md
+- docs/ux-blocks.md
 - examples/bash-blocks.md
 - examples/block-names.md
 - examples/block-types.md
@@ -323,6 +338,7 @@ files:
 - examples/variable-expansion.md
 - examples/vars-blocks.md
 - examples/wrapped-blocks.md
+- implementation-decisions.md
 - lib/ansi_formatter.rb
 - lib/ansi_string.rb
 - lib/argument_processor.rb
@@ -345,6 +361,7 @@ files:
 - lib/error_reporting.rb
 - lib/evaluate_shell_expressions.rb
 - lib/exceptions.rb
+- lib/executed_shell_command.rb
 - lib/fcb.rb
 - lib/filter.rb
 - lib/find_files.rb
@@ -381,8 +398,13 @@ files:
 - lib/table_extractor.rb
 - lib/tap.rb
 - lib/text_analyzer.rb
+- lib/transformed_shell_command.rb
 - lib/value_or_exception.rb
+- lib/wl.rb
 - lib/ww.rb
+- requirements.md
+- semantic-tokens.md
+- tasks.md
 homepage: https://rubygems.org/gems/markdown_exec
 licenses:
 - MIT

data/docs/ux-blocks-examples.md

@@ -1,120 +0,0 @@
-# UX Block Examples
-
-This file contains a collection of unique UX block examples from the documentation.
-
-## Basic Examples
-
-### Simple Variable Display and Edit
-```ux
-init: Guest
-name: USER_NAME
-prompt: Enter your name
-```
-
-### Command Output Initialization
-```ux
-name: CURRENT_DIR
-init: :exec
-exec: basename $(pwd)
-transform: :chomp
-```
-
-### Echo-based Initialization
-```ux
-name: SHELL_VERSION
-init: :echo
-echo: $SHELL
-```
-
-### Selection from Allowed Values
-```ux
-name: ENVIRONMENT
-allow:
-  - development
-  - staging
-  - production
-prompt: Select environment
-```
-
-## Validation Examples
-
-### Email Validation
-```ux
-name: USER_EMAIL
-prompt: Enter email address
-validate: '(?<local>[^@]+)@(?<domain>[^@]+)'
-transform: '%{local}@%{domain}'
-```
-
-### Version Number Validation
-```ux
-name: VERSION
-prompt: Enter version number
-validate: '(?<major>\d+)\.(?<minor>\d+)\.(?<patch>\d+)'
-transform: '%{major}.%{minor}.%{patch}'
-```
-
-## Complex Examples
-
-### Git Branch Selection with Validation
-```ux
-name: BRANCH_NAME
-init: ":exec"
-exec: "git branch --format='%(refname:short)'"
-validate: '^(?<type>feature|bugfix|hotfix)/(?<ticket>[A-Z]+-\d+)-(?<desc>.+)$'
-transform: "${type}/${ticket}-${desc}"
-prompt: "Select or enter branch name"
-```
-
-### Environment Configuration with Dependencies
-```ux
-name: DATABASE_URL
-require:
-  - ENVIRONMENT
-  - DB_HOST
-  - DB_PORT
-format: "postgresql://${DB_USER}:${DB_PASS}@${DB_HOST}:${DB_PORT}/${DB_NAME}"
-```
-
-### Multi-step Configuration
-```ux
-name: DEPLOY_CONFIG
-require:
-  - ENVIRONMENT
-  - VERSION
-init: ":echo"
-echo: "Deploying ${VERSION} to ${ENVIRONMENT}"
-act: ":exec"
-exec: "deploy.sh ${ENVIRONMENT} ${VERSION}"
-```
-
-### Conditional Initialization
-```ux
-name: API_KEY
-init: ":allow"
-allow:
-  - ${PROD_API_KEY}
-  - ${STAGING_API_KEY}
-  - ${DEV_API_KEY}
-require:
-  - ENVIRONMENT
-```
-
-### Formatted Output with Validation
-```ux
-name: PHONE_NUMBER
-prompt: "Enter phone number"
-validate: '(?<country>\d{1,3})(?<area>\d{3})(?<number>\d{7})'
-transform: "+${country} (${area}) ${number}"
-format: "Phone: ${PHONE_NUMBER}"
-```
-
-### Command Output with Transformation
-```ux
-name: GIT_STATUS
-init: ":exec"
-exec: "git status --porcelain"
-validate: '(?<status>[AMDR])\s+(?<file>.+)'
-transform: "${status}: ${file}"
-format: "Changes: ${GIT_STATUS}"
-``` 

data/docs/ux-blocks-init-act.md

@@ -1,100 +0,0 @@
-# UX Block Init and Act Keys
-
-The `init` and `act` keys determine which other key is read for processing during initialization and activation respectively.
-
-## Algorithm
-
-1. **Init Phase** (when document is loaded) per FCB.init_source:
-   - If `init` is `false`: No initialization occurs
-   - If `init` is a string: That string becomes the initial value
-   - If `init` is `:allow`: First value from `allow` list is used
-   - If `init` is `:echo`: Value from `echo` key is evaluated and returned
-   - If `init` is `:exec`: Command from `exec` key is executed and stdout is returned
-   - If `init` is not present: Defaults to first available in order:
-     - `:allow` if `allow` exists
-     - `:default` if `default` exists
-     - `:echo` if `echo` exists
-     - `:exec` if `exec` exists
-     - `false` if none of the above exist
-
-2. **Act Phase** (when block is activated) per FCB.act_source:
-   - If `act` is `false`: Block cannot be activated
-   - If `act` is `:allow`: User selects from `allow` list
-   - If `act` is `:echo`: Value from `echo` key is evaluated and returned
-   - If `act` is `:edit`: User is prompted for input
-   - If `act` is `:exec`: Command from `exec` key is executed and stdout is returned
-   - If `act` is not present: Defaults to:
-     - If `init` is `false`:
-       - First available in order: `:allow`, `:echo`, `:edit`, `:exec`
-     - Otherwise:
-       - `:allow` if `allow` exists
-       - `:edit` if `allow` does not exist
-
-## Examples
-
-### Echo on Init, Exec on Act
-```ux
-name: DEPLOY_CONFIG
-init: :echo
-echo: "Deploying ${VERSION} to ${ENVIRONMENT}"
-act: :exec
-exec: "deploy.sh ${ENVIRONMENT} ${VERSION}"
-```
-Behavior:
-- On init: Evaluates echo string "Deploying ${VERSION} to ${ENVIRONMENT}"
-- On act: Executes deploy.sh with environment and version parameters
-
-### Allow on Init, Edit on Act
-```ux
-name: ENVIRONMENT
-init: :allow
-allow:
-  - development
-  - staging
-  - production
-act: :edit
-prompt: Select environment
-```
-Behavior:
-- On init: Uses first allowed value (development)
-- On act: Prompts user to select from allowed values
-
-### Exec on Init, Echo on Act
-```ux
-name: CURRENT_DIR
-init: :exec
-exec: basename $(pwd)
-act: :echo
-echo: "Current directory: ${CURRENT_DIR}"
-```
-Behavior:
-- On init: Executes basename command on current directory
-- On act: Evaluates echo string with current directory value
-
-### Allow on Both
-```ux
-name: API_KEY
-init: :allow
-allow:
-  - ${PROD_API_KEY}
-  - ${STAGING_API_KEY}
-  - ${DEV_API_KEY}
-act: :allow
-require:
-  - ENVIRONMENT
-```
-Behavior:
-- On init: Uses first allowed API key
-- On act: Shows menu of allowed API keys for selection
-
-### Echo on Both
-```ux
-name: SHELL_VERSION
-init: :echo
-echo: $SHELL
-act: :echo
-echo: "Using shell: ${SHELL_VERSION}"
-```
-Behavior:
-- On init: Gets shell value from environment
-- On act: Evaluates echo string with current shell value