start-command 0.7.4 → 0.7.6

package/docs/case-studies/issue-28/README.md

@@ -0,0 +1,405 @@
+# Case Study: Issue #28 - Shell Quoting with Pipe Operator
+
+## Issue Summary
+
+**Issue URL:** https://github.com/link-foundation/start/issues/28
+**Date Reported:** 2025-12-24
+**Reporter:** @konard
+**Labels:** bug, documentation, enhancement, question
+**Status:** Under Analysis
+
+### Problem Statement
+
+When using the `$` command (start-command) with pipe operators (`|`) and double quotes (`"`) in the command, users need to wrap the entire command in single quotes to get the expected behavior. Without the quotes, the shell parses the pipe operator **before** passing arguments to the `$` command.
+
+**Example - Undesired behavior:**
+
+```bash
+$ echo "hi" | agent
+```
+
+The shell interprets this as: "Run `$ echo "hi"` and pipe its output to `agent`"
+
+**Example - Desired behavior (requires single quotes):**
+
+```bash
+$ 'echo "hi" | agent'
+```
+
+The shell interprets this as: "Run `$` with the argument `echo "hi" | agent`"
+
+### Environment
+
+- **Platform:** macOS (darwin)
+- **Package:** start-command (version assumed latest)
+- **Shell:** /bin/zsh
+- **Bun Version:** 1.2.20
+- **Working Directory:** /Users/konard
+
+## Timeline of Events
+
+Based on the terminal logs provided in the issue:
+
+### Timestamp 1: 15:07:08.894 - Unquoted Command
+
+1. **User input:** `$ echo "hi" | agent`
+2. **Shell parsing:** The shell parsed this as TWO separate commands:
+   - Command 1: `$ echo "hi"` (run `$` with argument `echo "hi"`)
+   - Pipe: `|` (pipe stdout of command 1 to stdin of command 2)
+   - Command 2: `agent` (run `agent` with stdin from pipe)
+3. **Result:** The `$` command executed `echo "hi"` and logged it, showing:
+   ```
+   Command: echo hi
+   Exit Code: 0
+   ```
+4. **What agent received:** The `agent` received the JSON output from the `$` command (status messages, tool use messages, etc.) via the pipe - NOT the original command.
+5. **Agent's response:** The agent saw the log file and responded about the `echo hi` execution, not the "hi" message itself.
+
+### Timestamp 2: 15:07:33.420 - Quoted Command
+
+1. **User input:** `$ 'echo "hi" | agent'`
+2. **Shell parsing:** The shell parsed this as ONE command:
+   - Command: `$` with a single argument: `echo "hi" | agent`
+3. **Result:** The `$` command received the entire string and executed it as a pipeline
+4. **What agent received:** The output of `echo "hi"` which is `hi`
+5. **Agent's response:** "Hello! How can I help you today?" (responding to "hi" as input)
+
+## Sequence Diagrams
+
+### Scenario 1: Without Quotes (Problematic)
+
+```
+User Input:  $ echo "hi" | agent
+             └───┬────┘   └──┬──┘
+                 │           │
+    ┌────────────┴───────────┴────────────┐
+    │         Shell (zsh/bash)            │
+    │  Tokenizes command line into:       │
+    │    Command 1: ['$', 'echo', '"hi"'] │
+    │    Operator: |                      │
+    │    Command 2: ['agent']             │
+    └────────────────────────────────────┘
+                 │           │
+    ┌────────────┴───┐   ┌───┴────────────┐
+    │  $ (start-cmd) │   │     agent      │
+    │  executes:     │──→│  receives:     │
+    │  "echo hi"     │   │  JSON output   │
+    │                │   │  from $ cmd    │
+    └────────────────┘   └────────────────┘
+```
+
+### Scenario 2: With Quotes (Desired)
+
+```
+User Input:  $ 'echo "hi" | agent'
+             └─────────┬──────────┘
+                       │
+    ┌──────────────────┴──────────────────┐
+    │         Shell (zsh/bash)            │
+    │  Tokenizes command line into:       │
+    │    Command: ['$', 'echo "hi" | agent']
+    │  Single quotes prevent parsing      │
+    └──────────────────────────────────────┘
+                       │
+    ┌──────────────────┴──────────────────┐
+    │          $ (start-command)          │
+    │  Receives: "echo \"hi\" | agent"    │
+    │  Spawns shell with:                 │
+    │    /bin/zsh -c 'echo "hi" | agent'  │
+    │                                     │
+    │  ┌──────────┐      ┌─────────┐     │
+    │  │ echo "hi"│─────→│  agent  │     │
+    │  │ stdout:  │      │ stdin:  │     │
+    │  │ "hi"     │      │ "hi"    │     │
+    │  └──────────┘      └─────────┘     │
+    └──────────────────────────────────────┘
+```
+
+## Root Cause Analysis
+
+### PRIMARY ROOT CAUSE: Shell Grammar and Operator Precedence
+
+The root cause is **not a bug in start-command** but rather a fundamental aspect of how POSIX shells parse command lines.
+
+According to the [Bash Reference Manual - Pipelines](https://www.gnu.org/software/bash/manual/html_node/Pipelines.html):
+
+> "A pipeline is a sequence of one or more commands separated by one of the control operators `|` or `|&`."
+
+The shell's parsing order is:
+
+1. **Tokenization:** The command line is split into tokens
+2. **Operator Recognition:** Pipe (`|`), redirections (`>`, `<`), and logical operators (`&&`, `||`) are identified
+3. **Command Grouping:** Commands are grouped based on operators
+4. **Quote Processing:** Quotes affect tokenization but don't change operator recognition
+
+### Why This Happens
+
+When you type:
+
+```bash
+$ echo "hi" | agent
+```
+
+The shell sees:
+
+- `$` - a command (the start-command binary)
+- `echo` - an argument to `$`
+- `"hi"` - another argument to `$` (quotes are stripped by shell)
+- `|` - **pipe operator** (this is a metacharacter, not an argument)
+- `agent` - a separate command that receives piped input
+
+The pipe operator has special meaning to the shell and is **never** passed as an argument to commands unless quoted or escaped.
+
+### Single Quotes Behavior
+
+From the [GNU Bash Reference - Quoting](https://www.gnu.org/software/bash/manual/html_node/Quoting.html):
+
+> "Enclosing characters in single quotes (`'`) preserves the literal value of each character within the quotes."
+
+This means:
+
+```bash
+$ 'echo "hi" | agent'
+```
+
+The shell sees:
+
+- `$` - a command
+- `echo "hi" | agent` - a single string argument (the `|` inside is literal, not a pipe operator)
+
+### Operator Precedence Summary
+
+From [Shell Grammar Rules](https://bargenqua.st/posts/bash-pipes/):
+
+1. Redirections (`<`, `>`, `>>`) - highest precedence
+2. Pipes (`|`) - next highest
+3. Command separators (`;`, `&`, `&&`, `||`) - lower precedence
+
+## Impact Assessment
+
+### Who Is Affected
+
+Users who want to:
+
+1. Pipe content **through** the `$` command to another command
+2. Use the `$` command to execute complex pipelines
+3. Use `$` as part of a larger shell pipeline
+
+### Severity
+
+**Low-Medium**: The current behavior is technically correct - it follows POSIX shell semantics. However, it may be unintuitive for users who expect the entire command line to be passed to the `$` command.
+
+## Proposed Solutions
+
+### Solution 1: Pipe TO `$` (Recommended - Preferred Approach)
+
+Instead of quoting, put `$` on the **receiving** command in the pipeline:
+
+```bash
+# Preferred - pipe TO the $-wrapped command
+echo "hi" | $ agent
+```
+
+This works because:
+
+1. The shell creates a pipeline: `echo "hi"` piped to `$ agent`
+2. `$ agent` receives "hi" on stdin
+3. The `$` command wraps `agent`, which processes the input correctly
+
+**Pros:**
+
+- No quoting needed
+- Simple and intuitive
+- Works naturally with shell pipelines
+
+**Cons:**
+
+- Only works when piping TO a command (not for entire pipelines)
+
+### Solution 2: Documentation Enhancement (Also Recommended)
+
+Add clear documentation explaining:
+
+1. How shell parsing affects the `$` command
+2. When to use quotes around complex commands
+3. Examples of common use cases
+
+**Pros:**
+
+- No code changes required
+- Educates users about shell behavior
+- Maintains POSIX compliance
+
+**Cons:**
+
+- Requires users to learn quoting rules
+- May still feel unintuitive
+
+### Solution 3: Use a Multi-Character Command Name
+
+Instead of `$`, use a multi-character name like `start` or `run`:
+
+```bash
+start 'echo "hi" | agent'    # Still needs quotes
+start echo "hi" | agent       # Same issue
+```
+
+**Pros:**
+
+- Less visual confusion with shell's `$` variable syntax
+- Easier to google and document
+
+**Cons:**
+
+- Doesn't solve the fundamental issue
+- Breaks backward compatibility
+
+### Solution 4: Wrapper Script with Here-Document
+
+Create an alternative invocation method:
+
+```bash
+$$ <<'CMD'
+echo "hi" | agent
+CMD
+```
+
+**Pros:**
+
+- No quoting issues
+- Can handle multi-line commands
+- Clear visual separation
+
+**Cons:**
+
+- Requires new syntax
+- More verbose for simple commands
+
+### Solution 5: Shell Function Alternative
+
+Provide a shell function that users can source:
+
+```bash
+# In ~/.zshrc or ~/.bashrc
+run() {
+  $ "$*"
+}
+
+# Usage:
+run echo "hi" | agent  # Still has the same issue!
+```
+
+**Pros:**
+
+- More conventional naming
+
+**Cons:**
+
+- **Does not solve the problem** - shell parsing happens before function call
+
+### Solution 6: Interactive Mode Detection
+
+When `$` detects it's being piped to/from, emit a warning:
+
+```bash
+$ echo "hi" | agent
+# Warning: $ stdout is being piped. Did you mean: $ 'echo "hi" | agent'?
+```
+
+**Pros:**
+
+- Helps users understand what's happening
+- Non-breaking change
+
+**Cons:**
+
+- Adds noise to output
+- May interfere with legitimate piping use cases
+
+### Solution 7: ZSH Global Alias (ZSH Only)
+
+ZSH supports [global aliases](https://vonheikemen.github.io/devlog/tools/zsh-global-aliases/) that expand anywhere in the command:
+
+```zsh
+# In ~/.zshrc
+alias -g '|$'='| $'  # Not helpful
+```
+
+**Analysis:** This doesn't solve the problem because the issue is with the **input** to `$`, not its output.
+
+## Recommended Approach
+
+After careful analysis, the **recommended approach is Solution 1: Pipe TO `$`** combined with documentation enhancement:
+
+1. **Preferred syntax**: `echo "hi" | $ agent` - pipe TO the `$`-wrapped command
+2. **Alternative**: `$ 'echo "hi" | agent'` - quote the entire pipeline when needed
+3. **Documentation**: Clear docs explaining both approaches
+
+### Implementation Plan
+
+1. **Documentation (docs/PIPES.md)** - New file:
+   - Detailed explanation of pipe usage
+   - Preferred approach: piping TO `$`
+   - Alternative approach: quoting
+   - Examples and troubleshooting
+
+2. **Documentation (docs/USAGE.md)**:
+   - Brief section on pipes with link to PIPES.md
+   - Updated examples showing preferred syntax
+
+3. **README.md Update**:
+   - Add piping examples showing preferred syntax
+   - Link to detailed documentation
+
+4. **CLI Help Update** (Future Enhancement):
+   - Add a note about piping in `$ --help`
+
+5. **Optional Warning** (Future Enhancement):
+   - Detect if stdout is a pipe
+   - Detect if command doesn't contain pipe-like patterns
+   - Emit helpful warning
+
+## Key Learnings
+
+1. **Shell parsing is fundamental**: Metacharacters like `|`, `&`, `;` are processed by the shell before commands receive arguments. This is not something applications can change.
+
+2. **Single quotes are your friend**: When you want literal characters, use single quotes. When you want variable expansion inside, use double quotes with escaped special characters.
+
+3. **The `$` symbol is contextual**: In shells, `$` typically introduces variable expansion (`$HOME`, `$PATH`). Using it as a command name, while valid, can be visually confusing.
+
+4. **Education over code changes**: Some issues are best solved by educating users rather than adding complex workarounds that might introduce new problems.
+
+5. **POSIX compliance matters**: Shell behavior is standardized. Fighting against it often creates more problems than it solves.
+
+## Related Research
+
+### Shell Quoting Resources
+
+- [GNU Bash Reference Manual - Quoting](https://www.gnu.org/software/bash/manual/html_node/Quoting.html)
+- [GNU Bash Reference Manual - Pipelines](https://www.gnu.org/software/bash/manual/html_node/Pipelines.html)
+- [Advanced Quoting in Shell Scripts](https://scriptingosx.com/2020/04/advanced-quoting-in-shell-scripts/)
+- [ZSH Global Aliases](https://vonheikemen.github.io/devlog/tools/zsh-global-aliases/)
+
+### Shell Parsing Order
+
+1. Tokenization (splitting into words)
+2. Command identification
+3. Alias expansion (if applicable)
+4. Brace expansion
+5. Tilde expansion
+6. Parameter/variable expansion
+7. Command substitution
+8. Arithmetic expansion
+9. Word splitting
+10. Filename expansion (globbing)
+11. Quote removal
+
+The pipe operator (`|`) is recognized during **step 1 (tokenization)**, before any command receives its arguments.
+
+## References
+
+- [Bash Reference Manual](https://www.gnu.org/software/bash/manual/bash.html)
+- [ZSH Expansion and Substitution](https://zsh.sourceforge.io/Doc/Release/Expansion.html)
+- [POSIX Shell Command Language](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html)
+- [Issue #25 Case Study](../issue-25/README.md) - Related shell quoting issues
+- [Baeldung - Special Dollar Sign Variables](https://www.baeldung.com/linux/shell-special-dollar-sign-variables)

package/docs/case-studies/issue-28/issue-data.json

@@ -0,0 +1,105 @@
+{
+  "issue": {
+    "number": 28,
+    "title": "Can we somehow overcome the need of `'` quotes with `|` and `\"` in a sequence?",
+    "url": "https://github.com/link-foundation/start/issues/28",
+    "state": "open",
+    "labels": ["bug", "documentation", "enhancement", "question"],
+    "author": "konard",
+    "created_at": "2025-12-24"
+  },
+  "environment": {
+    "platform": "darwin",
+    "shell": "/bin/zsh",
+    "bun_version": "1.2.20",
+    "working_directory": "/Users/konard"
+  },
+  "scenarios": [
+    {
+      "name": "unquoted_command",
+      "input": "$ echo \"hi\" | agent",
+      "timestamp": "2025-12-24 15:07:08.894",
+      "shell_interpretation": {
+        "command_1": "$ echo \"hi\"",
+        "operator": "|",
+        "command_2": "agent"
+      },
+      "actual_execution": {
+        "start_command_received": "echo \"hi\"",
+        "start_command_output": "hi",
+        "agent_received": "JSON output from start-command stdout"
+      },
+      "user_expectation": "agent should receive 'hi' from echo",
+      "actual_result": "agent received start-command JSON output",
+      "exit_code": 0
+    },
+    {
+      "name": "quoted_command",
+      "input": "$ 'echo \"hi\" | agent'",
+      "timestamp": "2025-12-24 15:07:33.420",
+      "shell_interpretation": {
+        "command": "$",
+        "argument": "echo \"hi\" | agent"
+      },
+      "actual_execution": {
+        "start_command_received": "echo \"hi\" | agent",
+        "pipeline_executed": true,
+        "echo_output": "hi",
+        "agent_received": "hi"
+      },
+      "user_expectation": "agent should receive 'hi' from echo",
+      "actual_result": "agent received 'hi' from echo",
+      "exit_code": 0
+    }
+  ],
+  "root_cause": {
+    "category": "shell_parsing_behavior",
+    "is_bug": false,
+    "description": "The pipe operator (|) is a shell metacharacter that is parsed by the shell before command arguments are passed. This is standard POSIX shell behavior, not a bug in start-command.",
+    "affected_shells": ["bash", "zsh", "sh", "dash", "ksh"],
+    "shell_parsing_order": [
+      "1. Tokenization - command line split into tokens",
+      "2. Operator recognition - | && || ; & recognized as metacharacters",
+      "3. Command grouping - commands grouped based on operators",
+      "4. Quote processing - quoted strings preserved",
+      "5. Expansion - variables, commands, etc. expanded"
+    ]
+  },
+  "solutions": {
+    "recommended": "documentation_enhancement",
+    "options": [
+      {
+        "id": "documentation_enhancement",
+        "name": "Documentation Enhancement",
+        "description": "Add clear documentation about quoting requirements",
+        "pros": ["No code changes", "Educates users", "POSIX compliant"],
+        "cons": ["Requires users to learn quoting rules"],
+        "complexity": "low"
+      },
+      {
+        "id": "warning_detection",
+        "name": "Optional Warning Detection",
+        "description": "Detect when stdout is piped and emit helpful warning",
+        "pros": ["Helps users understand behavior", "Non-breaking"],
+        "cons": ["May add noise to output"],
+        "complexity": "medium"
+      },
+      {
+        "id": "here_document",
+        "name": "Here-Document Wrapper",
+        "description": "Support $$ <<'CMD' syntax for multi-line commands",
+        "pros": ["No quoting issues", "Clear syntax"],
+        "cons": ["More verbose", "New syntax to learn"],
+        "complexity": "high"
+      }
+    ]
+  },
+  "references": {
+    "bash_manual": "https://www.gnu.org/software/bash/manual/bash.html",
+    "pipelines_doc": "https://www.gnu.org/software/bash/manual/html_node/Pipelines.html",
+    "quoting_doc": "https://www.gnu.org/software/bash/manual/html_node/Quoting.html",
+    "posix_shell": "https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html"
+  },
+  "analysis_date": "2025-12-24",
+  "analyst": "AI Issue Solver"
+}

package/docs/case-studies/issue-28/raw-issue-data.md

@@ -0,0 +1,92 @@
+# Issue #28 Raw Data
+
+## Issue Title
+
+Can we somehow overcome the need of `'` quotes with `|` and `"` in a sequence?
+
+## Issue State
+
+OPEN
+
+## Issue Labels
+
+bug, documentation, enhancement, question
+
+## Issue Author
+
+konard
+
+## Issue Content
+
+The issue shows two terminal sessions demonstrating different behaviors:
+
+### Session 1: Without quoting the entire command
+
+```bash
+konard@MacBook-Pro-Konstantin ~ % $ echo "hi" | agent
+```
+
+**Result:** The `echo "hi"` command ran directly in the shell, and only its OUTPUT was piped to `agent`. The agent then processed the log file that was created by the `$` command (start-command), not the original command input.
+
+The agent received the OUTPUT of `$ echo "hi"` which was captured in a log file, and then read that log file.
+
+### Session 2: With quoting the entire command
+
+```bash
+konard@MacBook-Pro-Konstantin ~ % $ 'echo "hi" | agent'
+```
+
+**Result:** The entire string `echo "hi" | agent` was passed TO the `$` command (start-command). The start-command then executed this complete pipeline, which resulted in the `echo "hi"` output being piped to `agent` correctly.
+
+## Terminal Logs
+
+### Session 1 Output (problematic behavior)
+
+```json
+{
+  "type": "status",
+  "mode": "stdin-stream",
+  "message": "Agent CLI in continuous listening mode. Accepts JSON and plain text input.",
+  "hint": "Press CTRL+C to exit. Use --help for options.",
+  "acceptedFormats": ["JSON object with \"message\" field", "Plain text"],
+  "options": {
+    "interactive": true,
+    "autoMergeQueuedMessages": true,
+    "alwaysAcceptStdin": true,
+    "compactJson": false
+  }
+}
+```
+
+The agent then read a log file:
+
+```
+=== Start Command Log ===
+Timestamp: 2025-12-24 15:07:08.894
+Command: echo hi
+Shell: /bin/zsh
+Platform: darwin
+Bun Version: 1.2.20
+Working Directory: /Users/konard
+==================================================
+
+hi
+
+==================================================
+Finished: 2025-12-24 15:07:08.906
+Exit Code: 0
+```
+
+And responded:
+
+> "It looks like you executed the command `echo hi`, which successfully output \"hi\" with exit code 0..."
+
+### Session 2 Output (desired behavior)
+
+The agent received "hi" directly as input and responded:
+
+> "Hello! How can I help you today?"
+
+## User Request
+
+> Please download all logs and data related about the issue to this repository, make sure we compile that data to `./docs/case-studies/issue-{id}` folder, and use it to do deep case study analysis (also make sure to search online for additional facts and data), in which we will reconstruct timeline/sequence of events, find root causes of the problem, and propose possible solutions.