@machina.ai/cell-cli-core 1.20.2-rc1 → 1.25.0-rc1

package/dist/docs/core/index.md

@@ -0,0 +1,101 @@
+# Cell CLI core
+
+Cell CLI's core package (`packages/core`) is the backend portion of Cell CLI,
+handling communication with the Gemini API, managing tools, and processing
+requests sent from `packages/cli`. For a general overview of Cell CLI, see the
+[main documentation page](../index.md).
+
+## Navigating this section
+
+- **[Core tools API](./tools-api.md):** Information on how tools are defined,
+  registered, and used by the core.
+- **[Memory Import Processor](./memport.md):** Documentation for the modular
+  GEMINI.md import feature using @file.md syntax.
+- **[Policy Engine](./policy-engine.md):** Use the Policy Engine for
+  fine-grained control over tool execution.
+
+## Role of the core
+
+While the `packages/cli` portion of Cell CLI provides the user interface,
+`packages/core` is responsible for:
+
+- **Gemini API interaction:** Securely communicating with the Google Gemini API,
+  sending user prompts, and receiving model responses.
+- **Prompt engineering:** Constructing effective prompts for the Gemini model,
+  potentially incorporating conversation history, tool definitions, and
+  instructional context from `GEMINI.md` files.
+- **Tool management & orchestration:**
+  - Registering available tools (e.g., file system tools, shell command
+    execution).
+  - Interpreting tool use requests from the Gemini model.
+  - Executing the requested tools with the provided arguments.
+  - Returning tool execution results to the Gemini model for further processing.
+- **Session and state management:** Keeping track of the conversation state,
+  including history and any relevant context required for coherent interactions.
+- **Configuration:** Managing core-specific configurations, such as API key
+  access, model selection, and tool settings.
+
+## Security considerations
+
+The core plays a vital role in security:
+
+- **API key management:** It handles the `GEMINI_API_KEY` and ensures it's used
+  securely when communicating with the Gemini API.
+- **Tool execution:** When tools interact with the local system (e.g.,
+  `run_shell_command`), the core (and its underlying tool implementations) must
+  do so with appropriate caution, often involving sandboxing mechanisms to
+  prevent unintended modifications.
+
+## Chat history compression
+
+To ensure that long conversations don't exceed the token limits of the Gemini
+model, the core includes a chat history compression feature.
+
+When a conversation approaches the token limit for the configured model, the
+core automatically compresses the conversation history before sending it to the
+model. This compression is designed to be lossless in terms of the information
+conveyed, but it reduces the overall number of tokens used.
+
+You can find the token limits for each model in the
+[Google AI documentation](https://ai.google.dev/gemini-api/docs/models).
+
+## Model fallback
+
+Cell CLI includes a model fallback mechanism to ensure that you can continue to
+use the CLI even if the default "pro" model is rate-limited.
+
+If you are using the default "pro" model and the CLI detects that you are being
+rate-limited, it automatically switches to the "flash" model for the current
+session. This allows you to continue working without interruption.
+
+## File discovery service
+
+The file discovery service is responsible for finding files in the project that
+are relevant to the current context. It is used by the `@` command and other
+tools that need to access files.
+
+## Memory discovery service
+
+The memory discovery service is responsible for finding and loading the
+`GEMINI.md` files that provide context to the model. It searches for these files
+in a hierarchical manner, starting from the current working directory and moving
+up to the project root and the user's home directory. It also searches in
+subdirectories.
+
+This allows you to have global, project-level, and component-level context
+files, which are all combined to provide the model with the most relevant
+information.
+
+You can use the [`/memory` command](../cli/commands.md) to `show`, `add`, and
+`refresh` the content of loaded `GEMINI.md` files.
+
+## Citations
+
+When Gemini finds it is reciting text from a source it appends the citation to
+the output. It is enabled by default but can be disabled with the
+ui.showCitations setting.
+
+- When proposing an edit the citations display before giving the user the option
+  to accept.
+- Citations are always shown at the end of the model’s turn.
+- We deduplicate citations and display them in alphabetical order.

package/dist/docs/core/memport.md

@@ -0,0 +1,246 @@
+# Memory Import Processor
+
+The Memory Import Processor is a feature that allows you to modularize your
+GEMINI.md files by importing content from other files using the `@file.md`
+syntax.
+
+## Overview
+
+This feature enables you to break down large GEMINI.md files into smaller, more
+manageable components that can be reused across different contexts. The import
+processor supports both relative and absolute paths, with built-in safety
+features to prevent circular imports and ensure file access security.
+
+## Syntax
+
+Use the `@` symbol followed by the path to the file you want to import:
+
+```markdown
+# Main GEMINI.md file
+
+This is the main content.
+
+@./components/instructions.md
+
+More content here.
+
+@./shared/configuration.md
+```
+
+## Supported path formats
+
+### Relative paths
+
+- `@./file.md` - Import from the same directory
+- `@../file.md` - Import from parent directory
+- `@./components/file.md` - Import from subdirectory
+
+### Absolute paths
+
+- `@/absolute/path/to/file.md` - Import using absolute path
+
+## Examples
+
+### Basic import
+
+```markdown
+# My GEMINI.md
+
+Welcome to my project!
+
+@./get-started.md
+
+## Features
+
+@./features/overview.md
+```
+
+### Nested imports
+
+The imported files can themselves contain imports, creating a nested structure:
+
+```markdown
+# main.md
+
+@./header.md @./content.md @./footer.md
+```
+
+```markdown
+# header.md
+
+# Project Header
+
+@./shared/title.md
+```
+
+## Safety features
+
+### Circular import detection
+
+The processor automatically detects and prevents circular imports:
+
+```markdown
+# file-a.md
+
+@./file-b.md
+```
+
+```markdown
+# file-b.md
+
+@./file-a.md <!-- This will be detected and prevented -->
+```
+
+### File access security
+
+The `validateImportPath` function ensures that imports are only allowed from
+specified directories, preventing access to sensitive files outside the allowed
+scope.
+
+### Maximum import depth
+
+To prevent infinite recursion, there's a configurable maximum import depth
+(default: 5 levels).
+
+## Error handling
+
+### Missing files
+
+If a referenced file doesn't exist, the import will fail gracefully with an
+error comment in the output.
+
+### File access errors
+
+Permission issues or other file system errors are handled gracefully with
+appropriate error messages.
+
+## Code region detection
+
+The import processor uses the `marked` library to detect code blocks and inline
+code spans, ensuring that `@` imports inside these regions are properly ignored.
+This provides robust handling of nested code blocks and complex Markdown
+structures.
+
+## Import tree structure
+
+The processor returns an import tree that shows the hierarchy of imported files,
+similar to Claude's `/memory` feature. This helps users debug problems with
+their GEMINI.md files by showing which files were read and their import
+relationships.
+
+Example tree structure:
+
+```
+Memory Files
+ L project: GEMINI.md
+            L a.md
+              L b.md
+                L c.md
+              L d.md
+                L e.md
+                  L f.md
+            L included.md
+```
+
+The tree preserves the order that files were imported and shows the complete
+import chain for debugging purposes.
+
+## Comparison to Claude Code's `/memory` (`claude.md`) approach
+
+Claude Code's `/memory` feature (as seen in `claude.md`) produces a flat, linear
+document by concatenating all included files, always marking file boundaries
+with clear comments and path names. It does not explicitly present the import
+hierarchy, but the LLM receives all file contents and paths, which is sufficient
+for reconstructing the hierarchy if needed.
+
+> [!NOTE] The import tree is mainly for clarity during development and has
+> limited relevance to LLM consumption.
+
+## API reference
+
+### `processImports(content, basePath, debugMode?, importState?)`
+
+Processes import statements in GEMINI.md content.
+
+**Parameters:**
+
+- `content` (string): The content to process for imports
+- `basePath` (string): The directory path where the current file is located
+- `debugMode` (boolean, optional): Whether to enable debug logging (default:
+  false)
+- `importState` (ImportState, optional): State tracking for circular import
+  prevention
+
+**Returns:** Promise&lt;ProcessImportsResult&gt; - Object containing processed
+content and import tree
+
+### `ProcessImportsResult`
+
+```typescript
+interface ProcessImportsResult {
+  content: string; // The processed content with imports resolved
+  importTree: MemoryFile; // Tree structure showing the import hierarchy
+}
+```
+
+### `MemoryFile`
+
+```typescript
+interface MemoryFile {
+  path: string; // The file path
+  imports?: MemoryFile[]; // Direct imports, in the order they were imported
+}
+```
+
+### `validateImportPath(importPath, basePath, allowedDirectories)`
+
+Validates import paths to ensure they are safe and within allowed directories.
+
+**Parameters:**
+
+- `importPath` (string): The import path to validate
+- `basePath` (string): The base directory for resolving relative paths
+- `allowedDirectories` (string[]): Array of allowed directory paths
+
+**Returns:** boolean - Whether the import path is valid
+
+### `findProjectRoot(startDir)`
+
+Finds the project root by searching for a `.git` directory upwards from the
+given start directory. Implemented as an **async** function using non-blocking
+file system APIs to avoid blocking the Node.js event loop.
+
+**Parameters:**
+
+- `startDir` (string): The directory to start searching from
+
+**Returns:** Promise&lt;string&gt; - The project root directory (or the start
+directory if no `.git` is found)
+
+## Best Practices
+
+1. **Use descriptive file names** for imported components
+2. **Keep imports shallow** - avoid deeply nested import chains
+3. **Document your structure** - maintain a clear hierarchy of imported files
+4. **Test your imports** - ensure all referenced files exist and are accessible
+5. **Use relative paths** when possible for better portability
+
+## Troubleshooting
+
+### Common issues
+
+1. **Import not working**: Check that the file exists and the path is correct
+2. **Circular import warnings**: Review your import structure for circular
+   references
+3. **Permission errors**: Ensure the files are readable and within allowed
+   directories
+4. **Path resolution issues**: Use absolute paths if relative paths aren't
+   resolving correctly
+
+### Debug mode
+
+Enable debug mode to see detailed logging of the import process:
+
+```typescript
+const result = await processImports(content, basePath, true);
+```

package/dist/docs/core/policy-engine.md

@@ -0,0 +1,268 @@
+# Policy engine
+
+The Cell CLI includes a powerful policy engine that provides fine-grained
+control over tool execution. It allows users and administrators to define rules
+that determine whether a tool call should be allowed, denied, or require user
+confirmation.
+
+## Quick start
+
+To create your first policy:
+
+1.  **Create the policy directory** if it doesn't exist:
+    ```bash
+    mkdir -p ~/.cell-cli/policies
+    ```
+2.  **Create a new policy file** (e.g., `~/.cell-cli/policies/my-rules.toml`).
+    You can use any filename ending in `.toml`; all such files in this directory
+    will be loaded and combined:
+    ```toml
+    [[rule]]
+    toolName = "run_shell_command"
+    commandPrefix = "git status"
+    decision = "allow"
+    priority = 100
+    ```
+3.  **Run a command** that triggers the policy (e.g., ask Cell CLI to
+    `git status`). The tool will now execute automatically without prompting for
+    confirmation.
+
+## Core concepts
+
+The policy engine operates on a set of rules. Each rule is a combination of
+conditions and a resulting decision. When a large language model wants to
+execute a tool, the policy engine evaluates all rules to find the
+highest-priority rule that matches the tool call.
+
+A rule consists of the following main components:
+
+- **Conditions**: Criteria that a tool call must meet for the rule to apply.
+  This can include the tool's name, the arguments provided to it, or the current
+  approval mode.
+- **Decision**: The action to take if the rule matches (`allow`, `deny`, or
+  `ask_user`).
+- **Priority**: A number that determines the rule's precedence. Higher numbers
+  win.
+
+For example, this rule will ask for user confirmation before executing any `git`
+command.
+
+```toml
+[[rule]]
+toolName = "run_shell_command"
+commandPrefix = "git "
+decision = "ask_user"
+priority = 100
+```
+
+### Conditions
+
+Conditions are the criteria that a tool call must meet for a rule to apply. The
+primary conditions are the tool's name and its arguments.
+
+#### Tool Name
+
+The `toolName` in the rule must match the name of the tool being called.
+
+- **Wildcards**: For Model-hosting-protocol (MCP) servers, you can use a
+  wildcard. A `toolName` of `my-server__*` will match any tool from the
+  `my-server` MCP.
+
+#### Arguments pattern
+
+If `argsPattern` is specified, the tool's arguments are converted to a stable
+JSON string, which is then tested against the provided regular expression. If
+the arguments don't match the pattern, the rule does not apply.
+
+### Decisions
+
+There are three possible decisions a rule can enforce:
+
+- `allow`: The tool call is executed automatically without user interaction.
+- `deny`: The tool call is blocked and is not executed.
+- `ask_user`: The user is prompted to approve or deny the tool call. (In
+  non-interactive mode, this is treated as `deny`.)
+
+### Priority system and tiers
+
+The policy engine uses a sophisticated priority system to resolve conflicts when
+multiple rules match a single tool call. The core principle is simple: **the
+rule with the highest priority wins**.
+
+To provide a clear hierarchy, policies are organized into three tiers. Each tier
+has a designated number that forms the base of the final priority calculation.
+
+| Tier    | Base | Description                                                                |
+| :------ | :--- | :------------------------------------------------------------------------- |
+| Default | 1    | Built-in policies that ship with the Cell CLI.                             |
+| User    | 2    | Custom policies defined by the user.                                       |
+| Admin   | 3    | Policies managed by an administrator (e.g., in an enterprise environment). |
+
+Within a TOML policy file, you assign a priority value from **0 to 999**. The
+engine transforms this into a final priority using the following formula:
+
+`final_priority = tier_base + (toml_priority / 1000)`
+
+This system guarantees that:
+
+- Admin policies always override User and Default policies.
+- User policies always override Default policies.
+- You can still order rules within a single tier with fine-grained control.
+
+For example:
+
+- A `priority: 50` rule in a Default policy file becomes `1.050`.
+- A `priority: 100` rule in a User policy file becomes `2.100`.
+- A `priority: 20` rule in an Admin policy file becomes `3.020`.
+
+### Approval modes
+
+Approval modes allow the policy engine to apply different sets of rules based on
+the CLI's operational mode. A rule can be associated with one or more modes
+(e.g., `yolo`, `autoEdit`). The rule will only be active if the CLI is running
+in one of its specified modes. If a rule has no modes specified, it is always
+active.
+
+## Rule matching
+
+When a tool call is made, the engine checks it against all active rules,
+starting from the highest priority. The first rule that matches determines the
+outcome.
+
+A rule matches a tool call if all of its conditions are met:
+
+1.  **Tool name**: The `toolName` in the rule must match the name of the tool
+    being called.
+    - **Wildcards**: For Model-hosting-protocol (MCP) servers, you can use a
+      wildcard. A `toolName` of `my-server__*` will match any tool from the
+      `my-server` MCP.
+2.  **Arguments pattern**: If `argsPattern` is specified, the tool's arguments
+    are converted to a stable JSON string, which is then tested against the
+    provided regular expression. If the arguments don't match the pattern, the
+    rule does not apply.
+
+## Configuration
+
+Policies are defined in `.toml` files. The CLI loads these files from Default,
+User, and (if configured) Admin directories.
+
+### TOML rule schema
+
+Here is a breakdown of the fields available in a TOML policy rule:
+
+```toml
+[[rule]]
+# A unique name for the tool, or an array of names.
+toolName = "run_shell_command"
+
+# (Optional) The name of an MCP server. Can be combined with toolName
+# to form a composite name like "mcpName__toolName".
+mcpName = "my-custom-server"
+
+# (Optional) A regex to match against the tool's arguments.
+argsPattern = '"command":"(git|npm)'
+
+# (Optional) A string or array of strings that a shell command must start with.
+# This is syntactic sugar for `toolName = "run_shell_command"` and an `argsPattern`.
+commandPrefix = "git "
+
+# (Optional) A regex to match against the entire shell command.
+# This is also syntactic sugar for `toolName = "run_shell_command"`.
+# Note: This pattern is tested against the JSON representation of the arguments (e.g., `{"command":"<your_command>"}`), so anchors like `^` or `$` will apply to the full JSON string, not just the command text.
+# You cannot use commandPrefix and commandRegex in the same rule.
+commandRegex = "^git (commit|push)"
+
+# The decision to take. Must be "allow", "deny", or "ask_user".
+decision = "ask_user"
+
+# The priority of the rule, from 0 to 999.
+priority = 10
+
+# (Optional) An array of approval modes where this rule is active.
+modes = ["autoEdit"]
+```
+
+### Using arrays (lists)
+
+To apply the same rule to multiple tools or command prefixes, you can provide an
+array of strings for the `toolName` and `commandPrefix` fields.
+
+**Example:**
+
+This single rule will apply to both the `write_file` and `replace` tools.
+
+```toml
+[[rule]]
+toolName = ["write_file", "replace"]
+decision = "ask_user"
+priority = 10
+```
+
+### Special syntax for `run_shell_command`
+
+To simplify writing policies for `run_shell_command`, you can use
+`commandPrefix` or `commandRegex` instead of the more complex `argsPattern`.
+
+- `commandPrefix`: Matches if the `command` argument starts with the given
+  string.
+- `commandRegex`: Matches if the `command` argument matches the given regular
+  expression.
+
+**Example:**
+
+This rule will ask for user confirmation before executing any `git` command.
+
+```toml
+[[rule]]
+toolName = "run_shell_command"
+commandPrefix = "git "
+decision = "ask_user"
+priority = 100
+```
+
+### Special syntax for MCP tools
+
+You can create rules that target tools from Model-hosting-protocol (MCP) servers
+using the `mcpName` field or a wildcard pattern.
+
+**1. Using `mcpName`**
+
+To target a specific tool from a specific server, combine `mcpName` and
+`toolName`.
+
+```toml
+# Allows the `search` tool on the `my-jira-server` MCP
+[[rule]]
+mcpName = "my-jira-server"
+toolName = "search"
+decision = "allow"
+priority = 200
+```
+
+**2. Using a wildcard**
+
+To create a rule that applies to _all_ tools on a specific MCP server, specify
+only the `mcpName`.
+
+```toml
+# Denies all tools from the `untrusted-server` MCP
+[[rule]]
+mcpName = "untrusted-server"
+decision = "deny"
+priority = 500
+```
+
+## Default policies
+
+The Cell CLI ships with a set of default policies to provide a safe
+out-of-the-box experience.
+
+- **Read-only tools** (like `read_file`, `glob`) are generally **allowed**.
+- **Agent delegation** (like `delegate_to_agent`) defaults to **`ask_user`** to
+  ensure remote agents can prompt for confirmation, but local sub-agent actions
+  are executed silently and checked individually.
+- **Write tools** (like `write_file`, `run_shell_command`) default to
+  **`ask_user`**.
+- In **`yolo`** mode, a high-priority rule allows all tools.
+- In **`autoEdit`** mode, rules allow certain write operations to happen without
+  prompting.