@fuzzle/opencode-accountant 0.0.12 → 0.0.13-next.1

package/docs/tools/classify-statements.md

@@ -0,0 +1,404 @@
+# classify-statements Tool
+
+The `classify-statements` tool organizes bank statement CSV files by automatically detecting their provider and currency, then moves them to the appropriate directories for import processing.
+
+This tool is **restricted to the accountant agent only**.
+
+## Arguments
+
+| Argument | Type | Default | Description                  |
+| -------- | ---- | ------- | ---------------------------- |
+| (none)   | -    | -       | This tool takes no arguments |
+
+## Output Format
+
+**Note on paths:** All file paths use `{paths.*}` variables configured in `config/import/providers.yaml`. Default values:
+
+- `{paths.import}` = `import/incoming`
+- `{paths.pending}` = `import/pending`
+- `{paths.unrecognized}` = `import/unrecognized`
+
+### Success - All Files Classified
+
+When all CSV files are successfully classified:
+
+```json
+{
+  "success": true,
+  "classified": [
+    {
+      "filename": "transactions-ubs-2026-02.csv",
+      "provider": "ubs",
+      "currency": "chf",
+      "targetPath": "{paths.pending}/ubs/chf/transactions-ubs-2026-02.csv"
+    },
+    {
+      "filename": "account-statement_2026-02.csv",
+      "provider": "revolut",
+      "currency": "eur",
+      "targetPath": "{paths.pending}/revolut/eur/account-statement_2026-02.csv"
+    }
+  ],
+  "unrecognized": [],
+  "summary": {
+    "total": 2,
+    "classified": 2,
+    "unrecognized": 0
+  }
+}
+```
+
+### Success - With Filename Renaming
+
+When provider config includes `renamePattern` with metadata extraction:
+
+```json
+{
+  "success": true,
+  "classified": [
+    {
+      "filename": "transactions-ubs-0235-90250546.csv",
+      "originalFilename": "export.csv",
+      "provider": "ubs",
+      "currency": "chf",
+      "targetPath": "{paths.pending}/ubs/chf/transactions-ubs-0235-90250546.csv"
+    }
+  ],
+  "unrecognized": [],
+  "summary": {
+    "total": 1,
+    "classified": 1,
+    "unrecognized": 0
+  }
+}
+```
+
+The `originalFilename` field appears when the file was renamed using metadata extraction.
+
+### Success - Some Files Unrecognized
+
+When some files cannot be classified:
+
+```json
+{
+  "success": true,
+  "classified": [
+    {
+      "filename": "transactions-ubs-2026-02.csv",
+      "provider": "ubs",
+      "currency": "chf",
+      "targetPath": "{paths.pending}/ubs/chf/transactions-ubs-2026-02.csv"
+    }
+  ],
+  "unrecognized": [
+    {
+      "filename": "mystery-bank.csv",
+      "targetPath": "{paths.unrecognized}/mystery-bank.csv"
+    }
+  ],
+  "summary": {
+    "total": 2,
+    "classified": 1,
+    "unrecognized": 1
+  }
+}
+```
+
+Unrecognized files are moved to `{paths.unrecognized}` for manual review.
+
+### Failure - File Collisions
+
+When target files already exist (prevents overwriting):
+
+```json
+{
+  "success": false,
+  "error": "Cannot classify: 1 file(s) would overwrite existing pending files.",
+  "collisions": [
+    {
+      "filename": "transactions.csv",
+      "existingPath": "{paths.pending}/ubs/chf/transactions.csv"
+    }
+  ],
+  "classified": [],
+  "unrecognized": []
+}
+```
+
+**Important:** The tool uses a two-pass approach (detect → check collisions → move) to prevent partial classification. If ANY collision is detected, NO files are moved.
+
+### Configuration Error
+
+When `config/import/providers.yaml` is missing or invalid:
+
+```json
+{
+  "success": false,
+  "error": "Failed to load configuration: config/import/providers.yaml not found",
+  "classified": [],
+  "unrecognized": []
+}
+```
+
+### Agent Restriction Error
+
+When called by the wrong agent:
+
+```json
+{
+  "success": false,
+  "error": "This tool is restricted to the accountant agent only.",
+  "hint": "Use: Task(subagent_type='accountant', prompt='classify statements')",
+  "caller": "main assistant",
+  "classified": [],
+  "unrecognized": []
+}
+```
+
+## Provider Detection
+
+The tool detects providers using rules defined in `config/import/providers.yaml`:
+
+### Detection Methods
+
+1. **Filename Pattern** (optional): Regex match against filename
+2. **CSV Header** (required): Exact match of CSV header row
+3. **Currency Field** (required): Which column contains the currency
+
+### Detection Example
+
+```yaml
+providers:
+  revolut:
+    detect:
+      - filenamePattern: '^account-statement_'
+        header: 'Type,Product,Started Date,Completed Date,Description,Amount,Fee,Currency,State,Balance'
+        currencyField: Currency
+    currencies:
+      CHF: chf
+      EUR: eur
+```
+
+Detection process:
+
+1. Check if filename matches `filenamePattern` (if specified)
+2. Read CSV and check if header matches exactly
+3. Determine currency from `currencyField` column
+4. Map raw currency value (e.g., "EUR") to normalized folder name (e.g., "eur")
+
+### Filename Renaming
+
+Providers can specify `renamePattern` with metadata extraction:
+
+```yaml
+ubs:
+  detect:
+    - header: 'Trade date,Trade time,...'
+      currencyField: Currency
+      skipRows: 9
+      delimiter: ';'
+      renamePattern: 'transactions-ubs-{account-number}.csv'
+      metadata:
+        - field: account-number
+          row: 0
+          column: 1
+          normalize: spaces-to-dashes
+```
+
+This extracts metadata from the CSV (e.g., account number from row 0, column 1) and uses it in the output filename.
+
+## Directory Structure
+
+```
+your-project/
+├── config/
+│   └── import/
+│       └── providers.yaml          # Defines all paths and detection rules
+├── {paths.import}/                 # Drop CSV files here (default: import/incoming)
+│   ├── bank1.csv
+│   └── bank2.csv
+│
+├── {paths.pending}/                # Classified files (default: import/pending)
+│   ├── <provider>/                 # e.g., revolut, ubs
+│   │   └── <currency>/             # e.g., chf, eur, usd, btc
+│   │       └── classified.csv
+│   ├── ubs/
+│   │   └── chf/
+│   │       └── transactions-ubs-0235-90250546.csv
+│   └── revolut/
+│       └── eur/
+│           └── account-statement_2026-02.csv
+│
+└── {paths.unrecognized}/           # Unclassified files (default: import/unrecognized)
+    └── mystery-bank.csv
+```
+
+## Typical Workflow
+
+### Scenario 1: Successful Classification
+
+1. Drop CSV files into `{paths.import}/` (e.g., `import/incoming/`)
+2. Run `classify-statements` tool (no arguments)
+3. Check output - all files classified successfully
+4. Files organized in `{paths.pending}/<provider>/<currency>/`
+5. Proceed to `import-statements` tool
+
+### Scenario 2: Handling Unrecognized Files
+
+1. Run `classify-statements` tool
+2. Review `unrecognized` array in output
+3. Check files in `{paths.unrecognized}/` directory
+4. Options to resolve:
+   - **Add provider config**: Update `config/import/providers.yaml` with detection rules
+   - **Manual classification**: Move file to correct `{paths.pending}/<provider>/<currency>/` directory
+   - **Investigate format**: Check if CSV format matches expected patterns
+5. Re-run tool after adding configuration
+
+### Scenario 3: Resolving Collisions
+
+1. Run `classify-statements` tool
+2. Tool reports collision - file would overwrite existing file
+3. Check `collisions` array for affected files
+4. Options to resolve:
+   - **Archive existing**: Move existing pending file to `{paths.done}/` if already processed
+   - **Rename**: Rename one of the conflicting files
+   - **Remove**: Delete duplicate file if confirmed redundant
+5. Re-run tool after resolving collision
+
+**Important:** No files are moved until ALL collisions are resolved. This prevents partial/inconsistent state.
+
+## Handling Unrecognized Files
+
+### What "Unrecognized" Means
+
+A file is unrecognized when:
+
+- Filename doesn't match any `filenamePattern` (if patterns are specified)
+- CSV header doesn't match any configured provider's `header`
+- CSV is malformed or has unexpected structure
+- Currency value doesn't map to configured currencies
+
+### Common Causes
+
+| Cause                     | Solution                                                                        |
+| ------------------------- | ------------------------------------------------------------------------------- |
+| New bank/provider         | Add provider config to `config/import/providers.yaml`                           |
+| Non-standard CSV format   | Check CSV structure; add detection rules with correct header/skipRows/delimiter |
+| Filename pattern mismatch | Update `filenamePattern` or remove it (header-only detection)                   |
+| Unknown currency          | Add currency mapping to provider's `currencies` section                         |
+| Metadata in header rows   | Use `skipRows` to skip non-CSV rows before header                               |
+| Wrong delimiter           | Specify `delimiter` (e.g., `";"` for semicolon-delimited)                       |
+
+### Adding Provider Detection
+
+Example: Adding a new bank called "SwissBank":
+
+```yaml
+providers:
+  swissbank:
+    detect:
+      - filenamePattern: '^swissbank-'
+        header: 'Date,Description,Amount,Balance,Currency'
+        currencyField: Currency
+    currencies:
+      CHF: chf
+      EUR: eur
+```
+
+After updating config, re-run `classify-statements` to classify previously unrecognized files.
+
+## Collision Safety
+
+The tool uses a **two-pass approach** to ensure atomic operations:
+
+### Two-Pass Process
+
+**Pass 1: Detection & Planning**
+
+- Scan all CSV files in `{paths.import}/`
+- Detect provider/currency for each file
+- Determine target path for each file
+- Build complete move plan
+
+**Pass 2: Collision Check**
+
+- Check if ANY target file already exists
+- If collisions found: abort with error (no files moved)
+- If no collisions: proceed to Pass 3
+
+**Pass 3: Move Files**
+
+- Execute all planned moves atomically
+- All files moved successfully or none at all
+
+### Why This Matters
+
+Without collision checking, partial classification could occur:
+
+- Some files moved, others fail mid-process
+- Inconsistent state requiring manual cleanup
+- Risk of lost data or confusion about what was processed
+
+With two-pass approach:
+
+- All-or-nothing operation
+- Easy to retry after fixing collisions
+- No partial/inconsistent states
+
+### Resolving Collisions
+
+Check the `collisions` array in the error output:
+
+```json
+"collisions": [
+  {
+    "filename": "transactions.csv",
+    "existingPath": "{paths.pending}/ubs/chf/transactions.csv"
+  }
+]
+```
+
+Then:
+
+1. Inspect existing file: `cat {paths.pending}/ubs/chf/transactions.csv`
+2. Determine if it's already processed (check if transactions were imported)
+3. If processed: Move to `{paths.done}/` or delete
+4. If not processed: Rename one of the files or merge manually
+5. Re-run `classify-statements`
+
+## Error Handling
+
+### Common Errors
+
+| Error               | Cause                                             | Solution                                                              |
+| ------------------- | ------------------------------------------------- | --------------------------------------------------------------------- |
+| File collision      | Target file already exists in pending directory   | Move existing file to done, rename, or delete; then re-run            |
+| Configuration error | Missing or invalid `config/import/providers.yaml` | Ensure config file exists with proper YAML syntax and required fields |
+| Agent restriction   | Called by wrong agent                             | Use `Task(subagent_type='accountant', prompt='classify statements')`  |
+| Permission error    | Cannot read/write directories                     | Check file permissions on import/pending/unrecognized directories     |
+| No CSV files found  | Import directory is empty                         | Add CSV files to `{paths.import}` directory first                     |
+| CSV parsing error   | Malformed CSV file                                | Check CSV structure; ensure proper delimiter and header row           |
+
+### Configuration File Required Fields
+
+Ensure `config/import/providers.yaml` contains:
+
+```yaml
+paths:
+  import: <path> # Required
+  pending: <path> # Required
+  done: <path> # Required (used by import-statements tool)
+  unrecognized: <path> # Required
+  rules: <path> # Required (used by import-statements tool)
+
+providers:
+  <provider-name>:
+    detect:
+      - header: <exact-csv-header> # Required
+        currencyField: <column-name> # Required
+        # Optional: filenamePattern, skipRows, delimiter, renamePattern, metadata
+    currencies:
+      <RAW-VALUE>: <normalized-folder> # Required (at least one)
+```
+
+Missing any required field will cause a configuration error.

package/docs/tools/import-statements.md

@@ -0,0 +1,305 @@
+# import-statements Tool
+
+The `import-statements` tool imports classified CSV bank statements into hledger using rules files. It operates in two modes:
+
+- **Check mode** (`checkOnly: true`, default): Validates transactions and reports any that cannot be categorized
+- **Import mode** (`checkOnly: false`): Imports validated transactions and moves processed files to the done directory
+
+## Year-Based Journal Routing
+
+Transactions are automatically routed to year-specific journal files based on transaction dates:
+
+- Transactions from 2025 → `ledger/2025.journal`
+- Transactions from 2026 → `ledger/2026.journal`
+
+**Automatic setup:**
+
+- If the year journal doesn't exist, it is created automatically
+- The include directive (`include ledger/YYYY.journal`) is added to `.hledger.journal` if not already present
+
+**Constraint:** Each CSV file must contain transactions from a single year. CSVs with transactions spanning multiple years are rejected during check mode with an error message listing the years found.
+
+## Arguments
+
+| Argument    | Type    | Default | Description                                 |
+| ----------- | ------- | ------- | ------------------------------------------- |
+| `provider`  | string  | -       | Filter by provider (e.g., `revolut`, `ubs`) |
+| `currency`  | string  | -       | Filter by currency (e.g., `chf`, `eur`)     |
+| `checkOnly` | boolean | `true`  | If true, only validate without importing    |
+
+## Output Format
+
+**Note on paths:** All file paths in the examples below use `{paths.*}` variables. These are configured in `config/import/providers.yaml`. Default values are:
+
+- `{paths.pending}` = `import/pending`
+- `{paths.done}` = `import/done`
+- `{paths.rules}` = `ledger/rules`
+
+### Check Mode - All Transactions Matched
+
+When all transactions have matching rules:
+
+```json
+{
+  "success": true,
+  "files": [
+    {
+      "csv": "{paths.pending}/ubs/chf/transactions-ubs-0235-90250546.0.csv",
+      "rulesFile": "{paths.rules}/ubs-0235-90250546.0.rules",
+      "transactions": 25,
+      "unknownPostings": [],
+      "transactionYear": 2026
+    }
+  ],
+  "summary": {
+    "filesProcessed": 1,
+    "totalTransactions": 25,
+    "matched": 25,
+    "unknown": 0
+  },
+  "message": "All transactions matched. Ready to import with checkOnly: false"
+}
+```
+
+### Check Mode - Unknown Postings Found
+
+When transactions don't match any `if` pattern in the rules file, the tool returns the full CSV row data for each unknown posting to provide context for classification:
+
+```json
+{
+  "success": false,
+  "files": [
+    {
+      "csv": "{paths.pending}/ubs/chf/transactions-ubs-0235-90250546.0.csv",
+      "rulesFile": "{paths.rules}/ubs-0235-90250546.0.rules",
+      "transactions": 25,
+      "unknownPostings": [
+        {
+          "date": "2026-01-16",
+          "description": "Connor, John",
+          "amount": "CHF95.25",
+          "account": "income:unknown",
+          "csvRow": {
+            "trade_date": "2026-01-16",
+            "trade_time": "",
+            "booking_date": "2026-01-16",
+            "value_date": "2026-01-16",
+            "currency": "CHF",
+            "debit": "",
+            "credit": "95.25",
+            "individual_amount": "",
+            "balance": "4746.23",
+            "transaction_no": "ABC123",
+            "description1": "Connor, John",
+            "description2": "Twint deposit",
+            "description3": "Ref: TW-12345",
+            "footnotes": ""
+          }
+        },
+        {
+          "date": "2026-01-30",
+          "description": "Balance closing of service prices",
+          "amount": "CHF-10.00",
+          "account": "expenses:unknown",
+          "csvRow": {
+            "trade_date": "2026-01-30",
+            "trade_time": "",
+            "booking_date": "2026-01-30",
+            "value_date": "2026-01-30",
+            "currency": "CHF",
+            "debit": "10.00",
+            "credit": "",
+            "individual_amount": "",
+            "balance": "2364.69",
+            "transaction_no": "DEF456",
+            "description1": "Balance closing of service prices",
+            "description2": "",
+            "description3": "",
+            "footnotes": ""
+          }
+        }
+      ]
+    }
+  ],
+  "summary": {
+    "filesProcessed": 1,
+    "totalTransactions": 25,
+    "matched": 23,
+    "unknown": 2
+  }
+}
+```
+
+### Check Mode - Missing Rules File
+
+When a CSV file has no matching rules file:
+
+```json
+{
+  "success": false,
+  "files": [
+    {
+      "csv": "{paths.pending}/ubs/chf/transactions.csv",
+      "error": "No matching rules file found. Create a rules file with 'source' directive pointing to this CSV."
+    }
+  ],
+  "summary": {
+    "filesProcessed": 1,
+    "filesWithoutRules": 1
+  }
+}
+```
+
+### Import Mode - Success
+
+When importing with all transactions matched:
+
+```json
+{
+  "success": true,
+  "files": [
+    {
+      "csv": "{paths.pending}/ubs/chf/transactions.csv",
+      "rulesFile": "{paths.rules}/ubs.rules",
+      "imported": true,
+      "movedTo": "{paths.done}/ubs/chf/transactions.csv"
+    }
+  ],
+  "summary": {
+    "filesProcessed": 1,
+    "filesImported": 1,
+    "totalTransactions": 25
+  },
+  "message": "Successfully imported 1 file(s)"
+}
+```
+
+### Import Mode - Blocked by Unknown Postings
+
+Import mode runs a check first and aborts if any unknowns exist:
+
+```json
+{
+  "success": false,
+  "error": "Cannot import: 2 transactions have unknown accounts. Run with checkOnly: true to see details and add rules.",
+  "hint": "Run with checkOnly: true first to identify and fix unknown postings"
+}
+```
+
+## Unknown Posting Types
+
+hledger assigns transactions to `income:unknown` or `expenses:unknown` based on the direction:
+
+| Transaction Type           | Account Assigned   |
+| -------------------------- | ------------------ |
+| Money coming in (positive) | `income:unknown`   |
+| Money going out (negative) | `expenses:unknown` |
+
+## Fixing Unknown Postings
+
+When the tool reports unknown postings, the `csvRow` field contains all available data from the original CSV to help determine the correct account. This includes additional description fields, transaction references, and other metadata that may help with classification.
+
+Add `if` rules to the appropriate rules file based on the posting details:
+
+```
+# Example: Categorize a friend's reimbursement
+# (csvRow showed description2: "Twint deposit" confirming it's a payment app transfer)
+if Connor, John
+    account1 income:reimbursements
+
+# Example: Categorize bank service charges
+if Balance closing of service prices
+    account1 expenses:fees:bank
+```
+
+Then run the tool again with `checkOnly: true` to verify the rules work.
+
+### CSV Row Field Names
+
+The `csvRow` object uses field names from the `fields` directive in the rules file. Common fields include:
+
+| Field            | Description                                             |
+| ---------------- | ------------------------------------------------------- |
+| `trade_date`     | When the transaction occurred                           |
+| `booking_date`   | When it was booked                                      |
+| `description1`   | Primary description                                     |
+| `description2`   | Secondary description (often useful for classification) |
+| `description3`   | Additional reference information                        |
+| `transaction_no` | Unique transaction identifier                           |
+| `debit`          | Debit amount (money out)                                |
+| `credit`         | Credit amount (money in)                                |
+
+The exact field names depend on your rules file configuration.
+
+## Error Handling
+
+### hledger Errors
+
+If hledger fails to parse a CSV or rules file:
+
+```json
+{
+  "success": false,
+  "files": [
+    {
+      "csv": "{paths.pending}/ubs/chf/transactions.csv",
+      "rulesFile": "{paths.rules}/ubs.rules",
+      "error": "hledger error: Parse error at line 5: invalid date format"
+    }
+  ],
+  "summary": {
+    "filesProcessed": 1,
+    "filesWithErrors": 1
+  }
+}
+```
+
+### Configuration Errors
+
+If the config file is missing or invalid:
+
+```json
+{
+  "success": false,
+  "error": "Failed to load configuration: Configuration file not found: config/import/providers.yaml",
+  "hint": "Ensure config/import/providers.yaml exists with a 'rules' path configured"
+}
+```
+
+### Multi-Year CSV Error
+
+If a CSV contains transactions from multiple years:
+
+```json
+{
+  "success": false,
+  "files": [
+    {
+      "csv": "{paths.pending}/ubs/chf/transactions.csv",
+      "rulesFile": "{paths.rules}/ubs.rules",
+      "totalTransactions": 10,
+      "matchedTransactions": 10,
+      "unknownPostings": [],
+      "error": "CSV contains transactions from multiple years (2025, 2026). Split the CSV by year before importing."
+    }
+  ],
+  "summary": {
+    "filesProcessed": 1,
+    "filesWithErrors": 1,
+    "totalTransactions": 0,
+    "matched": 0,
+    "unknown": 0
+  }
+}
+```
+
+### Missing Main Journal
+
+If `.hledger.journal` doesn't exist when attempting import:
+
+```json
+{
+  "success": false,
+  "error": ".hledger.journal not found at /path/to/.hledger.journal. Create it first with appropriate includes."
+}
+```