codymaster 4.1.3 → 4.3.0

package/skills/cm-frappe-agent/resources/rest-api-patterns.md

@@ -0,0 +1,182 @@
+# Frappe REST API Patterns
+
+> Reference for interacting with remote Frappe/ERPNext sites via REST API.
+
+## Authentication
+
+All requests require API key header:
+```
+Authorization: token <api_key>:<api_secret>
+```
+
+### Test Authentication
+```bash
+curl -sS "https://{site}/api/method/frappe.auth.get_logged_user" \
+  -H "Authorization: token {key}:{secret}"
+```
+
+### Security Rules
+- **NEVER** reveal API keys or secrets in output
+- **NEVER** expose internal file paths or server configs
+- **ALWAYS** confirm before destructive operations (DELETE, bulk updates)
+- **NEVER** fabricate document data or fake API responses
+- Store credentials in `.env` file, never in code
+
+---
+
+## CRUD Operations
+
+### Get Single Document
+```bash
+curl -sS "https://{site}/api/resource/{DocType}/{name}" \
+  -H "Authorization: token {key}:{secret}"
+```
+
+### List Documents (with filters)
+```bash
+curl -sS "https://{site}/api/resource/{DocType}" \
+  -G --data-urlencode 'filters=[["field","=","value"]]' \
+  --data-urlencode 'fields=["name","field1","field2"]' \
+  --data-urlencode 'limit_page_length=20' \
+  -H "Authorization: token {key}:{secret}"
+```
+
+**Fallback for blocked fields** (e.g., `is_single`, `istable`):
+```bash
+curl -sS "https://{site}/api/method/frappe.client.get_list" \
+  -H "Authorization: token {key}:{secret}" \
+  -H "Content-Type: application/json" \
+  -d '{"doctype":"DocType","fields":["name","module"],"limit_page_length":0}'
+```
+
+### Create Document
+```bash
+curl -sS -X POST "https://{site}/api/resource/{DocType}" \
+  -H "Authorization: token {key}:{secret}" \
+  -H "Content-Type: application/json" \
+  -d '{"field1":"value1","field2":"value2"}'
+```
+
+### Update Document
+```bash
+curl -sS -X PUT "https://{site}/api/resource/{DocType}/{name}" \
+  -H "Authorization: token {key}:{secret}" \
+  -H "Content-Type: application/json" \
+  -d '{"field_to_update":"new_value"}'
+```
+
+### Delete Document
+```bash
+curl -sS -X DELETE "https://{site}/api/resource/{DocType}/{name}" \
+  -H "Authorization: token {key}:{secret}"
+```
+
+---
+
+## RPC (Server Method Calls)
+
+### Run Server Method
+```bash
+curl -sS "https://{site}/api/method/{dotted.method.path}" \
+  -H "Authorization: token {key}:{secret}" \
+  -H "Content-Type: application/json" \
+  -d '{"arg1":"value1"}'
+```
+
+### Get Value (Shortcut)
+```bash
+curl -sS "https://{site}/api/method/frappe.client.get_value" \
+  -H "Authorization: token {key}:{secret}" \
+  -H "Content-Type: application/json" \
+  -d '{"doctype":"Employee","filters":{"user_id":"user@example.com"},"fieldname":["name","employee_name"]}'
+```
+
+### Get Count
+```bash
+curl -sS "https://{site}/api/method/frappe.client.get_count" \
+  -H "Authorization: token {key}:{secret}" \
+  -H "Content-Type: application/json" \
+  -d '{"doctype":"Employee","filters":{"status":"Active"}}'
+```
+
+### Run Report
+```bash
+curl -sS "https://{site}/api/method/frappe.desk.query_report.run" \
+  -H "Authorization: token {key}:{secret}" \
+  -H "Content-Type: application/json" \
+  -d '{"report_name":"Employee Leave Balance","filters":{"company":"My Company"}}'
+```
+
+---
+
+## Batch Operations
+
+### Bulk Update (via RPC)
+```bash
+curl -sS -X POST "https://{site}/api/method/frappe.client.bulk_update" \
+  -H "Authorization: token {key}:{secret}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "docs": [
+      {"doctype":"ToDo","name":"TODO-001","status":"Closed"},
+      {"doctype":"ToDo","name":"TODO-002","status":"Closed"}
+    ]
+  }'
+```
+
+### File Upload
+```bash
+curl -sS -X POST "https://{site}/api/method/upload_file" \
+  -H "Authorization: token {key}:{secret}" \
+  -F "file=@/path/to/file.pdf" \
+  -F "doctype=File" \
+  -F "docname=Home"
+```
+
+---
+
+## Web Form Operations
+
+### Get Web Form Config
+```bash
+curl -sS "https://{site}/api/resource/Web%20Form" \
+  -G --data-urlencode 'filters=[["route","=","my-form"]]' \
+  --data-urlencode 'fields=["name","title","custom_css","client_script"]' \
+  -H "Authorization: token {key}:{secret}"
+```
+
+### Update Web Form
+```bash
+curl -sS -X PUT "https://{site}/api/resource/Web%20Form/{name}" \
+  -H "Authorization: token {key}:{secret}" \
+  -H "Content-Type: application/json" \
+  -d '{"custom_css":"...","client_script":"..."}'
+```
+
+---
+
+## Response Format
+
+| Type | Format |
+|------|--------|
+| Success | `{"data": {...}}` or `{"message": ...}` |
+| Error | `{"exception": "...", "exc_type": "...", "exc": "..."}` |
+
+---
+
+## Error Handling Patterns
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| `DataError: Field not allowed` | Blocked field in resource API | Use `frappe.client.get_list` RPC fallback |
+| `403 Forbidden` | Insufficient API key permissions | Check User > API Access settings |
+| `404 Not Found` | DocType/document doesn't exist | Verify name spelling, check module |
+| `417 Expectation Failed` | Validation error on create/update | Check required fields, valid values |
+| `409 Conflict` | Duplicate name or concurrent edit | Use unique naming, add `If-Match` header |
+
+## URL Encoding Tips
+
+- Use `--data-urlencode` for filters/fields with special characters
+- DocType names with Unicode: URL-encode the name (e.g., `Web%20Form`)
+- Use `-G` flag with `--data-urlencode` for GET requests with encoded params
+- Always use `jq` for JSON parsing: `| jq '.data'`

package/skills/cm-frappe-agent/resources/scaffold_checklist.md

@@ -0,0 +1,82 @@
+# Frappe App Scaffold Checklist
+
+Use this checklist when creating a new Frappe custom app from scratch.
+
+## Phase 1: Foundation
+
+- [ ] `bench new-app my_app` — scaffold the app
+- [ ] Create folder structure:
+  ```
+  mkdir -p my_app/{engines,api,tasks,setup,tests,scripts,fixtures}
+  mkdir -p my_app/public/js
+  touch my_app/{engines,api,tasks,setup,tests}/__init__.py
+  ```
+- [ ] `bench --site mysite install-app my_app`
+- [ ] `bench --site mysite set-config developer_mode 1`
+
+## Phase 2: Data Model
+
+- [ ] Design DocTypes on paper first (fields, links, workflows)
+- [ ] Identify which DocTypes are Submittable (need approval flow)
+- [ ] Identify Master DocTypes (config, types, rules)
+- [ ] Create DocTypes via Frappe UI or JSON files
+- [ ] Add `module` property to all DocTypes
+
+## Phase 3: Hooks & Setup
+
+- [ ] Configure `hooks.py`:
+  - [ ] `app_name`, `app_title`, `required_apps`
+  - [ ] `app_include_js` for shared client JS
+  - [ ] `after_install`, `after_migrate`
+  - [ ] `doc_events` for submit/cancel/insert hooks
+  - [ ] `fixtures` for Roles, Custom Fields, Workflows
+  - [ ] `scheduler_events` for daily/weekly/monthly tasks
+  - [ ] `permission_query_conditions` for row-level security
+- [ ] Write `setup/install.py` (idempotent role/field/state creation)
+
+## Phase 4: Business Logic
+
+- [ ] Create `engines/` with pure-logic functions (no Frappe imports)
+- [ ] Create config cascade functions (with Frappe DB)
+- [ ] Create main pipeline function (aggregates everything)
+- [ ] Write doc-event hook functions (on_submit, on_cancel, after_insert)
+
+## Phase 5: APIs
+
+- [ ] Create external API (`api/external.py`) for webhook/integration
+- [ ] Create internal API (`api/internal.py`) for UI-facing calls
+- [ ] Add permission checks (`frappe.has_permission()`)
+- [ ] Add permission query functions for row-level security
+
+## Phase 6: Scheduler Tasks
+
+- [ ] `tasks/daily.py` — aggregation, reminders
+- [ ] `tasks/weekly.py` — weekly calculations
+- [ ] `tasks/monthly.py` — monthly calculations, benefit checks
+- [ ] Register all in `hooks.py` → `scheduler_events`
+
+## Phase 7: Client Side
+
+- [ ] Create `public/js/my_app.js` with shared utilities
+- [ ] Write DocType `.js` files (refresh, field triggers, custom buttons)
+- [ ] Add list view settings with workflow color indicators
+
+## Phase 8: Reports
+
+- [ ] Create Script Reports with `execute()`, `get_columns()`, `get_data()`
+- [ ] Add charts via `get_chart()` (bar, line, pie)
+- [ ] Add summary cards via `get_report_summary()`
+- [ ] Create `.js` filter definitions
+
+## Phase 9: Testing
+
+- [ ] Write standalone tests for pure-logic functions
+- [ ] Run: `python -m pytest my_app/tests/ -v`
+
+## Phase 10: Deploy
+
+- [ ] `bench --site mysite migrate`
+- [ ] `bench build --app my_app`
+- [ ] Export fixtures: `bench --site mysite export-fixtures --app my_app`
+- [ ] Test on staging before production
+- [ ] Push to git

package/skills/cm-frappe-agent/resources/upgrade_patterns.md

@@ -0,0 +1,113 @@
+# Frappe App Upgrade & Migration Patterns
+
+## Adding New DocTypes to Existing App
+
+1. Create DocType JSON via Frappe UI
+2. Write controller (`.py`) following existing patterns
+3. Add doc_events in `hooks.py` if needed
+4. Add to fixtures list if it has workflow
+5. Update `setup/install.py` if new roles/states needed
+6. `bench --site mysite migrate && bench build --app my_app`
+
+## Adding Fields to Existing DocType
+
+```bash
+# 1. Modify DocType JSON (via UI or directly)
+# 2. Run migration
+bench --site mysite migrate
+
+# 3. If adding Custom Fields to core DocTypes (Employee, etc.)
+# Add to setup/install.py → _create_custom_fields()
+# Add to hooks.py → fixtures → Custom Field filter
+```
+
+## Upgrading Engine Logic
+
+1. Modify pure-logic functions in `engines/`
+2. Update tests to cover new behavior
+3. Run `python -m pytest my_app/tests/ -v`
+4. No migration needed — pure Python changes
+
+## Adding New API Endpoint
+
+```python
+# 1. Add function in api/my_module.py
+@frappe.whitelist()
+def new_endpoint(param1, param2):
+    if not frappe.has_permission("My DocType", "read"):
+        frappe.throw(_("Permission denied"), frappe.PermissionError)
+    return engine.process(param1, param2)
+```
+
+No hooks.py changes needed — Frappe auto-discovers whitelisted methods.
+
+## Adding New Scheduler Task
+
+```python
+# 1. Add function in tasks/daily.py (or new file)
+def my_new_task():
+    # ... logic
+    frappe.db.commit()
+
+# 2. Register in hooks.py
+scheduler_events = {
+    "daily": [
+        "my_app.tasks.daily.my_new_task",  # Add here
+    ],
+}
+```
+
+## Changing Workflow States
+
+⚠️ **CONFIRM before doing this — affects existing records!**
+
+1. Update Workflow JSON
+2. Add new states to `setup/install.py` → `_create_workflow_states()`
+3. Add new states to `hooks.py` → fixtures → Workflow State filter
+4. `bench --site mysite migrate`
+5. Check existing records are not in broken state
+
+## Adding New Reports
+
+```bash
+# 1. Create report folder
+mkdir -p my_app/my_app/report/new_report/
+touch my_app/my_app/report/new_report/__init__.py
+
+# 2. Create report files
+# new_report.py — execute(), get_columns(), get_data(), get_chart()
+# new_report.js — filters definition
+# new_report.json — report metadata
+
+# 3. Build
+bench build --app my_app
+```
+
+## Data Migration Script
+
+```python
+"""
+scripts/migrate_data.py
+Run: bench --site mysite execute my_app.scripts.migrate_data.run
+
+⚠️ ALWAYS backup before running migration scripts!
+"""
+import frappe
+
+def run():
+    frappe.logger("my_app").info("Starting data migration...")
+    
+    # Process in batches of 100
+    records = frappe.get_all("Old DocType", fields=["*"], limit=100)
+    for rec in records:
+        try:
+            frappe.get_doc({
+                "doctype": "New DocType",
+                "field1": rec.old_field,
+            }).insert(ignore_permissions=True)
+        except Exception as exc:
+            frappe.log_error(str(exc), "Migration error")
+    
+    frappe.db.commit()
+    frappe.logger("my_app").info("Migration complete")
+```

package/skills/cm-frappe-agent/resources/web-form-patterns.md

@@ -0,0 +1,252 @@
+# Web Form Development Patterns
+
+> Patterns for building Frappe Web Forms (portal pages) — client scripts, CSS, and gotchas.
+
+---
+
+## Key Differences: Web Form vs Desk Form
+
+| Aspect | Desk Form (`.js`) | Web Form (Client Script) |
+|--------|-------------------|--------------------------|
+| API | `frappe.ui.form.Form` | `frappe.web_form` (extends FieldGroup) |
+| Set values | `frm.set_value()` | `frappe.web_form.set_values({...})` ← **PLURAL** |
+| Ready event | `refresh(frm)` | `frappe.ready(function() { ... })` |
+| Context | `cur_frm` | `frappe.web_form` |
+| Field access | `frm.fields_dict.fieldname` | `frappe.web_form.fields_dict.fieldname` |
+
+> [!WARNING]
+> The most common mistake: using `set_value()` (singular) instead of `set_values()` (plural) in Web Forms. WebForm extends FieldGroup, NOT Form.
+
+---
+
+## Client Script Patterns
+
+### Basic Structure
+```javascript
+frappe.ready(function() {
+    // All Web Form code goes inside frappe.ready()
+    // This ensures the form is fully loaded before running
+
+    // Set initial values
+    frappe.web_form.set_values({
+        date: frappe.datetime.get_today(),
+        status: 'Open'
+    });
+
+    // Listen for field changes
+    frappe.web_form.on('change', function(field, value) {
+        if (field === 'category') {
+            handle_category_change(value);
+        }
+    });
+});
+```
+
+### Set Read-Only Fields
+```javascript
+frappe.ready(function() {
+    // Read-only fields CAN still be set via set_values()
+    frappe.web_form.set_values({
+        employee_name: frappe.session.user_fullname,
+        submit_date: frappe.datetime.get_today()
+    });
+});
+```
+
+### Conditional Field Visibility
+```javascript
+frappe.ready(function() {
+    frappe.web_form.on('change', function(field, value) {
+        if (field === 'request_type') {
+            // Show/hide fields based on selection
+            let show_urgency = (value === 'Emergency');
+            frappe.web_form.fields_dict.urgency_level.$wrapper.toggle(show_urgency);
+            frappe.web_form.fields_dict.emergency_contact.$wrapper.toggle(show_urgency);
+        }
+    });
+});
+```
+
+### Custom Validation Before Submit
+```javascript
+frappe.ready(function() {
+    frappe.web_form.validate = function() {
+        let values = frappe.web_form.get_values();
+
+        if (!values.email || !values.email.includes('@')) {
+            frappe.msgprint(__('Please enter a valid email address'));
+            return false; // Prevents submission
+        }
+
+        if (values.start_date > values.end_date) {
+            frappe.msgprint(__('End date must be after start date'));
+            return false;
+        }
+
+        return true; // Allow submission
+    };
+});
+```
+
+### Call Server Methods
+```javascript
+frappe.ready(function() {
+    frappe.web_form.on('change', function(field, value) {
+        if (field === 'employee_id') {
+            frappe.call({
+                method: 'my_app.api.get_employee_details',
+                args: { employee_id: value },
+                callback: function(r) {
+                    if (r.message) {
+                        frappe.web_form.set_values({
+                            employee_name: r.message.employee_name,
+                            department: r.message.department
+                        });
+                    }
+                }
+            });
+        }
+    });
+});
+```
+
+### After Load / After Save
+```javascript
+frappe.ready(function() {
+    // After form loads
+    frappe.web_form.after_load = function() {
+        console.log('Form loaded with data:', frappe.web_form.doc);
+    };
+
+    // After form saves
+    frappe.web_form.after_save = function() {
+        frappe.msgprint(__('Thank you! Your submission has been recorded.'));
+    };
+});
+```
+
+---
+
+## Custom CSS Patterns
+
+### Professional Form Styling
+```css
+/* Card-style form container */
+.web-form-container {
+    max-width: 800px;
+    margin: 0 auto;
+    padding: 2rem;
+}
+
+/* Section headings */
+.web-form-container .section-head {
+    font-size: 1.25rem;
+    font-weight: 600;
+    color: var(--text-color);
+    border-bottom: 2px solid var(--primary);
+    padding-bottom: 0.5rem;
+    margin-bottom: 1.5rem;
+}
+
+/* Success message styling */
+.web-form-container .success-page {
+    text-align: center;
+    padding: 3rem 1rem;
+}
+
+/* Hide specific fields */
+.frappe-control[data-fieldname="internal_notes"] {
+    display: none;
+}
+
+/* Custom button styling */
+.web-form-container .btn-primary-dark {
+    background: var(--primary);
+    border: none;
+    padding: 0.75rem 2rem;
+    font-size: 1rem;
+    border-radius: 8px;
+}
+```
+
+### Responsive Layout
+```css
+@media (max-width: 768px) {
+    .web-form-container {
+        padding: 1rem;
+    }
+
+    .web-form-container .form-group {
+        margin-bottom: 1rem;
+    }
+}
+```
+
+---
+
+## Common Gotchas
+
+### 1. Timing Issues
+```javascript
+// ❌ WRONG: Code runs before form is ready
+frappe.web_form.set_values({ field: 'value' });
+
+// ✅ CORRECT: Always wrap in frappe.ready()
+frappe.ready(function() {
+    frappe.web_form.set_values({ field: 'value' });
+});
+```
+
+### 2. set_value vs set_values
+```javascript
+// ❌ WRONG: set_value doesn't exist on WebForm
+frappe.web_form.set_value('field', 'value');
+
+// ✅ CORRECT: Use set_values (plural) with object
+frappe.web_form.set_values({ field: 'value' });
+```
+
+### 3. Guest Access
+```javascript
+// For forms allowing guest submissions, check login state:
+frappe.ready(function() {
+    if (frappe.session.user === 'Guest') {
+        // Show limited fields
+        frappe.web_form.fields_dict.internal_field.$wrapper.hide();
+    }
+});
+```
+
+### 4. File Uploads in Web Forms
+```javascript
+// Web Forms handle file uploads differently
+// Use the Attach field type in DocType, Web Form will auto-create upload widget
+// For custom upload handling:
+frappe.ready(function() {
+    frappe.web_form.on('change', function(field, value) {
+        if (field === 'attachment' && value) {
+            console.log('File uploaded:', value);
+        }
+    });
+});
+```
+
+---
+
+## Managing Web Forms via REST API
+
+### Get Web Form Configuration
+```bash
+curl -sS "https://{site}/api/resource/Web%20Form" \
+  -G --data-urlencode 'filters=[["route","=","my-form"]]' \
+  --data-urlencode 'fields=["name","title","custom_css","client_script"]' \
+  -H "Authorization: token {key}:{secret}"
+```
+
+### Update Web Form Client Script
+```bash
+curl -sS -X PUT "https://{site}/api/resource/Web%20Form/{name}" \
+  -H "Authorization: token {key}:{secret}" \
+  -H "Content-Type: application/json" \
+  -d '{"client_script":"frappe.ready(function() { ... })"}'
+```