codymaster 4.6.0 → 5.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (161) hide show
  1. package/CHANGELOG.md +74 -8
  2. package/README.md +192 -95
  3. package/dist/advisory-handoff.js +89 -0
  4. package/dist/advisory-report.js +105 -0
  5. package/dist/browse-server.js +251 -0
  6. package/dist/cli/command-registry.js +34 -0
  7. package/dist/cli/commands/agent.js +120 -0
  8. package/dist/cli/commands/bench.js +69 -0
  9. package/dist/cli/commands/brain.js +108 -0
  10. package/dist/cli/commands/dashboard.js +93 -0
  11. package/dist/cli/commands/design-studio.js +111 -0
  12. package/dist/cli/commands/distro.js +25 -0
  13. package/dist/cli/commands/engineering.js +596 -0
  14. package/dist/cli/commands/evolve.js +123 -0
  15. package/dist/cli/commands/mcp-serve.js +104 -0
  16. package/dist/cli/commands/project.js +324 -0
  17. package/dist/cli/commands/skill-chain.js +269 -0
  18. package/dist/cli/commands/system.js +89 -0
  19. package/dist/cli/commands/task.js +254 -0
  20. package/dist/cli/update-check.js +83 -0
  21. package/dist/cm-config.js +92 -0
  22. package/dist/cm-suggest.js +77 -0
  23. package/dist/codybench/judges/automated.js +31 -0
  24. package/dist/codybench/runners/claude-code.js +32 -0
  25. package/dist/codybench/suites/memory-retention.js +85 -0
  26. package/dist/codybench/suites/tdd-regression.js +35 -0
  27. package/dist/codybench/suites/token-efficiency.js +55 -0
  28. package/dist/codybench/types.js +2 -0
  29. package/dist/context-db.js +157 -0
  30. package/dist/continuity.js +2 -6
  31. package/dist/distro-validate.js +54 -0
  32. package/dist/execution-analyzer.js +138 -0
  33. package/dist/guardian-core.js +74 -0
  34. package/dist/index.js +36 -2759
  35. package/dist/indexer/skills-lib.js +533 -0
  36. package/dist/indexer/skills-map.js +1374 -0
  37. package/dist/indexer/skills.js +16 -0
  38. package/dist/learning-promoter.js +246 -0
  39. package/dist/mcp-context-server.js +289 -1
  40. package/dist/mcp-skills-tools.js +81 -0
  41. package/dist/retro-summary.js +70 -0
  42. package/dist/second-opinion-providers.js +79 -0
  43. package/dist/skill-chain.js +63 -1
  44. package/dist/skill-evolver.js +456 -0
  45. package/dist/skill-execution-cache.js +254 -0
  46. package/dist/smart-brain-router.js +184 -0
  47. package/dist/sprint-pipeline.js +228 -0
  48. package/dist/storage-backend.js +14 -67
  49. package/dist/token-budget.js +88 -0
  50. package/dist/utils/cli-utils.js +76 -0
  51. package/dist/utils/skill-utils.js +32 -0
  52. package/package.json +17 -7
  53. package/scripts/build-skills.mjs +51 -0
  54. package/scripts/gate-0-repo-hygiene.js +75 -0
  55. package/scripts/postinstall.js +34 -28
  56. package/scripts/security-scan.js +1 -1
  57. package/scripts/validate-skills.mjs +42 -0
  58. package/skills/CLAUDE.md +2 -7
  59. package/skills/_shared/helpers.md +2 -8
  60. package/skills/cm-ads-tracker/SKILL.md +3 -6
  61. package/skills/cm-browse/SKILL.md +34 -0
  62. package/skills/cm-conductor-worktrees/SKILL.md +28 -0
  63. package/skills/cm-content-factory/SKILL.md +1 -1
  64. package/skills/cm-content-factory/landing/docs/content/changelog.md +36 -0
  65. package/skills/cm-content-factory/landing/docs/content/deployment.md +46 -0
  66. package/skills/cm-content-factory/landing/docs/content/execution-flow.md +67 -0
  67. package/skills/cm-content-factory/landing/docs/content/memory-system.md +38 -0
  68. package/skills/cm-content-factory/landing/docs/content/openspace.md +27 -0
  69. package/skills/cm-content-factory/landing/docs/content/use-cases.md +26 -0
  70. package/skills/cm-content-factory/landing/docs/content/v5-intro.md +28 -0
  71. package/skills/cm-content-factory/landing/docs/index.html +240 -0
  72. package/skills/cm-content-factory/landing/index.html +100 -100
  73. package/skills/cm-content-factory/landing/script.js +42 -0
  74. package/skills/cm-content-factory/landing/translations.js +400 -400
  75. package/skills/cm-continuity/SKILL.md +32 -33
  76. package/skills/cm-design-studio/SKILL.md +34 -0
  77. package/skills/cm-ecosystem-roadmap/SKILL.md +15 -0
  78. package/skills/cm-engineering-meta/SKILL.md +73 -0
  79. package/skills/cm-growth-hacking/SKILL.md +1 -12
  80. package/skills/cm-guardian-runtime/SKILL.md +26 -0
  81. package/skills/cm-mcp-engineering/SKILL.md +22 -0
  82. package/skills/cm-notebooklm/SKILL.md +1 -17
  83. package/skills/cm-post-deploy-canary/SKILL.md +22 -0
  84. package/skills/cm-project-bootstrap/SKILL.md +11 -0
  85. package/skills/cm-qa-visual-cli/SKILL.md +22 -0
  86. package/skills/cm-retro-cli/SKILL.md +23 -0
  87. package/skills/cm-second-opinion-cli/SKILL.md +23 -0
  88. package/skills/cm-secret-shield/SKILL.md +2 -2
  89. package/skills/cm-security-gate/SKILL.md +1 -0
  90. package/skills/cm-skill-chain/SKILL.md +25 -4
  91. package/skills/cm-skill-evolution/SKILL.md +83 -0
  92. package/skills/cm-skill-health/SKILL.md +83 -0
  93. package/skills/cm-skill-index/SKILL.md +11 -3
  94. package/skills/cm-skill-search/SKILL.md +49 -0
  95. package/skills/cm-skill-share/SKILL.md +58 -0
  96. package/skills/cm-sprint-bus/SKILL.md +33 -0
  97. package/skills/cm-start/SKILL.md +0 -10
  98. package/skills/cm-tdd/SKILL.md +59 -72
  99. package/skills/profiles/README.md +21 -0
  100. package/skills/profiles/core.txt +23 -0
  101. package/skills/profiles/design.txt +6 -0
  102. package/skills/profiles/full.txt +62 -0
  103. package/skills/profiles/growth.txt +10 -0
  104. package/skills/profiles/knowledge.txt +7 -0
  105. package/install.sh +0 -901
  106. package/scripts/test-gemini.js +0 -13
  107. package/skills/cm-frappe-agent/SKILL.md +0 -134
  108. package/skills/cm-frappe-agent/agents/doctype-architect.md +0 -596
  109. package/skills/cm-frappe-agent/agents/erpnext-customizer.md +0 -643
  110. package/skills/cm-frappe-agent/agents/frappe-backend.md +0 -814
  111. package/skills/cm-frappe-agent/agents/frappe-custom-frontend.md +0 -557
  112. package/skills/cm-frappe-agent/agents/frappe-debugger.md +0 -625
  113. package/skills/cm-frappe-agent/agents/frappe-fixer.md +0 -275
  114. package/skills/cm-frappe-agent/agents/frappe-frontend.md +0 -660
  115. package/skills/cm-frappe-agent/agents/frappe-installer.md +0 -158
  116. package/skills/cm-frappe-agent/agents/frappe-performance.md +0 -307
  117. package/skills/cm-frappe-agent/agents/frappe-planner.md +0 -419
  118. package/skills/cm-frappe-agent/agents/frappe-remote-ops.md +0 -153
  119. package/skills/cm-frappe-agent/agents/github-workflow.md +0 -286
  120. package/skills/cm-frappe-agent/commands/frappe-app.md +0 -351
  121. package/skills/cm-frappe-agent/commands/frappe-backend.md +0 -162
  122. package/skills/cm-frappe-agent/commands/frappe-bench.md +0 -254
  123. package/skills/cm-frappe-agent/commands/frappe-debug.md +0 -263
  124. package/skills/cm-frappe-agent/commands/frappe-doctype-create.md +0 -272
  125. package/skills/cm-frappe-agent/commands/frappe-doctype-field.md +0 -310
  126. package/skills/cm-frappe-agent/commands/frappe-erpnext.md +0 -210
  127. package/skills/cm-frappe-agent/commands/frappe-fix.md +0 -59
  128. package/skills/cm-frappe-agent/commands/frappe-frontend.md +0 -210
  129. package/skills/cm-frappe-agent/commands/frappe-fullstack.md +0 -243
  130. package/skills/cm-frappe-agent/commands/frappe-github.md +0 -57
  131. package/skills/cm-frappe-agent/commands/frappe-install.md +0 -52
  132. package/skills/cm-frappe-agent/commands/frappe-plan.md +0 -442
  133. package/skills/cm-frappe-agent/commands/frappe-remote.md +0 -58
  134. package/skills/cm-frappe-agent/commands/frappe-test.md +0 -356
  135. package/skills/cm-frappe-agent/docs/README.md +0 -51
  136. package/skills/cm-frappe-agent/docs/agents-catalog.md +0 -113
  137. package/skills/cm-frappe-agent/docs/architecture.md +0 -149
  138. package/skills/cm-frappe-agent/docs/commands-catalog.md +0 -82
  139. package/skills/cm-frappe-agent/docs/resources-catalog.md +0 -66
  140. package/skills/cm-frappe-agent/docs/sitemap-urls.txt +0 -52
  141. package/skills/cm-frappe-agent/docs/sitemap.md +0 -81
  142. package/skills/cm-frappe-agent/docs/sop/user-guide.md +0 -178
  143. package/skills/cm-frappe-agent/docs/sop/vibe-coding-guide.md +0 -122
  144. package/skills/cm-frappe-agent/resources/7-layer-architecture.md +0 -985
  145. package/skills/cm-frappe-agent/resources/bench_commands.md +0 -73
  146. package/skills/cm-frappe-agent/resources/code-patterns-guide.md +0 -948
  147. package/skills/cm-frappe-agent/resources/common_pitfalls.md +0 -266
  148. package/skills/cm-frappe-agent/resources/doctype-registry.md +0 -158
  149. package/skills/cm-frappe-agent/resources/installation-guide.md +0 -289
  150. package/skills/cm-frappe-agent/resources/rest-api-patterns.md +0 -182
  151. package/skills/cm-frappe-agent/resources/scaffold_checklist.md +0 -82
  152. package/skills/cm-frappe-agent/resources/upgrade_patterns.md +0 -113
  153. package/skills/cm-frappe-agent/resources/web-form-patterns.md +0 -252
  154. package/skills/cm-frappe-agent/skills/bench-commands/SKILL.md +0 -621
  155. package/skills/cm-frappe-agent/skills/client-scripts/SKILL.md +0 -642
  156. package/skills/cm-frappe-agent/skills/doctype-patterns/SKILL.md +0 -576
  157. package/skills/cm-frappe-agent/skills/frappe-api/SKILL.md +0 -740
  158. package/skills/cm-frappe-agent/skills/remote-operations/SKILL.md +0 -47
  159. package/skills/cm-frappe-agent/skills/server-scripts/SKILL.md +0 -608
  160. package/skills/cm-frappe-agent/skills/web-forms/SKILL.md +0 -46
  161. package/skills/frappe-app-builder.zip +0 -0
@@ -1,948 +0,0 @@
1
- # Production Code Patterns Guide
2
-
3
- > Battle-tested patterns from real Frappe production apps. Use these as reference
4
- > when building controllers, engines, APIs, tasks, reports, and client-side JS.
5
-
6
- ---
7
-
8
- ## 🏗️ Architecture — The 7-Layer Frappe App
9
-
10
- Every non-trivial Frappe app should follow this layered architecture:
11
-
12
- ```
13
- my_app/
14
- ├── my_app/
15
- │ └── doctype/ # Layer 1: Data Model (DocType JSON + controllers)
16
- │ ├── my_doctype/
17
- │ │ ├── my_doctype.json
18
- │ │ ├── my_doctype.py # Server controller
19
- │ │ └── my_doctype.js # Client controller
20
- │ └── ...
21
- ├── engines/ # Layer 2: Business Logic (PURE Python, no Frappe in core)
22
- │ ├── scoring_engine.py
23
- │ └── benefit_engine.py
24
- ├── api/ # Layer 3: API Endpoints (@frappe.whitelist)
25
- │ ├── external.py # Webhooks / external system integration
26
- │ └── internal.py # Internal UI-facing APIs
27
- ├── tasks/ # Layer 4: Scheduled Tasks (daily/weekly/monthly/cron)
28
- │ ├── daily.py
29
- │ ├── weekly.py
30
- │ └── monthly.py
31
- ├── setup/ # Layer 5: Install & Migrate hooks
32
- │ └── install.py
33
- ├── tests/ # Layer 6: Tests (standalone, no Frappe needed for pure logic)
34
- │ ├── test_engine.py
35
- │ └── test_benefit.py
36
- ├── public/js/ # Layer 7: Shared Client JS (badges, utilities, list views)
37
- │ └── my_app.js
38
- ├── hooks.py # The nervous system — connects everything
39
- ├── fixtures/ # Exported Roles, Workflows, Custom Fields
40
- └── modules.txt
41
- ```
42
-
43
- ### Why This Architecture?
44
-
45
- | Layer | Why Separate? |
46
- |-------|--------------|
47
- | **DocType** | Frappe manages schema via JSON. Controllers handle validate/submit/cancel |
48
- | **Engines** | Pure logic = testable without Frappe instance. Import in API, tasks, controllers |
49
- | **API** | Clean separation of external webhooks vs internal UI calls |
50
- | **Tasks** | Scheduler events are simple wrappers calling engine functions |
51
- | **Setup** | Idempotent install/migrate = safe `bench migrate` every time |
52
- | **Tests** | Pure-logic tests run with `pytest -v`, no bench needed |
53
- | **Shared JS** | Global utilities, badges, list view settings, reusable across all DocTypes |
54
-
55
- ---
56
-
57
- ## 📐 Layer 1: DocType Design
58
-
59
- ### Controller Pattern (Python)
60
-
61
- ```python
62
- import frappe
63
- from frappe import _
64
- from frappe.model.document import Document
65
- from frappe.utils import now
66
-
67
-
68
- class WarehouseViolation(Document):
69
- def validate(self):
70
- """Auto-fill derived fields BEFORE save."""
71
- self.fetch_employee_details()
72
- self.set_period()
73
- self.fetch_penalty_from_type()
74
-
75
- def fetch_employee_details(self):
76
- """Pull company, branch, name from Employee — avoid manual entry."""
77
- if self.employee:
78
- emp = frappe.db.get_value(
79
- "Employee", self.employee,
80
- ["company", "branch", "employee_name"], as_dict=True,
81
- )
82
- if emp:
83
- self.company = emp.company
84
- self.branch = emp.branch
85
- self.employee_name = emp.employee_name
86
-
87
- def set_period(self):
88
- """Auto-derive period (YYYY-MM) from date field."""
89
- if self.date:
90
- self.period = str(self.date)[:7]
91
-
92
- def fetch_penalty_from_type(self):
93
- """Auto-fill penalty points from master data if not manually set."""
94
- if self.violation_type and not self.penalty_points:
95
- pts = frappe.db.get_value(
96
- "Warehouse Violation Type", self.violation_type, "penalty_points"
97
- )
98
- if pts:
99
- self.penalty_points = pts
100
-
101
- def on_submit(self):
102
- """Record who confirmed and when."""
103
- self.confirmed_by = frappe.session.user
104
- self.confirmed_at = now()
105
- self.db_update()
106
- frappe.msgprint(
107
- _("Confirmed. Penalty: {0} pts for {1}.").format(
108
- self.penalty_points, self.employee_name
109
- ),
110
- indicator="orange", alert=True,
111
- )
112
-
113
- def on_cancel(self):
114
- frappe.msgprint(
115
- _("Cancelled for {0}.").format(self.employee_name), alert=True
116
- )
117
- ```
118
-
119
- **Key patterns:**
120
- - `validate()` = auto-fill derived fields (period from date, employee details from Link)
121
- - `on_submit()` = record audit trail (who, when), show user feedback
122
- - `on_cancel()` = cleanup or notify
123
- - Use `self.db_update()` after modifying fields in `on_submit` (doc already saved)
124
- - Use `frappe.db.get_value()` for single-field lookups (fast, no full doc load)
125
-
126
- ### Controller Pattern (JavaScript)
127
-
128
- ```javascript
129
- frappe.ui.form.on("Warehouse Violation", {
130
- refresh(frm) {
131
- // Color-coded workflow state indicator
132
- if (frm.doc.workflow_state) {
133
- const colors = {
134
- "Pending": "orange", "Confirmed": "green",
135
- "Appealed": "blue", "Waived": "red"
136
- };
137
- frm.page.set_indicator(
138
- frm.doc.workflow_state,
139
- colors[frm.doc.workflow_state] || "gray"
140
- );
141
- }
142
-
143
- // Dashboard indicators for key metrics
144
- if (frm.doc.penalty_points) {
145
- frm.dashboard.add_indicator(
146
- __("Penalty: {0} pts", [frm.doc.penalty_points]),
147
- frm.doc.penalty_points >= 8 ? "red" :
148
- frm.doc.penalty_points >= 5 ? "orange" : "blue"
149
- );
150
- }
151
-
152
- // Custom action buttons (non-submitted docs only)
153
- if (!frm.is_new() && frm.doc.docstatus === 0) {
154
- frm.add_custom_button(__("View Dashboard"), function () {
155
- frappe.set_route("query-report", "Employee Summary", {
156
- period: frm.doc.period
157
- });
158
- }, __("Actions"));
159
- }
160
- },
161
-
162
- date(frm) {
163
- // Auto-fill period from date
164
- myapp.autoSetPeriod(frm, "date", "period");
165
- },
166
-
167
- violation_type(frm) {
168
- // Auto-fill penalty from master data
169
- if (frm.doc.violation_type) {
170
- frappe.db.get_value(
171
- "Warehouse Violation Type",
172
- frm.doc.violation_type,
173
- "penalty_points",
174
- (r) => {
175
- if (r && r.penalty_points) {
176
- frm.set_value("penalty_points", r.penalty_points);
177
- }
178
- }
179
- );
180
- }
181
- }
182
- });
183
- ```
184
-
185
- ### DocType Design Rules
186
-
187
- ```
188
- ✅ DO:
189
- - Use Link fields to reference other DocTypes (Employee, Company, Branch)
190
- - Add "period" field (Data, YYYY-MM) auto-derived from date
191
- - Use Submittable DocTypes for records that need approval workflows
192
- - Set "module" in DocType JSON to your app module name
193
- - Add naming_rule or autoname for meaningful document names
194
-
195
- ❌ DON'T:
196
- - Hardcode company/branch/employee names — always use Link fields
197
- - Put business logic in DocType controllers — use engines/ instead
198
- - Forget to add "module" property — makes fixtures export fail
199
- - Create DocType without think about workflow states first
200
- ```
201
-
202
- ---
203
-
204
- ## ⚙️ Layer 2: Business Logic Engines
205
-
206
- **The most important pattern.** Separate pure logic from Frappe ORM.
207
-
208
- ### Pure-Logic Functions (No Frappe Imports)
209
-
210
- ```python
211
- """
212
- All pure-logic functions have NO frappe DB calls.
213
- Safe to unit-test without a running Frappe instance.
214
- """
215
- PPH_LEVEL_ORDER = ["needs_improvement", "average", "good", "excellent"]
216
-
217
- _DEFAULT_THRESHOLDS = [
218
- {"level": "needs_improvement", "min_pph": 0, "max_pph": 35},
219
- {"level": "average", "min_pph": 35, "max_pph": 55},
220
- {"level": "good", "min_pph": 55, "max_pph": 75},
221
- {"level": "excellent", "min_pph": 75, "max_pph": None},
222
- ]
223
-
224
- def classify_pph_level(pph: float, thresholds: list) -> str:
225
- """Classify using half-open intervals [min, max). Falls back to defaults."""
226
- if not thresholds:
227
- thresholds = _DEFAULT_THRESHOLDS
228
- for t in sorted(thresholds, key=lambda x: x["min_pph"]):
229
- min_v = float(t["min_pph"])
230
- max_v = t.get("max_pph")
231
- if pph >= min_v and (max_v is None or pph < float(max_v)):
232
- return t["level"]
233
- return thresholds[-1]["level"]
234
-
235
- def pph_level_gte(level: str, min_level: str) -> bool:
236
- """True if level >= min_level in ordering."""
237
- return PPH_LEVEL_ORDER.index(level) >= PPH_LEVEL_ORDER.index(min_level)
238
- ```
239
-
240
- ### Config Cascade Pattern (Frappe DB)
241
-
242
- ```python
243
- def get_config_with_cascade(
244
- company: str, branch: str = None, designation: str = None,
245
- as_of_date: str = None,
246
- ) -> list:
247
- """
248
- Fetch config with specificity cascade:
249
- 1. Company + Branch + Designation (most specific)
250
- 2. Company + Branch
251
- 3. Company only
252
- Returns the most specific matching rows.
253
- """
254
- if not as_of_date:
255
- as_of_date = frappe.utils.today()
256
-
257
- date_filter = """
258
- AND effective_from <= %(date)s
259
- AND (effective_to IS NULL OR effective_to = '' OR effective_to >= %(date)s)
260
- AND is_active = 1
261
- """
262
-
263
- for b, d in [(branch, designation), (branch, None), (None, None)]:
264
- conds = "company = %(company)s"
265
- params = {"company": company, "date": as_of_date}
266
-
267
- if b:
268
- conds += " AND branch = %(branch)s"
269
- params["branch"] = b
270
- else:
271
- conds += " AND (branch IS NULL OR branch = '')"
272
-
273
- if d:
274
- conds += " AND designation = %(designation)s"
275
- params["designation"] = d
276
- else:
277
- conds += " AND (designation IS NULL OR designation = '')"
278
-
279
- rows = frappe.db.sql(
280
- f"SELECT * FROM `tabConfig DocType` WHERE {conds} {date_filter}",
281
- params, as_dict=True,
282
- )
283
- if rows:
284
- return rows
285
- return []
286
- ```
287
-
288
- ### Main Pipeline Pattern
289
-
290
- ```python
291
- def calculate_period_score(employee: str, period_type: str, period_value: str) -> dict:
292
- """
293
- Full scoring pipeline for one employee over a period.
294
- Returns a dict matching the target DocType fields.
295
- """
296
- start_date, end_date = get_period_date_range(period_type, period_value)
297
-
298
- emp = frappe.db.get_value("Employee", employee,
299
- ["company", "branch", "designation"], as_dict=True)
300
- if not emp:
301
- frappe.throw(f"Employee {employee!r} not found")
302
-
303
- # Step 1: Aggregate raw data
304
- raw_data = frappe.db.sql("""
305
- SELECT value, hours FROM `tabRaw Data`
306
- WHERE employee = %(emp)s AND date BETWEEN %(s)s AND %(e)s
307
- """, {"emp": employee, "s": start_date, "e": end_date}, as_dict=True)
308
-
309
- # Step 2: Classify using pure-logic functions
310
- avg_value = statistics.mean([r.value for r in raw_data]) if raw_data else 0
311
- level = classify_level(avg_value, get_config(emp.company, emp.branch))
312
-
313
- # Step 3: Aggregate related documents (submitted only = docstatus=1)
314
- penalties = frappe.db.sql("""
315
- SELECT COALESCE(SUM(points), 0) AS total
316
- FROM `tabPenalty` WHERE employee = %(emp)s AND period = %(p)s AND docstatus = 1
317
- """, {"emp": employee, "p": period_value[:7]}, as_dict=True)
318
-
319
- # Step 4: Return structured result matching DocType fields
320
- return {
321
- "employee": employee,
322
- "period_type": period_type,
323
- "period_value": period_value,
324
- "company": emp.company,
325
- "avg_value": avg_value,
326
- "level": level,
327
- "total_penalty": penalties[0].total if penalties else 0,
328
- }
329
- ```
330
-
331
- **Key patterns:**
332
- - Separate pure functions (classify, compare) from DB functions (fetch, aggregate)
333
- - Always filter by `docstatus = 1` for submitted documents
334
- - Use `COALESCE(SUM(...), 0)` to avoid NULL results
335
- - Return dicts that match target DocType field names exactly
336
-
337
- ---
338
-
339
- ## 🌐 Layer 3: API Endpoints
340
-
341
- ### External Webhook Pattern (WMS/ERP Integration)
342
-
343
- ```python
344
- @frappe.whitelist(allow_guest=False)
345
- def receive_data(data):
346
- """
347
- POST /api/method/my_app.api.external.receive_data
348
- Body: {"data": [{...}, {...}]}
349
- Auth: token api_key:api_secret
350
- """
351
- if isinstance(data, str):
352
- data = json.loads(data)
353
-
354
- created = skipped = 0
355
- errors = []
356
-
357
- for rec in data:
358
- emp_code = rec.get("employee_code")
359
- try:
360
- employee = frappe.db.get_value(
361
- "Employee", {"employee_number": emp_code}, "name"
362
- )
363
- if not employee:
364
- errors.append({"code": emp_code, "error": f"Not found"})
365
- skipped += 1
366
- continue
367
-
368
- # Deduplicate: skip if already exists
369
- if frappe.db.exists("My DocType", {"employee": employee, "date": rec["date"]}):
370
- skipped += 1
371
- continue
372
-
373
- doc = frappe.get_doc({
374
- "doctype": "My DocType",
375
- "employee": employee,
376
- "date": rec.get("date"),
377
- "value": rec.get("value", 0),
378
- "source": "external_system",
379
- })
380
- doc.insert(ignore_permissions=True)
381
- created += 1
382
-
383
- except Exception as exc:
384
- errors.append({"code": emp_code, "error": str(exc)})
385
- skipped += 1
386
- frappe.log_error(str(exc), "receive_data")
387
-
388
- frappe.db.commit()
389
- return {"status": "ok" if not errors else "partial",
390
- "created": created, "skipped": skipped, "errors": errors}
391
- ```
392
-
393
- ### Internal API Pattern (UI + Dashboard)
394
-
395
- ```python
396
- @frappe.whitelist()
397
- def calculate_for_employee(employee, period_type, period_value, overwrite=False):
398
- """Manual trigger with idempotent upsert."""
399
- if not frappe.has_permission("My DocType", "write"):
400
- frappe.throw(_("Permission denied"), frappe.PermissionError)
401
-
402
- data = engine.calculate(employee, period_type, period_value)
403
-
404
- existing = frappe.db.get_value("My DocType", {
405
- "employee": employee, "period_type": period_type,
406
- "period_value": period_value,
407
- })
408
-
409
- if existing:
410
- if not overwrite:
411
- return {"status": "exists", "name": existing}
412
- doc = frappe.get_doc("My DocType", existing)
413
- if doc.docstatus == 1:
414
- frappe.throw(_("Cannot overwrite submitted record"))
415
- doc.update(data)
416
- doc.save(ignore_permissions=True)
417
- frappe.db.commit()
418
- return {"status": "updated", "name": doc.name}
419
-
420
- doc = frappe.get_doc({"doctype": "My DocType", **data})
421
- doc.insert(ignore_permissions=True)
422
- frappe.db.commit()
423
- return {"status": "created", "name": doc.name}
424
- ```
425
-
426
- ### Permission Query Pattern
427
-
428
- ```python
429
- def get_permission_query(user: str) -> str:
430
- """Row-level security: user sees only their own records unless manager/admin."""
431
- roles = frappe.get_roles(user)
432
- admin_roles = {"System Manager", "HR Reviewer", "Department Manager"}
433
- if admin_roles & set(roles):
434
- return "" # No filter = see all
435
- emp = frappe.db.get_value("Employee", {"user_id": user}, "name")
436
- if emp:
437
- return f"`tabMy DocType`.employee = '{emp}'"
438
- return "1=0" # See nothing
439
- ```
440
-
441
- **Key patterns:**
442
- - `allow_guest=False` for webhooks (requires API key auth)
443
- - Parse `data` as string or dict (Frappe may pass either)
444
- - Batch processing with created/skipped/errors counters
445
- - Idempotent upsert: check existing → update or insert
446
- - `frappe.db.commit()` after bulk operations
447
- - Permission queries for row-level security
448
-
449
- ---
450
-
451
- ## ⏰ Layer 4: Scheduler Tasks
452
-
453
- ### Pattern: Batch Processing All Employees
454
-
455
- ```python
456
- def run_monthly_calculation(year_month: str = None):
457
- """Monthly task: calculate scores for ALL active employees."""
458
- if not year_month:
459
- year_month = add_months(today(), -1)[:7]
460
-
461
- frappe.logger("my_app").info(f"[Monthly] Starting for {year_month}")
462
- employees = frappe.get_all("Employee", filters={"status": "Active"}, fields=["name"])
463
- ok = err = 0
464
-
465
- for emp in employees:
466
- try:
467
- data = engine.calculate(emp.name, "month", year_month)
468
- existing = frappe.db.get_value("Score", {
469
- "employee": emp.name, "period_type": "month",
470
- "period_value": year_month,
471
- })
472
- if existing:
473
- doc = frappe.get_doc("Score", existing)
474
- if doc.docstatus == 0: # Only update drafts
475
- doc.update(data)
476
- doc.save(ignore_permissions=True)
477
- else:
478
- frappe.get_doc({"doctype": "Score", **data}).insert(ignore_permissions=True)
479
- ok += 1
480
- except Exception as exc:
481
- err += 1
482
- frappe.log_error(str(exc), f"Monthly — {emp.name}")
483
-
484
- frappe.db.commit()
485
- frappe.logger("my_app").info(f"[Monthly] Done: {ok} ok, {err} errors")
486
- ```
487
-
488
- ### Pattern: Reminder Notifications
489
-
490
- ```python
491
- def send_pending_reminders():
492
- """Notify managers about records pending > 3 days."""
493
- cutoff = add_days(today(), -3)
494
- pending = frappe.db.sql("""
495
- SELECT r.name, r.employee, e.employee_name, e.reports_to
496
- FROM `tabMy Record` r
497
- LEFT JOIN `tabEmployee` e ON r.employee = e.name
498
- WHERE r.docstatus = 0 AND r.creation <= %(cutoff)s
499
- """, {"cutoff": cutoff}, as_dict=True)
500
-
501
- by_manager = {}
502
- for row in pending:
503
- if row.reports_to:
504
- by_manager.setdefault(row.reports_to, []).append(row)
505
-
506
- for mgr, records in by_manager.items():
507
- mgr_email = frappe.db.get_value("Employee", mgr, "user_id")
508
- if mgr_email:
509
- frappe.sendmail(
510
- recipients=[mgr_email],
511
- subject=f"[My App] {len(records)} records need review",
512
- message=format_reminder_html(records),
513
- )
514
- ```
515
-
516
- ### hooks.py Scheduler Config
517
-
518
- ```python
519
- scheduler_events = {
520
- "daily": [
521
- "my_app.tasks.daily.run_daily_aggregation",
522
- "my_app.tasks.daily.send_pending_reminders",
523
- ],
524
- "weekly": [
525
- "my_app.tasks.weekly.run_weekly_calculation",
526
- ],
527
- "cron": {
528
- "0 1 1 * *": [ # 1st of month at 01:00
529
- "my_app.tasks.monthly.run_monthly_calculation",
530
- ]
531
- },
532
- }
533
- ```
534
-
535
- ---
536
-
537
- ## 🔗 hooks.py — The Nervous System
538
-
539
- ```python
540
- app_name = "my_app"
541
- app_title = "My App"
542
- app_publisher = "My Company"
543
- app_description = "App description"
544
- app_license = "MIT"
545
-
546
- required_apps = ["frappe/hrms"] # Declare dependencies
547
-
548
- # ── Assets ────────────────────────────────────────────────────────────────
549
- app_include_js = ["/assets/my_app/js/my_app.js"]
550
-
551
- # ── DocType JS overrides (for core DocTypes like Employee) ────────────────
552
- doctype_js = {
553
- "Employee": "public/js/employee.js"
554
- }
555
-
556
- # ── After install / migrate ───────────────────────────────────────────────
557
- after_install = "my_app.setup.install.after_install"
558
- after_migrate = "my_app.setup.install.after_migrate"
559
-
560
- # ── Doc Events (server-side hooks) ────────────────────────────────────────
561
- doc_events = {
562
- "My Submittable Doc": {
563
- "on_submit": "my_app.engines.my_engine.on_doc_submit",
564
- "on_cancel": "my_app.engines.my_engine.on_doc_cancel",
565
- },
566
- "My Auto Doc": {
567
- "after_insert": "my_app.engines.my_engine.on_doc_insert",
568
- },
569
- }
570
-
571
- # ── Fixtures (portable across sites) ─────────────────────────────────────
572
- fixtures = [
573
- {"dt": "Role", "filters": [["name", "in", ["My Role 1", "My Role 2"]]]},
574
- {"dt": "Custom Field", "filters": [["module", "=", "My App"]]},
575
- {"dt": "Workflow", "filters": [["document_type", "in", ["My Doc"]]]},
576
- {"dt": "Workflow State", "filters": [["name", "in", [
577
- "Draft", "Pending", "Approved", "Rejected"
578
- ]]]},
579
- ]
580
-
581
- # ── Permission query conditions ───────────────────────────────────────────
582
- permission_query_conditions = {
583
- "My Score Doc": "my_app.api.internal.get_permission_query",
584
- }
585
- ```
586
-
587
- ---
588
-
589
- ## 🛠️ Layer 5: Setup / Install
590
-
591
- ```python
592
- def after_install():
593
- """Run once after bench install-app. Must be idempotent."""
594
- _create_roles()
595
- _create_custom_fields()
596
- _create_workflow_states()
597
- frappe.db.commit()
598
-
599
- def after_migrate():
600
- """Run after every bench migrate. Must be idempotent."""
601
- _ensure_roles()
602
- _create_workflow_states()
603
- frappe.db.commit()
604
-
605
- def _create_roles():
606
- for role in ["My Role 1", "My Role 2"]:
607
- if not frappe.db.exists("Role", role):
608
- frappe.get_doc({
609
- "doctype": "Role", "role_name": role, "desk_access": 1
610
- }).insert(ignore_permissions=True)
611
-
612
- def _create_custom_fields():
613
- """Add custom fields to core DocTypes (e.g., Employee)."""
614
- fields = [
615
- {
616
- "dt": "Employee",
617
- "fieldname": "my_app_code",
618
- "fieldtype": "Data",
619
- "label": "My App Code",
620
- "insert_after": "employee_number",
621
- "unique": 1, "search_index": 1,
622
- "module": "My App",
623
- },
624
- ]
625
- for cf in fields:
626
- if not frappe.db.exists("Custom Field", {"dt": cf["dt"], "fieldname": cf["fieldname"]}):
627
- frappe.get_doc({"doctype": "Custom Field", **cf}).insert(ignore_permissions=True)
628
-
629
- def _create_workflow_states():
630
- states = [
631
- {"state": "Draft", "icon": "edit", "style": ""},
632
- {"state": "Pending", "icon": "time", "style": "Warning"},
633
- {"state": "Approved", "icon": "ok-sign", "style": "Success"},
634
- {"state": "Rejected", "icon": "remove", "style": "Danger"},
635
- ]
636
- for ws in states:
637
- if not frappe.db.exists("Workflow State", ws["state"]):
638
- frappe.get_doc({
639
- "doctype": "Workflow State",
640
- "workflow_state_name": ws["state"],
641
- "icon": ws["icon"], "style": ws["style"],
642
- }).insert(ignore_permissions=True)
643
- ```
644
-
645
- ---
646
-
647
- ## 📊 Layer 6: Reports
648
-
649
- ### Script Report Pattern
650
-
651
- **Python (employee_scoring_summary.py):**
652
-
653
- ```python
654
- def execute(filters=None):
655
- filters = filters or {}
656
- columns = get_columns()
657
- data = get_data(filters)
658
- chart = get_chart(data)
659
- report_summary = get_report_summary(data)
660
- return columns, data, None, chart, report_summary
661
-
662
- def get_columns():
663
- return [
664
- {"fieldname": "employee", "label": "Employee", "fieldtype": "Link",
665
- "options": "Employee", "width": 120},
666
- {"fieldname": "value", "label": "Value", "fieldtype": "Float",
667
- "precision": 1, "width": 90},
668
- # ... more columns
669
- ]
670
-
671
- def get_data(filters):
672
- conditions = "WHERE d.docstatus != 2"
673
- values = {}
674
- if filters.get("company"):
675
- conditions += " AND d.company = %(company)s"
676
- values["company"] = filters["company"]
677
- # ... more filters
678
- return frappe.db.sql(f"""
679
- SELECT d.employee, d.value, ...
680
- FROM `tabMy Doc` d {conditions}
681
- ORDER BY d.value DESC
682
- """, values, as_dict=True)
683
-
684
- def get_chart(data):
685
- if not data:
686
- return None
687
- top = data[:10]
688
- return {
689
- "data": {
690
- "labels": [r.employee_name for r in top],
691
- "datasets": [
692
- {"name": "Value", "values": [r.value for r in top]},
693
- ],
694
- },
695
- "type": "bar",
696
- "colors": ["#2980b9"],
697
- }
698
-
699
- def get_report_summary(data):
700
- if not data:
701
- return []
702
- total = len(data)
703
- avg = sum(r.value for r in data) / total if total else 0
704
- return [
705
- {"value": total, "label": "Total", "datatype": "Int", "indicator": "Blue"},
706
- {"value": round(avg, 1), "label": "Average", "datatype": "Float",
707
- "indicator": "Green" if avg >= 50 else "Orange"},
708
- ]
709
- ```
710
-
711
- **JavaScript (employee_scoring_summary.js):**
712
-
713
- ```javascript
714
- frappe.query_reports["Employee Scoring Summary"] = {
715
- filters: [
716
- {
717
- fieldname: "company",
718
- label: __("Company"),
719
- fieldtype: "Link",
720
- options: "Company",
721
- default: frappe.defaults.get_user_default("Company"),
722
- },
723
- {
724
- fieldname: "period",
725
- label: __("Period (YYYY-MM)"),
726
- fieldtype: "Data",
727
- default: frappe.datetime.get_today().substring(0, 7),
728
- },
729
- ],
730
- };
731
- ```
732
-
733
- ---
734
-
735
- ## 🎨 Layer 7: Shared Client JS
736
-
737
- ```javascript
738
- /**
739
- * Global client-side utilities namespace.
740
- * Loaded via app_include_js in hooks.py.
741
- */
742
- window.myapp = window.myapp || {};
743
-
744
- // Color-coded badge
745
- myapp.levelBadge = function (level, colorMap) {
746
- const info = colorMap[level] || { color: "#888", label: level || "N/A" };
747
- return `<span style="background:${info.color};color:#fff;padding:2px 8px;
748
- border-radius:4px;font-size:12px;font-weight:600;">${info.label}</span>`;
749
- };
750
-
751
- // Promise-based Frappe method call
752
- myapp.callMethod = function (method, args) {
753
- return new Promise((resolve, reject) => {
754
- frappe.call({
755
- method, args,
756
- callback: (r) => r?.message != null ? resolve(r.message) : reject(r),
757
- error: reject,
758
- });
759
- });
760
- };
761
-
762
- // Auto-set period from date field
763
- myapp.autoSetPeriod = function (frm, dateField, periodField) {
764
- const d = frm.doc[dateField];
765
- if (d && d.length >= 7) frm.set_value(periodField, d.substring(0, 7));
766
- };
767
-
768
- // List View Settings with workflow color indicators
769
- frappe.listview_settings['My Submittable Doc'] = {
770
- get_indicator(doc) {
771
- const colors = {
772
- "Pending": "orange", "Approved": "green",
773
- "Rejected": "red", "Draft": "gray"
774
- };
775
- const c = colors[doc.workflow_state] || "gray";
776
- return [__(doc.workflow_state), c, "workflow_state,=," + doc.workflow_state];
777
- }
778
- };
779
- ```
780
-
781
- ---
782
-
783
- ## 🧪 Layer 6b: Testing
784
-
785
- ### Standalone Pure-Logic Tests (No Frappe!)
786
-
787
- ```python
788
- """
789
- Run: python -m pytest my_app/tests/test_engine.py -v
790
- No Frappe instance needed — tests only pure-logic functions.
791
- """
792
- import unittest
793
-
794
- # Inline pure-logic (copy from engine, no frappe imports)
795
- PPH_LEVEL_ORDER = ["needs_improvement", "average", "good", "excellent"]
796
- _DEFAULT_THRESHOLDS = [
797
- {"level": "needs_improvement", "min_pph": 0, "max_pph": 35},
798
- {"level": "average", "min_pph": 35, "max_pph": 55},
799
- ]
800
- # ... paste pure functions here
801
-
802
- class TestClassifyLevel(unittest.TestCase):
803
- def test_boundary(self):
804
- self.assertEqual(classify_level(35, _DEFAULT_THRESHOLDS), "average")
805
-
806
- def test_empty_defaults(self):
807
- self.assertEqual(classify_level(60, []), "good")
808
-
809
- class TestLevelOrdering(unittest.TestCase):
810
- def test_gte(self):
811
- self.assertTrue(level_gte("excellent", "good"))
812
- self.assertFalse(level_gte("average", "good"))
813
-
814
- if __name__ == "__main__":
815
- unittest.main(verbosity=2)
816
- ```
817
-
818
- **Why inline pure functions in test files?**
819
- - Tests run with `pytest -v` — no bench, no MariaDB, no Frappe site
820
- - CI/CD friendly — fast, isolated, reliable
821
- - Keep the test file self-contained
822
-
823
- ---
824
-
825
- ## 🔄 Workflow Design
826
-
827
- ### Common Workflow Patterns
828
-
829
- | DocType Category | States | Submittable? |
830
- |-----------------|--------|--------------|
831
- | **Violation/Penalty** | Pending → Confirmed → Appealed / Waived | Yes |
832
- | **Score/Review** | Draft → Manager Review → HR Approved → Final | Yes |
833
- | **Recovery/Request** | Pending → Approved / Rejected | Yes |
834
- | **Bonus/Reward** | Pending Approval → Approved / Rejected | Yes |
835
- | **Config/Master** | N/A (no workflow, just CRUD) | No |
836
-
837
- ### Workflow State Colors
838
-
839
- ```python
840
- STATE_STYLES = {
841
- "Draft": ("edit", ""), # Gray
842
- "Pending": ("time", "Warning"), # Orange
843
- "Approved": ("ok-sign", "Success"), # Green
844
- "Rejected": ("remove", "Danger"), # Red
845
- "Review": ("eye-open","Warning"), # Orange
846
- "Final": ("star", "Success"), # Green
847
- }
848
- ```
849
-
850
- ---
851
-
852
- ## 🌍 Layer 8: Multi-Language (i18n)
853
-
854
- ### Translation Workflow
855
-
856
- Frappe uses bare strings wrapped in translation functions: `_("String")` in Python and `__("String")` in JavaScript. Do not use translation keys; use the English baseline string as the key.
857
-
858
- 1. **Wrap all UI-facing strings:**
859
- - Python: `frappe._("User {0} not found").format(user_id)`
860
- - JS: `__("User {0} not found", [user_id])`
861
- 2. **Export strings to CSV:**
862
- `bench --site <site> get-untranslated <language-code> <path/to/output.csv>`
863
- 3. **Translate and Import:**
864
- Add translations to the CSV, then place the translated translations in `my_app/translations/<lang>.csv`.
865
-
866
- ### Translation Rules
867
- ```
868
- ✅ DO:
869
- - Use English as the default bare string.
870
- - Use `{0}`, `{1}` for interpolation (positional args) to allow word reordering in other languages.
871
- - Run `bench --site <site> migrate` to clear caches and load new translations.
872
-
873
- ❌ DON'T:
874
- - Translate log messages or internal system errors meant for developers.
875
- - Use concatenation (`_("Hello") + " " + user`) — always interpolate (`_("Hello {0}").format(user)`).
876
- ```
877
-
878
- ---
879
-
880
- ## 🚀 Layer 9: CI/CD & GitHub Actions
881
-
882
- When building or fixing CI/CD pipelines (`ci.yml`, `linter.yml`) for Frappe apps, adhere to these stability rules:
883
-
884
- ### Versions & Environments
885
- - **Python:** Strictly unify the Python version across ALL jobs (e.g., `python-version: '3.12'`). Avoid alpha versions.
886
- - **Node.js:** Use Node 20+ (e.g., `node-version: 20`). Frappe v15 ecosystem relies on `jsdom@29+` which explicitly drops Node 18.
887
- - **GitHub Actions:** Use stable tags (`@v4`, `@v5`). Never guess action versions.
888
-
889
- ### Frappe Installation in CI
890
- - **Skip Assets:** Always use `--skip-assets` with `bench get-app` to prevent OOM and esbuild race conditions.
891
- - **Local App Registration:** `bench get-app /path/to/local/app --skip-assets` does NOT auto-add to `sites/apps.txt`.
892
- - **apps.txt Fix:** Manually append safely:
893
- ```bash
894
- grep -q my_app sites/apps.txt 2>/dev/null || printf '\nmy_app\n' >> sites/apps.txt
895
- ```
896
-
897
- ---
898
-
899
- ## ⚔️ Strict Constraints
900
-
901
- ### NEVER
902
- - Push directly to production branch
903
- - Delete or modify DocType JSON files without `bench migrate` after
904
- - Hardcode employee/company references — always use Link fields
905
- - Skip `frappe.db.commit()` after bulk operations
906
- - Use `frappe.db.sql` for INSERT/UPDATE when ORM is available
907
- - Put complex business logic directly in DocType controllers
908
- - Ignore `docstatus` when querying submitted documents
909
-
910
- ### ALWAYS
911
- - Run `bench --site <site> migrate` after changing DocType schemas
912
- - Run `bench build --app <app>` after changing JS/CSS
913
- - Use `@frappe.whitelist()` decorator for API endpoints
914
- - Use `frappe.has_permission()` before operations in APIs
915
- - Separate pure logic into `engines/` for testability
916
- - Use `frappe.logger("app_name")` for structured logging
917
- - Make `after_install` and `after_migrate` idempotent
918
- - Filter by `docstatus = 1` when aggregating submitted records
919
- - Use `COALESCE(SUM(...), 0)` to avoid NULL in SQL aggregations
920
- - Add `frappe.db.commit()` at the end of batch task functions
921
-
922
- ### CONFIRM BEFORE RUNNING
923
- - `bench --site <site> reinstall` (destroys data)
924
- - Bulk data migration scripts
925
- - Modifying workflow states (affects existing records)
926
- - `bench drop-site` or `--force` commands
927
-
928
- ---
929
-
930
- ## Example: Scaffold a New Frappe Custom App
931
-
932
- **WARNING:** Always use `bench new-app` to create the initial app structure! Do not create it manually, as Frappe Cloud and other tools rely on standard boilerplate files (`setup.py`, `MANIFEST.in`, `patches.txt`, `hooks.py`, etc.) that are generated by `bench`.
933
-
934
- ```bash
935
- # 1. Create app
936
- bench new-app my_custom_app
937
-
938
- # 2. Create module structure inside the generated app
939
- mkdir -p apps/my_custom_app/my_custom_app/{engines,api,tasks,setup,tests,scripts,fixtures}
940
- mkdir -p apps/my_custom_app/my_custom_app/public/js
941
- touch apps/my_custom_app/my_custom_app/{engines,api,tasks,setup,tests}/__init__.py
942
-
943
- # 3. Install on site
944
- bench --site mysite.localhost install-app my_custom_app
945
-
946
- # 4. Enable dev mode
947
- bench --site mysite.localhost set-config developer_mode 1
948
- ```