opencode-skills-collection 4.0.71 → 4.0.72
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled-skills/.antigravity-install-manifest.json +7 -1
- package/bundled-skills/chatexport-need-miner/SKILL.md +196 -0
- package/bundled-skills/dali-short-address-commissioner/SKILL.md +185 -0
- package/bundled-skills/eol-resistor-calculator/SKILL.md +211 -0
- package/bundled-skills/esl-price-sync/SKILL.md +210 -0
- package/bundled-skills/marlin-bed-leveling/SKILL.md +181 -0
- package/bundled-skills/oneroster-csv-validator/SKILL.md +205 -0
- package/bundled-skills/telegram-mini-app/references/detailed-guide.md +26 -0
- package/package.json +1 -1
- package/skills_index.json +197 -0
|
@@ -0,0 +1,210 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: esl-price-sync
|
|
3
|
+
description: "Synchronizes retail prices between ERP/POS systems and Electronic Shelf Labels (SES-imagotag, ZKONG, Pricer, Hanshow, SOLUM) with delta watermarking, idempotency, and battery modeling. Trigger phrases: esl price sync, electronic shelf labels, zkong sync, ses imagotag price, ghost pricing audit."
|
|
4
|
+
category: architecture
|
|
5
|
+
risk: safe
|
|
6
|
+
source: community
|
|
7
|
+
source_repo: wwewtech/esl-price-sync
|
|
8
|
+
source_type: community
|
|
9
|
+
date_added: "2026-09-22"
|
|
10
|
+
author: wwewtech
|
|
11
|
+
tags: [retail-tech, esl, iot, supply-chain, systems-integration, enterprise-software]
|
|
12
|
+
tools: [claude, cursor, gemini, windsurf]
|
|
13
|
+
license: "MIT"
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# ESL Price Sync: Deterministic Retail Shelf-Edge Pricing Architecture
|
|
17
|
+
|
|
18
|
+
Ensure strict real-time parity between ERP/POS price masters and Electronic Shelf Label (ESL) fleets (SES-imagotag, ZKONG, Pricer, Hanshow, SOLUM) via bounded delta streams, monotonic idempotency tokens, and Sub-GHz RF battery preservation models.
|
|
19
|
+
|
|
20
|
+
## When to Use This Skill
|
|
21
|
+
|
|
22
|
+
Activate this skill when:
|
|
23
|
+
- Designing, implementing, or auditing integration pipelines between enterprise retail masters (SAP, Oracle Retail, Dynamics 365, Lightspeed) and ESL server platforms.
|
|
24
|
+
- The user asks: "How do I sync price updates to ESL tags without draining batteries?", "Debug ghost pricing where POS doesn't match the shelf tag", "Design an idempotent delta feed for ZKONG or SES-imagotag", or "Schedule promotional pricing with automatic rollback".
|
|
25
|
+
- Reconciling price discrepancy logs between central ERP databases, ESL middleware, and wireless base station ACK telemetry.
|
|
26
|
+
- Calculating battery life depletion curves for e-paper tags running on CR2450 coin cells under varying refresh frequencies.
|
|
27
|
+
|
|
28
|
+
Do NOT use this skill when:
|
|
29
|
+
- Managing e-commerce web store frontends or dynamic pricing algorithms (this skill governs faithful delivery of decided prices to physical tags).
|
|
30
|
+
- Troubleshooting individual dead hardware tags before gateway RF coverage and RSSI signal propagation have been verified.
|
|
31
|
+
- Performing live write operations on production store fleets without a preceding read-only audit verification pass.
|
|
32
|
+
|
|
33
|
+
## Core Mental Models & Non-Negotiable Rules
|
|
34
|
+
|
|
35
|
+
1. **The 5-Layer ESL Architecture Pipeline**:
|
|
36
|
+
- Every price synchronization event traverses five distinct operational tiers:
|
|
37
|
+
$$\mathbf{Price\ Master\ (ERP/POS)} \xrightarrow{\text{Extract}} \mathbf{Transform\ Middleware} \xrightarrow{\text{API/Message}} \mathbf{ESL\ Server} \xrightarrow{\text{Sub-GHz/2.4GHz}} \mathbf{Base\ Station\ (AP)} \xrightarrow{\text{RF\ Packet}} \mathbf{Physical\ Tag}$$
|
|
38
|
+
- **Traceability Invariant**: Every price payload MUST carry a deterministic `trace_id` recording timestamps and status codes across all five layers. A "200 OK" from the ESL central server API is NEVER treated as proof that the physical shelf display has updated.
|
|
39
|
+
|
|
40
|
+
2. **Strict Delta Discipline & Cryptographic Watermarking**:
|
|
41
|
+
- Pushing a full 50,000-SKU store catalog on a scheduled cron is strictly forbidden.
|
|
42
|
+
- Updates MUST be generated by calculating a cryptographic row hash:
|
|
43
|
+
$$\text{RowHash} = \text{SHA256}(\text{SKU} + \text{BasePrice} + \text{PromoPrice} + \text{Currency} + \text{UnitOfMeasure} + \text{Barcode} + \text{TemplateID})$$
|
|
44
|
+
- Only records where $\text{RowHash}_{new} \neq \text{RowHash}_{pushed}$ are queued for transmission.
|
|
45
|
+
- Full pushes are strictly restricted to initial store commissioning and disaster recovery restores.
|
|
46
|
+
|
|
47
|
+
3. **Monotonic Idempotency & Out-of-Order Packet Defense**:
|
|
48
|
+
- High-volume retail message brokers inevitably deliver packets out of order during network re-balancing.
|
|
49
|
+
- Every transmission packet MUST include an `idempotency_key` and a monotonically increasing `version_timestamp` (ISO 8601 UTC).
|
|
50
|
+
- If an incoming price payload has $\text{version\_timestamp} \le \text{last\_applied\_version}$, the ESL middleware MUST discard the payload to eliminate race condition reversions.
|
|
51
|
+
|
|
52
|
+
4. **RF Gateway Rate-Limiting & Battery Longevity Budget**:
|
|
53
|
+
- Each physical e-paper display refresh costs between $12\ \mu\text{Ah}$ and $25\ \mu\text{Ah}$.
|
|
54
|
+
- A dual-CR2450 battery pack provides $1,200\text{ mAh}$ nominal capacity ($960\text{ mAh}$ usable considering passivation and low-temperature shelf environments).
|
|
55
|
+
- **The 3-Update Daily Rule**: Limiting tag updates to $\le 3$ refreshes per day guarantees a $5.5\text{--}7\text{ year}$ operational lifespan.
|
|
56
|
+
- **Base Station Queue Throttling**: Gateways operate over shared Sub-GHz (868/915 MHz) or proprietary 2.4 GHz RF channels. Batch transmissions must not exceed 150 tag updates per gateway per minute to prevent RF collision retries from draining batteries.
|
|
57
|
+
|
|
58
|
+
5. **Read-Only Audit Mode by Default**:
|
|
59
|
+
- Diagnostic and reconciliation tasks MUST execute in read-only audit mode.
|
|
60
|
+
- Output discrepancy reports containing SKU, ERP Price, Gateway ACK Price, and Divergence Layer.
|
|
61
|
+
- State-changing write updates require explicit operator consent.
|
|
62
|
+
|
|
63
|
+
## Named Sins & Anti-Patterns (Что категорически ЗАПРЕЩЕНО)
|
|
64
|
+
|
|
65
|
+
| Anti-Pattern | Manifestation in Code/Workflow | Mandatory Production Counter-Rule |
|
|
66
|
+
| :--- | :--- | :--- |
|
|
67
|
+
| **Scheduled Full Catalog Floods** | Blasting 60,000 tags every night at midnight. | Stream strict deltas using SHA256 content watermarks. |
|
|
68
|
+
| **Ghost Price Blindness** | Assuming API `200 OK` means the customer sees the new price. | Track gateway ACK packet receipts (`status: ACK_CONFIRMED`). |
|
|
69
|
+
| **Non-Idempotent Retries** | Retrying failed HTTP calls without idempotency keys. | Attach `UUIDv5(sku, store_id, price_timestamp)` to all mutations. |
|
|
70
|
+
| **Timezone Ambiguity** | Pushing promo prices without UTC/local time offsets. | Enforce ISO 8601 UTC timestamps with explicit store local offsets. |
|
|
71
|
+
| **Gateway RF Flooding** | Blasting 5,000 updates/second to a single RF base station. | Enforce leaky-bucket queue throttling ($\le 150\text{ tags/min/AP}$). |
|
|
72
|
+
| **Silent Canvas Overflow** | Pushing a 6-digit price into a 4-digit template field ($1499 becomes $14). | Validate text bounds and template layout dimensions before enqueueing. |
|
|
73
|
+
| **Destructive Rollback Omission** | Deploying flash sales without pre-computing revert state. | Cache previous active state to enable 1-click atomic rollback. |
|
|
74
|
+
| **Orphan Tag Neglect** | Leaving decommissioned tags in the active database. | Reconcile missing heartbeat tags after 72 hours; flag as `ORPHAN`. |
|
|
75
|
+
| **Mid-Day Peak Syncing** | Pushing massive price shifts during peak Saturday shopping hours. | Schedule bulk promotions before store opening or during off-peak hours. |
|
|
76
|
+
| **Unshielded Write Broadcasts** | Executing write scripts without an audit dry-run. | Enforce dry-run simulation mode prior to initiating physical RF broadcasts. |
|
|
77
|
+
|
|
78
|
+
## Concrete Archetypes / Presets
|
|
79
|
+
|
|
80
|
+
### Archetype 1: Idempotent Delta Synchronizer (Python 3.10+)
|
|
81
|
+
```python
|
|
82
|
+
import hashlib
|
|
83
|
+
import time
|
|
84
|
+
from typing import Dict, List, Optional
|
|
85
|
+
|
|
86
|
+
def compute_sku_hash(item: dict) -> str:
|
|
87
|
+
payload = f"{item['sku']}:{item['price']}:{item['promo_price']}:{item['uom']}:{item['template']}"
|
|
88
|
+
return hashlib.sha256(payload.encode('utf-8')).hexdigest()
|
|
89
|
+
|
|
90
|
+
class ESLDeltaSyncPipeline:
|
|
91
|
+
def __init__(self, watermark_db: dict, max_rate_per_min: int = 150):
|
|
92
|
+
self.watermark_db = watermark_db # SKU -> last_pushed_hash
|
|
93
|
+
self.max_rate_per_min = max_rate_per_min
|
|
94
|
+
self.last_tx_time = 0.0
|
|
95
|
+
|
|
96
|
+
def process_catalog_stream(self, incoming_feed: List[dict], dry_run: bool = True) -> dict:
|
|
97
|
+
to_push = []
|
|
98
|
+
unchanged = 0
|
|
99
|
+
|
|
100
|
+
for item in incoming_feed:
|
|
101
|
+
sku = item['sku']
|
|
102
|
+
current_hash = compute_sku_hash(item)
|
|
103
|
+
last_hash = self.watermark_db.get(sku)
|
|
104
|
+
|
|
105
|
+
if current_hash != last_hash:
|
|
106
|
+
idempotency_key = hashlib.sha1(f"{sku}:{item['version']}:{current_hash}".encode()).hexdigest()
|
|
107
|
+
to_push.append({
|
|
108
|
+
"sku": sku,
|
|
109
|
+
"tag_id": item['tag_id'],
|
|
110
|
+
"price_data": item,
|
|
111
|
+
"idempotency_key": idempotency_key,
|
|
112
|
+
"content_hash": current_hash
|
|
113
|
+
})
|
|
114
|
+
else:
|
|
115
|
+
unchanged += 1
|
|
116
|
+
|
|
117
|
+
if dry_run:
|
|
118
|
+
return {"mode": "DRY_RUN", "queued_for_rf": len(to_push), "unchanged": unchanged}
|
|
119
|
+
|
|
120
|
+
# Rate-limited physical transmission
|
|
121
|
+
sent = 0
|
|
122
|
+
min_interval = 60.0 / self.max_rate_per_min
|
|
123
|
+
for job in to_push:
|
|
124
|
+
now = time.time()
|
|
125
|
+
elapsed = now - self.last_tx_time
|
|
126
|
+
if elapsed < min_interval:
|
|
127
|
+
time.sleep(min_interval - elapsed)
|
|
128
|
+
|
|
129
|
+
transmit_to_esl_gateway(job)
|
|
130
|
+
self.watermark_db[job['sku']] = job['content_hash']
|
|
131
|
+
self.last_tx_time = time.time()
|
|
132
|
+
sent += 1
|
|
133
|
+
|
|
134
|
+
return {"mode": "LIVE_COMMITTED", "sent_deltas": sent, "unchanged": unchanged}
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
### Archetype 2: Reconciliation Audit Report Schema
|
|
138
|
+
```json
|
|
139
|
+
{
|
|
140
|
+
"audit_timestamp": "2026-09-22T10:15:00Z",
|
|
141
|
+
"store_id": "STORE-0842",
|
|
142
|
+
"total_audited_skus": 14200,
|
|
143
|
+
"in_sync": 14188,
|
|
144
|
+
"ghost_pricing_discrepancies": [
|
|
145
|
+
{
|
|
146
|
+
"sku": "SKU-99014",
|
|
147
|
+
"barcode": "4006381333931",
|
|
148
|
+
"tag_mac": "00:1A:7D:DA:71:04",
|
|
149
|
+
"aisle": "Aisle 4 - Cereals",
|
|
150
|
+
"erp_master_price": 4.99,
|
|
151
|
+
"esl_server_price": 4.99,
|
|
152
|
+
"tag_ack_price": 3.99,
|
|
153
|
+
"divergence_layer": "RF Base Station (Packet dropped due to antenna shadow)",
|
|
154
|
+
"last_ack_timestamp": "2026-09-20T04:12:00Z",
|
|
155
|
+
"recommended_action": "Force targeted re-transmit on Channel 11"
|
|
156
|
+
}
|
|
157
|
+
]
|
|
158
|
+
}
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
### Archetype 3: Battery Longevity Estimation Model
|
|
162
|
+
```python
|
|
163
|
+
def estimate_tag_battery_years(
|
|
164
|
+
updates_per_day: float,
|
|
165
|
+
ambient_temp_celsius: float = 21.0,
|
|
166
|
+
battery_mah: float = 1200.0,
|
|
167
|
+
uAh_per_refresh: float = 18.0,
|
|
168
|
+
standby_uA: float = 1.2
|
|
169
|
+
) -> float:
|
|
170
|
+
# Temperature de-rating factor (freezer sections de-rate capacity significantly)
|
|
171
|
+
temp_factor = 0.65 if ambient_temp_celsius < 0.0 else (0.85 if ambient_temp_celsius < 10.0 else 1.0)
|
|
172
|
+
effective_mah = battery_mah * 0.85 * temp_factor # 85% usable chemistry margin
|
|
173
|
+
|
|
174
|
+
daily_drain_uAh = (updates_per_day * uAh_per_refresh) + (standby_uA * 24.0)
|
|
175
|
+
daily_drain_mAh = daily_drain_uAh / 1000.0
|
|
176
|
+
|
|
177
|
+
lifespan_days = effective_mah / daily_drain_mAh
|
|
178
|
+
return round(lifespan_days / 365.25, 2)
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
## The Pre-Emit Quality Gate Checklist
|
|
182
|
+
|
|
183
|
+
Before finalizing any ESL integration, sync workflow, or audit output:
|
|
184
|
+
- [ ] **Delta Discipline**: Confirmed that only changed row hashes are queued; no scheduled full catalog pushes.
|
|
185
|
+
- [ ] **Idempotency Proof**: Monotonic versioning and idempotency keys attached to every modification payload.
|
|
186
|
+
- [ ] **5-Layer Traceability**: Explicit separation between ERP status, server queue, and base station physical tag ACK.
|
|
187
|
+
- [ ] **RF Rate-Limiting**: Gateway transmission queued at $\le 150$ updates/min per AP.
|
|
188
|
+
- [ ] **Battery Budget Checked**: Verified tag refresh schedule does not breach 3 updates/day lifespan limit.
|
|
189
|
+
- [ ] **Layout Safety**: Text width and font truncation verified against display canvas dimensions.
|
|
190
|
+
- [ ] **Dry-Run Audit Default**: Executed read-only reconciliation prior to proposing live write broadcasts.
|
|
191
|
+
|
|
192
|
+
## Self-Correction & Verification Loop
|
|
193
|
+
|
|
194
|
+
### Bad (Vibe-Coded AI Slop Output):
|
|
195
|
+
> "To update the electronic labels, run a Python script that loops over all 50,000 products in your database and POSTs them to the ZKONG API every hour. If prices are wrong on the shelves, just run the script again."
|
|
196
|
+
*Problems: Fatal catalog flooding, battery destruction, zero delta tracking, zero idempotency, ignores gateway rate-limiting, blind to physical tag ACK status.*
|
|
197
|
+
|
|
198
|
+
### Good (Production ESL Price Sync Output):
|
|
199
|
+
> 1. Execute read-only audit comparing ERP pricing against ESL gateway ACK states; identify 12 divergent SKUs.
|
|
200
|
+
> 2. Compute SHA256 delta hashes for all modified SKUs since watermark `2026-09-22T08:00:00Z`.
|
|
201
|
+
> 3. Generate idempotent mutation payloads with `UUIDv5` tokens and monotonic timestamp versioning.
|
|
202
|
+
> 4. Verify tag layout: Confirm that promo prices fit within the 2.9-inch 3-color e-paper template without numeric truncation.
|
|
203
|
+
> 5. Queue rate-limited dispatch at 120 tags/min on Base Station 03, logging transmission to physical tag ACK confirmation.
|
|
204
|
+
|
|
205
|
+
## Limitations
|
|
206
|
+
|
|
207
|
+
- ESL APIs, radio limits, acknowledgements, battery figures, and rate limits depend on the vendor, firmware, tag model, and local radio rules. Confirm each value in the deployed system's documentation before use.
|
|
208
|
+
- Start with read-only reconciliation and a small authorized pilot. A server acknowledgement does not prove the physical label displayed the intended value; require tag-level confirmation and a rollback plan before wider writes.
|
|
209
|
+
- Protect retail pricing, product, and device identifiers as business data. Do not transmit credentials or customer data in diagnostic logs or reports.
|
|
210
|
+
- The scripts and payloads in this guide are illustrative and are not vendor-certified integrations.
|
|
@@ -0,0 +1,181 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: marlin-bed-leveling
|
|
3
|
+
description: "Calibrates Marlin 2.x 3D printer firmware bed leveling: Unified Bed Leveling (UBL), Bilinear ABL, M420 S1 post-homing, Z-probe offsets, G26 mesh prints, and EEPROM slots. Trigger phrases: marlin bed leveling, ubl calibration, m420 s1, z probe offset, g29 bed level, first layer adhesion."
|
|
4
|
+
category: development
|
|
5
|
+
risk: safe
|
|
6
|
+
source: community
|
|
7
|
+
source_repo: wwewtech/marlin-bed-leveling
|
|
8
|
+
source_type: community
|
|
9
|
+
date_added: "2026-09-22"
|
|
10
|
+
author: wwewtech
|
|
11
|
+
tags: [3d-printing, marlin-firmware, gcode, hardware-calibration, fdm, additive-manufacturing]
|
|
12
|
+
tools: [claude, cursor, gemini, windsurf]
|
|
13
|
+
license: "MIT"
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# Marlin Bed Leveling: Deterministic Calibration & First-Layer Mesh Engineering
|
|
17
|
+
|
|
18
|
+
Master Marlin 2.x bed leveling architectures (Unified Bed Leveling UBL, Bilinear ABL, Manual Mesh MBL) to guarantee flawless first-layer adhesion through rigorous G-code phase workflows, probe offset math, and persistent EEPROM compensation.
|
|
19
|
+
|
|
20
|
+
## When to Use This Skill
|
|
21
|
+
|
|
22
|
+
Activate this skill when:
|
|
23
|
+
- Calibrating, tuning, or troubleshooting automatic bed leveling on Marlin 2.x 3D printers (Ender 3, CR-10, Prusa clones, custom CoreXY / Cartesian machines).
|
|
24
|
+
- The user asks: "How do I set up UBL in Marlin?", "My printer homes with G28 and ignores the mesh", "Calculate BLTouch Z-probe offset", "Generate G26 validation print G-code", or "Edit warped mesh points with M421".
|
|
25
|
+
- Diagnosing first-layer defects: nozzle dragging across PEI sheets, corner detachment, uneven squish, or compensation disabling after homing.
|
|
26
|
+
- Sizing slicer start G-code sequences (Cura, PrusaSlicer, OrcaSlicer, Bambu Studio) with thermal soaking and mesh restoration.
|
|
27
|
+
|
|
28
|
+
Do NOT use this skill when:
|
|
29
|
+
- Calibrating Klipper or RepRapFirmware printers (different firmware syntax and mesh compensation systems).
|
|
30
|
+
- Fixing gross mechanical skew: loose V-slot rollers, bent Z leadscrews, or wobbly gantry arms (hardware faults cannot be solved by software leveling).
|
|
31
|
+
- Delta printer kinematics setup requiring radius and tower endstop adjustments before mesh probing.
|
|
32
|
+
|
|
33
|
+
## Core Mental Models & Non-Negotiable Rules
|
|
34
|
+
|
|
35
|
+
1. **The Post-Homing Enable Mandate (`G28` Disables Leveling)**:
|
|
36
|
+
- In Marlin firmware, executing `G28` (Home All Axes) automatically **DISABLES bed leveling compensation by default** (`RESTORE_LEVELING_AFTER_G28` is often disabled in vendor firmware).
|
|
37
|
+
- In every print start G-code, the leveling restoration command MUST be issued **strictly AFTER `G28`**:
|
|
38
|
+
```gcode
|
|
39
|
+
G28 ; Home all axes (silently turns leveling OFF)
|
|
40
|
+
M420 S1 ; Re-enable bed leveling compensation from EEPROM mesh
|
|
41
|
+
```
|
|
42
|
+
- Failing to include `M420 S1` (or `G29 A` for UBL) after `G28` causes the printer to execute the entire print with zero leveling compensation.
|
|
43
|
+
|
|
44
|
+
2. **The 5-Phase UBL Calibration Loop**:
|
|
45
|
+
Unified Bed Leveling (UBL) is a state machine requiring five sequential phases:
|
|
46
|
+
- **Phase 1 (Probe Reachable Grid)**: `G29 P1` probes all coordinates physically reachable by the probe offset.
|
|
47
|
+
- **Phase 2 (Manual Edge Probing)**: `G29 P2` uses a paper feeler gauge for points the BLTouch cannot reach due to gantry geometry.
|
|
48
|
+
- **Phase 3 (Mathematical Extrapolation)**: `G29 P3 T0.0` fills unprobed edge coordinates by extrapolating slopes from adjacent measured points.
|
|
49
|
+
- **Phase 4 (Validation & Micro-Tuning)**: `G26 B60 H210 F1.75` prints a single-layer validation test pattern; inspect and edit coordinates with `M421 I... J... Q...`.
|
|
50
|
+
- **Phase 5 (Storage & Activation)**: `G29 S1` saves to EEPROM mesh slot 1; `G29 A` activates UBL; `M500` commits to permanent EEPROM storage.
|
|
51
|
+
|
|
52
|
+
3. **Thermal Equilibrium Pre-Flight Invariant**:
|
|
53
|
+
- Aluminum build plates, magnetic stickers, and spring-steel PEI sheets expand and warp non-linearly under thermal expansion.
|
|
54
|
+
- Probing a bed at room temperature (20°C) and then printing at 60°C or 100°C bakes a $0.08\text{--}0.20\text{mm}$ error into every mesh point.
|
|
55
|
+
- **Mandatory Pre-Flight**: Beds MUST be heated to target operating temperature and soaked for $\ge 5\text{ minutes}$ prior to running `G29` probing.
|
|
56
|
+
|
|
57
|
+
4. **Z-Probe Offset Sign Convention (`M851 Z...`)**:
|
|
58
|
+
- For downward-deploying probes (BLTouch, CRTouch, inductive sensors) where the probe trigger point triggers *below* the nozzle tip:
|
|
59
|
+
$$\mathbf{Z\text{-}Offset\ is\ strictly\ NEGATIVE}\ (e.g.,\ \text{M851 Z-1.85})$$
|
|
60
|
+
- A *positive* Z-offset on a downward-deploying probe instructs the firmware that the nozzle is lower than the probe trigger, driving the nozzle directly into the bed on the first move.
|
|
61
|
+
- Adjusting Z-offset with paper gauge:
|
|
62
|
+
- More resistance / tighter squish $\implies$ more negative (e.g., `-1.80` $\to$ `-1.85`).
|
|
63
|
+
- Less resistance / looser gap $\implies$ less negative (e.g., `-1.80` $\to$ `-1.75`).
|
|
64
|
+
|
|
65
|
+
5. **Mesh Fade Height Geometry (`M420 Z...`)**:
|
|
66
|
+
- Leveling compensation must gradually fade out as Z-height increases so top surface layers print geometrically flat rather than conforming to the bed warp.
|
|
67
|
+
- Set fade height to `M420 Z10.0` (fades compensation smoothly over the first 10mm of print height). Setting `Z0` disables fade (perpetuating bed ripples to the top of the model); setting `Z > 25` is unnecessary ballast.
|
|
68
|
+
|
|
69
|
+
## Named Sins & Anti-Patterns (Что категорически ЗАПРЕЩЕНО)
|
|
70
|
+
|
|
71
|
+
| Anti-Pattern | Manifestation in Code/Workflow | Mandatory Production Counter-Rule |
|
|
72
|
+
| :--- | :--- | :--- |
|
|
73
|
+
| **The Forgotten `M420 S1`** | `G28` followed immediately by print moves. | Always place `M420 S1` (or `G29 A`) immediately after `G28`. |
|
|
74
|
+
| **Cold Bed Mesh Probing** | Running `G29 P1` with bed heater at 20°C. | Heat bed to printing temperature and soak for $\ge 5$ minutes first. |
|
|
75
|
+
| **Positive Z-Probe Offset** | Setting `M851 Z+1.5` on a BLTouch probe. | Downward-deploying probes MUST have a negative Z-offset (`M851 Z-...`). |
|
|
76
|
+
| **Unsaved Mesh Discard** | Probing a 7x7 mesh without running `M500`. | Always commit mesh to EEPROM via `G29 S1` followed by `M500`. |
|
|
77
|
+
| **Software-Fixing Physical Tilt** | Compensating for a 2.5mm physical gantry tilt with UBL. | Physically tram bed corners (`G35` or thumb screws) before probing. |
|
|
78
|
+
| **Blind Unchecked Extrapolation** | Running `G29 P3` without verifying boundary values. | Inspect extrapolated border points with `M420 V` or `G29 T`. |
|
|
79
|
+
| **Cold Extruder G26 Validation** | Running G26 test without nozzle heating. | Heat nozzle to printing temperature (`G26 B60 H210 F1.75`). |
|
|
80
|
+
| **Loose Probe Mount Blindness** | Probing while BLTouch bracket has physical wobble. | Check physical probe mount rigidity; probe repeatability requires $s \le 0.005\text{mm}$ (`M48`). |
|
|
81
|
+
| **Excessive Fade Height** | Setting `M420 Z0` or `M420 Z50`. | Use `M420 Z10.0` to smoothly transition from warped bed to true flat geometry. |
|
|
82
|
+
| **Blind Point Over-Editing** | Manually offsetting a single mesh point by 0.5mm via `M421`. | Clean the build plate or check for debris under PEI sheet before altering matrix. |
|
|
83
|
+
|
|
84
|
+
## Concrete Archetypes / Presets
|
|
85
|
+
|
|
86
|
+
### Archetype 1: End-to-End Unified Bed Leveling (UBL) Calibration Macro
|
|
87
|
+
```gcode
|
|
88
|
+
; ==============================================================================
|
|
89
|
+
; MARLIN 2.x UBL FULL CALIBRATION SEQUENCE
|
|
90
|
+
; ==============================================================================
|
|
91
|
+
M140 S60 ; Start heating bed to 60°C
|
|
92
|
+
M190 S60 ; Wait for bed to reach 60°C
|
|
93
|
+
G4 S300 ; Thermal soak for 300 seconds (5 minutes)
|
|
94
|
+
|
|
95
|
+
G28 ; Home all axes
|
|
96
|
+
M117 Probing Reachable Points...
|
|
97
|
+
G29 P1 ; Phase 1: Automated probe of reachable grid points
|
|
98
|
+
|
|
99
|
+
M117 Extrapolating Mesh Edges...
|
|
100
|
+
G29 P3 T0.0 ; Phase 3: Mathematically extrapolate unprobed perimeter points
|
|
101
|
+
G29 P3 T0.0 ; Second pass to ensure corner fill
|
|
102
|
+
|
|
103
|
+
G29 T ; Print ASCII topology matrix to terminal for verification
|
|
104
|
+
G29 S1 ; Save calibrated mesh to EEPROM slot 1
|
|
105
|
+
G29 A ; Activate UBL leveling system
|
|
106
|
+
M500 ; Save EEPROM permanently to board flash
|
|
107
|
+
M117 UBL Mesh 1 Saved and Activated!
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
### Archetype 2: Production Slicer Start G-Code (Cura / PrusaSlicer / Orca)
|
|
111
|
+
```gcode
|
|
112
|
+
; ==============================================================================
|
|
113
|
+
; PRODUCTION SLICER START G-CODE (MARLIN UBL / BILINEAR)
|
|
114
|
+
; ==============================================================================
|
|
115
|
+
M140 S[first_layer_bed_temperature] ; Start heating bed
|
|
116
|
+
M104 S150 ; Pre-heat nozzle to 150°C (prevent oozing)
|
|
117
|
+
M190 S[first_layer_bed_temperature] ; Wait for bed to stabilize
|
|
118
|
+
|
|
119
|
+
G28 ; Home all axes (turns leveling OFF)
|
|
120
|
+
M420 S1 Z10.0 ; Restore mesh from EEPROM & set 10mm fade
|
|
121
|
+
; (OR for UBL: G29 A followed by G29 L1)
|
|
122
|
+
|
|
123
|
+
G1 Z50 F3000 ; Move nozzle up to safe clearance
|
|
124
|
+
M109 S[first_layer_temperature] ; Heat nozzle to final printing temp
|
|
125
|
+
|
|
126
|
+
; Priming Purge Line on bed edge
|
|
127
|
+
G92 E0 ; Reset extruder
|
|
128
|
+
G1 X0.1 Y20 Z0.28 F5000.0 ; Move to start position
|
|
129
|
+
G1 X0.1 Y200.0 Z0.28 F1500.0 E15 ; Draw first purge line
|
|
130
|
+
G1 X0.4 Y200.0 Z0.28 F5000.0 ; Move to side
|
|
131
|
+
G1 X0.4 Y20 Z0.28 F1500.0 E30 ; Draw second purge line
|
|
132
|
+
G92 E0 ; Reset extruder
|
|
133
|
+
G1 Z2.0 F3000 ; Z-hop before moving to print
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
### Archetype 3: Probe Repeatability & Offset Verification (`M48` & `M851`)
|
|
137
|
+
```gcode
|
|
138
|
+
; ==============================================================================
|
|
139
|
+
; PROBE REPEATABILITY & Z-OFFSET CALIBRATION
|
|
140
|
+
; ==============================================================================
|
|
141
|
+
G28 ; Home axes
|
|
142
|
+
G1 X117.5 Y117.5 Z10 F5000 ; Move to center of 235x235 bed
|
|
143
|
+
M48 P10 X117.5 Y117.5 V2 E L2 ; Run 10-point repeatability test
|
|
144
|
+
; Standard deviation must be <= 0.005 mm. If > 0.010 mm, tighten probe mount!
|
|
145
|
+
|
|
146
|
+
; Manual Z-Offset adjustment with paper feeler gauge:
|
|
147
|
+
; 1. Move to Z=0: G1 Z0 F1000
|
|
148
|
+
; 2. Baby-step Z until paper drags with slight friction
|
|
149
|
+
; 3. If nozzle moved down 0.15mm: M851 Z-1.85 (subtract 0.15 from current -1.70)
|
|
150
|
+
; 4. Commit: M500
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
## The Pre-Emit Quality Gate Checklist
|
|
154
|
+
|
|
155
|
+
Before emitting any Marlin leveling advice, macro, or slicer start code:
|
|
156
|
+
- [ ] **Post-Homing Verification**: `M420 S1` (or `G29 A`) placed strictly after `G28` in the execution flow.
|
|
157
|
+
- [ ] **Thermal Soak Pre-Flight**: Bed heating and 5-minute soak mandated prior to `G29` probing.
|
|
158
|
+
- [ ] **Negative Offset Math**: Downward-deploying probes have verified negative `M851 Z-...` values.
|
|
159
|
+
- [ ] **EEPROM Commit**: `G29 S1` followed immediately by `M500` to preserve the mesh.
|
|
160
|
+
- [ ] **Fade Height Limit**: `M420 Z10.0` configured for smooth transition to flat geometry.
|
|
161
|
+
- [ ] **Physical Tramming Precedence**: Confirmed that manual corner leveling is performed before software compensation.
|
|
162
|
+
- [ ] **Probe Repeatability**: Confirmed that `M48` probe standard deviation is $\le 0.005\text{mm}$.
|
|
163
|
+
|
|
164
|
+
## Self-Correction & Verification Loop
|
|
165
|
+
|
|
166
|
+
### Bad (Vibe-Coded AI Slop Output):
|
|
167
|
+
> "Put `G29` before `G28` in your start G-code. If your BLTouch is scraping, set `M851 Z1.5`. You don't need to heat the bed during leveling because Marlin compensates for it automatically."
|
|
168
|
+
*Problems: Fatal G28 homing wipes out the preceding G29 mesh, positive Z-offset crashes nozzle into bed, dangerous cold probing advice.*
|
|
169
|
+
|
|
170
|
+
### Good (Production Marlin Bed Leveling Output):
|
|
171
|
+
> 1. Tram the four bed corners manually using `G35` or thumb screws to get mechanical variance under $\pm 0.1\text{mm}$.
|
|
172
|
+
> 2. Heat bed to 60°C and nozzle to 150°C; let soak for 5 minutes so thermal expansion stabilizes.
|
|
173
|
+
> 3. Verify probe repeatability using `M48 P10` (standard deviation must be $\le 0.005\text{mm}$).
|
|
174
|
+
> 4. Probe grid with `G29 P1`, extrapolate unprobed corners with `G29 P3 T0.0`, save to slot 1 with `G29 S1`, and commit to flash with `M500`.
|
|
175
|
+
> 5. In your slicer start G-code, place `M420 S1 Z10.0` immediately after `G28` to restore the active mesh with a 10mm fade height.
|
|
176
|
+
|
|
177
|
+
## Limitations
|
|
178
|
+
|
|
179
|
+
- Marlin commands and mesh behavior vary with firmware version, configuration, probe, bed geometry, and slicer. Check the exact firmware build and printer documentation before applying a macro or offset.
|
|
180
|
+
- G-code can move a hot toolhead and damage equipment or cause burns. Supervise calibration, keep hands clear, use the printer's emergency stop when motion is unsafe, and verify travel limits before running generated sequences.
|
|
181
|
+
- Example coordinates, temperatures, tolerances, and fade values are illustrative; measure the actual printer and follow the manufacturer’s limits.
|
|
@@ -0,0 +1,205 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: oneroster-csv-validator
|
|
3
|
+
description: "Validates 1EdTech / IMS Global OneRoster v1.1 and v1.2 CSV roster sets: manifest integrity, bulk vs delta strictness, foreign key references, and encoding sanitization. Trigger phrases: oneroster csv validator, validate oneroster zip, clever roster error, oneroster sourcedid."
|
|
4
|
+
category: testing
|
|
5
|
+
risk: safe
|
|
6
|
+
source: community
|
|
7
|
+
source_repo: wwewtech/oneroster-csv-validator
|
|
8
|
+
source_type: community
|
|
9
|
+
date_added: "2026-09-22"
|
|
10
|
+
author: wwewtech
|
|
11
|
+
tags: [edtech, oneroster, csv-validation, ims-global, 1edtech, data-engineering]
|
|
12
|
+
tools: [claude, cursor, gemini, windsurf]
|
|
13
|
+
license: "MIT"
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
# OneRoster CSV Validator: Strict 1EdTech Data Integrity & Pre-Flight Linter
|
|
17
|
+
|
|
18
|
+
Perform rigorous pre-flight validation on OneRoster v1.1 and v1.2 CSV roster bundles prior to SIS/Clever/ClassLink ingestion: audit foreign key referential integrity, eliminate orphan enrollments, enforce bulk vs delta partitioning, and sanitize encoding traps.
|
|
19
|
+
|
|
20
|
+
## When to Use This Skill
|
|
21
|
+
|
|
22
|
+
Activate this skill when:
|
|
23
|
+
- Validating or debugging student information system (SIS) roster export packages (`.zip` archives or CSV directories) for OneRoster v1.1 or v1.2 compliance.
|
|
24
|
+
- The user asks: "Why is Clever rejecting our OneRoster zip file?", "Check for orphaned userSourcedIds in enrollments.csv", "Validate bulk vs delta mode in OneRoster CSVs", or "Sanitize UTF-8 BOM characters from school rosters".
|
|
25
|
+
- Auditing relational integrity across `manifest.csv`, `orgs.csv`, `users.csv`, `courses.csv`, `classes.csv`, `enrollments.csv`, and `academicSessions.csv`.
|
|
26
|
+
- Preparing automated pre-flight CI/CD pipelines to catch corrupted student or teacher records before nightly SIS sync runs.
|
|
27
|
+
|
|
28
|
+
Do NOT use this skill when:
|
|
29
|
+
- Integrating OneRoster REST / OAuth 2.0 API endpoints (this skill focuses strictly on CSV table bindings).
|
|
30
|
+
- Generating synthetic or fabricated student PII without authorization.
|
|
31
|
+
- Working on generic non-standard CSV spreadsheets that do not follow 1EdTech specifications.
|
|
32
|
+
|
|
33
|
+
## Core Mental Models & Non-Negotiable Rules
|
|
34
|
+
|
|
35
|
+
1. **The Manifest Root Authority Law**:
|
|
36
|
+
- `manifest.csv` MUST reside strictly at the root level of the ZIP archive (never nested inside a subfolder).
|
|
37
|
+
- It MUST declare the exact specification version: `oneroster.version,1.1` or `oneroster.version,1.2`.
|
|
38
|
+
- It MUST list every single file present in the exchange. If a CSV file is present in the archive but omitted from `manifest.csv`, or listed in `manifest.csv` but missing from the zip, the package is rejected.
|
|
39
|
+
|
|
40
|
+
2. **The Strict Bulk vs Delta Partition**:
|
|
41
|
+
- One exchange bundle = strictly ONE operational mode.
|
|
42
|
+
- **Bulk Exchange Mode**: Represents a complete master snapshot. In OneRoster 1.1, rows MUST NOT contain `dateLastModified` or `status` columns. In 1.2, all status fields MUST be `active`.
|
|
43
|
+
- **Delta Exchange Mode**: Represents incremental changes. Every row MUST contain a valid ISO 8601 UTC timestamp (`YYYY-MM-DDTHH:MM:SS.sssZ`) in `dateLastModified` and a valid status (`active` or `tobedeleted`).
|
|
44
|
+
- Mixing bulk and delta formats within the same archive is a fatal validation failure.
|
|
45
|
+
|
|
46
|
+
3. **The Foreign Key Referential Integrity Hierarchy**:
|
|
47
|
+
- Relational dependency chain:
|
|
48
|
+
$$\mathbf{orgs.csv} \longleftarrow \mathbf{courses.csv} \longleftarrow \mathbf{classes.csv} \longleftarrow \mathbf{enrollments.csv} \longrightarrow \mathbf{users.csv}$$
|
|
49
|
+
- Every `schoolSourcedId` in `users.csv` and `classes.csv` MUST resolve to an existing `sourcedId` in `orgs.csv`.
|
|
50
|
+
- Every `courseSourcedId` in `classes.csv` MUST resolve to `courses.csv`.
|
|
51
|
+
- Every `userSourcedId` and `classSourcedId` in `enrollments.csv` MUST resolve to active primary keys in `users.csv` and `classes.csv`.
|
|
52
|
+
- **Zero Orphan Tolerance**: A single orphan link (e.g. enrolling a deleted student ID) invalidates the batch.
|
|
53
|
+
|
|
54
|
+
4. **Strict Role Enum Constraint Axioms**:
|
|
55
|
+
- In `users.csv`: `role` MUST strictly match the OneRoster enum:
|
|
56
|
+
- v1.1: `administrator`, `proctor`, `student`, `teacher`.
|
|
57
|
+
- v1.2: adds `aide`, `guardian`, `parent`, `staff`.
|
|
58
|
+
- In `enrollments.csv`: `role` is strictly constrained to `administrator`, `proctor`, `student`, `teacher`. Custom vendor roles (`substitute`, `dean`) must be mapped to valid specification enums.
|
|
59
|
+
- Primary `sourcedId` values MUST be case-sensitively unique strings (RFC 4122 UUID format strongly recommended).
|
|
60
|
+
|
|
61
|
+
5. **RFC 4180 CSV & UTF-8 BOM Sanitization**:
|
|
62
|
+
- Files MUST be encoded in **UTF-8 without Byte Order Mark (BOM)**.
|
|
63
|
+
- Microsoft Excel frequently prepends `\xEF\xBB\xBF` to CSV exports, corrupting the first column name (`sourcedId` $\neq$ `sourcedId`).
|
|
64
|
+
- Fields containing commas, double-quotes, or newlines MUST be enclosed in double quotes (`"`). Literal quotes inside fields must be escaped as `""`.
|
|
65
|
+
|
|
66
|
+
## Named Sins & Anti-Patterns (Что категорически ЗАПРЕЩЕНО)
|
|
67
|
+
|
|
68
|
+
| Anti-Pattern | Manifestation in CSV Packages | Mandatory Production Counter-Rule |
|
|
69
|
+
| :--- | :--- | :--- |
|
|
70
|
+
| **Nested Archive Packaging** | Zipping a parent folder (`district_roster/manifest.csv`). | Manifest and CSVs MUST reside at the archive root (`/manifest.csv`). |
|
|
71
|
+
| **The UTF-8 BOM Trap** | Excel export injecting `\xEF\xBB\xBF` into `sourcedId`. | Strip BOM headers during pre-flight sanitization pass. |
|
|
72
|
+
| **Orphan Enrollment Links** | `enrollments.csv` pointing to non-existent user IDs. | Enforce foreign key validation across all entity files before import. |
|
|
73
|
+
| **Bulk / Delta Column Bleed** | Including `dateLastModified` or `tobedeleted` in bulk. | Enforce schema separation: bulk files must not contain delta headers. |
|
|
74
|
+
| **Circular Org Hierarchies** | School A listed as parent of School B, and vice-versa. | Perform directed acyclic graph (DAG) cycle detection on `orgs.csv`. |
|
|
75
|
+
| **Illegal Role Strings** | Using non-standard strings like `counselor` or `sub`. | Map all roles to standard enums (`administrator`, `teacher`, `student`). |
|
|
76
|
+
| **Malformed ISO 8601 Timestamps** | Writing `"09/22/2026 14:00"` instead of ISO 8601. | Enforce strict RFC 3339 UTC format: `YYYY-MM-DDTHH:MM:SS.sssZ`. |
|
|
77
|
+
| **Unescaped CSV Commas** | Unquoted commas in user names (`Doe, John`) shifting columns. | Wrap text fields containing commas in standard RFC 4180 quotes. |
|
|
78
|
+
| **Missing Mandatory Core Files** | Exporting a bulk zip without `academicSessions.csv`. | Confirm all 7 mandatory core files exist in zip and manifest. |
|
|
79
|
+
| **Hallucinated SourcedIDs** | Inventing arbitrary IDs during reconciliation. | Preserve authoritative SIS primary keys without synthetic generation. |
|
|
80
|
+
|
|
81
|
+
## Concrete Archetypes / Presets
|
|
82
|
+
|
|
83
|
+
### Archetype 1: Pre-Flight Python Referential Integrity Linter
|
|
84
|
+
```python
|
|
85
|
+
import csv
|
|
86
|
+
import io
|
|
87
|
+
import re
|
|
88
|
+
import zipfile
|
|
89
|
+
from typing import Dict, List, Set
|
|
90
|
+
|
|
91
|
+
CORE_FILES = [
|
|
92
|
+
"manifest.csv", "orgs.csv", "users.csv", "courses.csv",
|
|
93
|
+
"classes.csv", "enrollments.csv", "academicSessions.csv"
|
|
94
|
+
]
|
|
95
|
+
VALID_ROLES_1_1 = {"administrator", "proctor", "student", "teacher"}
|
|
96
|
+
|
|
97
|
+
def validate_oneroster_archive(zip_path: str) -> dict:
|
|
98
|
+
errors = []
|
|
99
|
+
warnings = []
|
|
100
|
+
|
|
101
|
+
with zipfile.ZipFile(zip_path, 'r') as z:
|
|
102
|
+
names = z.namelist()
|
|
103
|
+
|
|
104
|
+
# 1. Manifest Root Check
|
|
105
|
+
if "manifest.csv" not in names:
|
|
106
|
+
return {"status": "FATAL", "errors": ["manifest.csv missing from zip root."]}
|
|
107
|
+
|
|
108
|
+
# 2. Extract and Parse Core Sets
|
|
109
|
+
data: Dict[str, List[dict]] = {}
|
|
110
|
+
keys: Dict[str, Set[str]] = {}
|
|
111
|
+
|
|
112
|
+
for filename in CORE_FILES:
|
|
113
|
+
if filename in names:
|
|
114
|
+
raw_bytes = z.read(filename)
|
|
115
|
+
# Strip UTF-8 BOM if present
|
|
116
|
+
if raw_bytes.startswith(b'\xef\xbb\xbf'):
|
|
117
|
+
warnings.append(f"{filename} contains UTF-8 BOM; stripped automatically.")
|
|
118
|
+
raw_bytes = raw_bytes[3:]
|
|
119
|
+
|
|
120
|
+
reader = csv.DictReader(io.StringIO(raw_bytes.decode('utf-8', errors='replace')))
|
|
121
|
+
rows = list(reader)
|
|
122
|
+
data[filename] = rows
|
|
123
|
+
if rows and "sourcedId" in rows[0]:
|
|
124
|
+
keys[filename] = {r["sourcedId"] for r in rows if "sourcedId" in r}
|
|
125
|
+
else:
|
|
126
|
+
errors.append(f"Mandatory core file missing: {filename}")
|
|
127
|
+
|
|
128
|
+
# 3. Foreign Key Checks
|
|
129
|
+
if "orgs.csv" in keys and "users.csv" in data:
|
|
130
|
+
for r in data["users.csv"]:
|
|
131
|
+
org_ref = r.get("orgSourcedIds") or r.get("schoolSourcedId")
|
|
132
|
+
if org_ref and org_ref not in keys["orgs.csv"]:
|
|
133
|
+
errors.append(f"users.csv: user {r.get('sourcedId')} references non-existent org {org_ref}")
|
|
134
|
+
|
|
135
|
+
if "users.csv" in keys and "classes.csv" in keys and "enrollments.csv" in data:
|
|
136
|
+
for r in data["enrollments.csv"]:
|
|
137
|
+
uid = r.get("userSourcedId")
|
|
138
|
+
cid = r.get("classSourcedId")
|
|
139
|
+
if uid not in keys["users.csv"]:
|
|
140
|
+
errors.append(f"enrollments.csv: orphan enrollment references missing user {uid}")
|
|
141
|
+
if cid not in keys["classes.csv"]:
|
|
142
|
+
errors.append(f"enrollments.csv: orphan enrollment references missing class {cid}")
|
|
143
|
+
|
|
144
|
+
role = r.get("role")
|
|
145
|
+
if role not in VALID_ROLES_1_1:
|
|
146
|
+
errors.append(f"enrollments.csv: illegal role '{role}' on user {uid}")
|
|
147
|
+
|
|
148
|
+
return {
|
|
149
|
+
"status": "FAILED" if errors else "PASSED",
|
|
150
|
+
"errors": errors,
|
|
151
|
+
"warnings": warnings,
|
|
152
|
+
"total_records_checked": sum(len(v) for v in data.values())
|
|
153
|
+
}
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
### Archetype 2: Minimal Valid OneRoster v1.1 Manifest
|
|
157
|
+
```csv
|
|
158
|
+
propertyName,value
|
|
159
|
+
oneroster.version,1.1
|
|
160
|
+
file.orgs,bulk
|
|
161
|
+
file.users,bulk
|
|
162
|
+
file.courses,bulk
|
|
163
|
+
file.classes,bulk
|
|
164
|
+
file.enrollments,bulk
|
|
165
|
+
file.academicSessions,bulk
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
### Archetype 3: Delta Synchronization Row Example
|
|
169
|
+
```csv
|
|
170
|
+
sourcedId,status,dateLastModified,userSourcedId,classSourcedId,role,primary
|
|
171
|
+
enr_99201,active,2026-09-22T08:30:00.000Z,usr_0421,cls_bio101,student,true
|
|
172
|
+
enr_99202,tobedeleted,2026-09-22T08:30:00.000Z,usr_0884,cls_bio101,student,false
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
## The Pre-Emit Quality Gate Checklist
|
|
176
|
+
|
|
177
|
+
Before emitting validation results, CSV transforms, or roster scripts:
|
|
178
|
+
- [ ] **Root Location**: Confirmed `manifest.csv` is located strictly at `/` without folder nesting.
|
|
179
|
+
- [ ] **BOM Scrubbing**: Verified all CSV headers are clean UTF-8 without `\xEF\xBB\xBF`.
|
|
180
|
+
- [ ] **Mode Consistency**: Verified no delta columns exist in bulk packages, and timestamps exist in deltas.
|
|
181
|
+
- [ ] **Referential Integrity**: 100% check of `userSourcedId`, `classSourcedId`, `orgSourcedIds`, and `courseSourcedId`.
|
|
182
|
+
- [ ] **Role Enum Validation**: Confirmed all role values match 1EdTech specifications.
|
|
183
|
+
- [ ] **Cycle-Free Orgs**: DAG validation run on `orgs.csv` to ensure no circular parent references.
|
|
184
|
+
- [ ] **RFC 4180 Escaping**: Commas, quotes, and multi-line values enclosed in double quotes.
|
|
185
|
+
|
|
186
|
+
## Self-Correction & Verification Loop
|
|
187
|
+
|
|
188
|
+
### Bad (Vibe-Coded AI Slop Output):
|
|
189
|
+
> "Zip your CSV folder up and upload it. If Clever says there is a missing manifest, just add a blank manifest.csv inside the folder. If some student IDs are missing in enrollments, just delete those rows or invent temporary user IDs."
|
|
190
|
+
*Problems: Nested zip breaks root discovery, blank manifest violates schema, inventing fake IDs corrupts student records, ignores UTF-8 BOM.*
|
|
191
|
+
|
|
192
|
+
### Good (Production OneRoster CSV Validator Output):
|
|
193
|
+
> 1. Extract zip to verify `/manifest.csv` exists at the root path; confirm `oneroster.version` is declared as `1.1`.
|
|
194
|
+
> 2. Scan all CSV headers: detect and strip UTF-8 BOM (`\xEF\xBB\xBF`) from `users.csv` to prevent header corruption.
|
|
195
|
+
> 3. Perform foreign key referential integrity pass:
|
|
196
|
+
> - Flag 4 orphan rows in `enrollments.csv` referencing deleted student ID `usr_9914`.
|
|
197
|
+
> - Flag invalid role `"substitute_teacher"` on row 14 of `enrollments.csv` (must map to `"teacher"`).
|
|
198
|
+
> 4. Verify bulk purity: confirm no `dateLastModified` or `tobedeleted` status flags are present in bulk mode.
|
|
199
|
+
> 5. Emit clean, line-numbered pre-flight audit report before committing to SIS ingestion.
|
|
200
|
+
|
|
201
|
+
## Limitations
|
|
202
|
+
|
|
203
|
+
- OneRoster version details and required files can differ by exchange profile and vendor implementation. Validate against the exact 1EdTech specification and recipient's current import contract before rejecting or transforming a roster.
|
|
204
|
+
- Student and staff roster files contain sensitive education records. Process only with authorization, minimize retained data, redact identifiers in diagnostics, and do not upload source CSVs to external services.
|
|
205
|
+
- The included snippets are illustrative, not a complete certified validator; for example, large files, namespaces, delta semantics, and vendor extensions require additional handling and tests.
|
|
@@ -172,9 +172,35 @@ Making money from Mini Apps
|
|
|
172
172
|
| TON payments | Premium features | High |
|
|
173
173
|
| In-app purchases | Virtual goods | High |
|
|
174
174
|
| Ads (Telegram Ads) | Display ads | Medium |
|
|
175
|
+
| In-app ad networks (e.g. TADS) | Native / fullscreen / rewarded ads inside the Mini App | Medium |
|
|
175
176
|
| Referral | Share to earn | Medium |
|
|
176
177
|
| NFT sales | Digital collectibles | High |
|
|
177
178
|
|
|
179
|
+
### In-app ads (rewarded and native)
|
|
180
|
+
|
|
181
|
+
Ad networks built for Mini Apps serve ads inside the app itself, so you keep the user in the flow and can reward them (extra life, coins) for a click or a full view. Example with [TADS](https://tads.me) (`npm install react-tads-widget`):
|
|
182
|
+
|
|
183
|
+
```jsx
|
|
184
|
+
import { TadsWidgetProvider, TadsWidget } from 'react-tads-widget';
|
|
185
|
+
|
|
186
|
+
// wrap the app once
|
|
187
|
+
<TadsWidgetProvider><App /></TadsWidgetProvider>
|
|
188
|
+
|
|
189
|
+
// native block inside content; onClickReward makes it a rewarded ad
|
|
190
|
+
<TadsWidget id="YOUR_WIDGET_ID" type="static"
|
|
191
|
+
onClickReward={() => grantBonus()}
|
|
192
|
+
onAdsNotFound={() => hidePlaceholder()} />
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Fullscreen ads work the same way with `type="fullscreen"` and `onShowReward`; show them on a user action, never on load. Agent-readable docs: https://tads.me/llms.txt
|
|
196
|
+
|
|
197
|
+
#### Advertising safety and limitations
|
|
198
|
+
|
|
199
|
+
- Confirm the network, widget format, reward rules, Telegram requirements, and applicable privacy obligations for the app and audience before deployment. Tell users when content is an ad and what action earns a reward.
|
|
200
|
+
- Treat click/show callbacks as client-side UI signals only. Do not use them alone to grant transferable or purchasable value; validate reward events with a trusted server-side source where available, and make grants idempotent and abuse-resistant.
|
|
201
|
+
- Keep ad display optional and user-initiated where the format calls for it. Never show a fullscreen ad automatically on app load or make an ad interaction the only way to use a core feature.
|
|
202
|
+
- SDK APIs and publisher terms can change. Check the current TADS documentation and package version before copying examples into production.
|
|
203
|
+
|
|
178
204
|
### Telegram Stars (New!)
|
|
179
205
|
```javascript
|
|
180
206
|
// In your bot
|
package/package.json
CHANGED