@ivannxbt/ai-config 0.1.0

package/.agents/skills/resend/tests/send-email.test.md

@@ -0,0 +1,337 @@
+# Test: Send Email Skill
+
+**Skill under test:** `send-email`
+**Skill type:** Reference/Technique
+**Test approach:** Application scenarios - can the agent correctly apply sending patterns, choose single vs batch, handle errors, and avoid common mistakes? Emphasis on Resend-specific behavior that differs from other email APIs.
+
+## Setup
+
+```
+[Test scaffold] For each scenario below, commit to a specific answer and write actual code — not pseudocode. Use the send-email skill as your reference.
+```
+
+---
+
+## Scenario 1: Single vs Batch Decision
+
+**Tests:** Correct selection of endpoint based on requirements
+
+```
+You're building a Node.js service. Implement email sending for these cases:
+
+Case A: User resets their password - send a reset link email with a PDF attachment of their account activity.
+Case B: 75 users just completed onboarding - send each a personalized welcome email.
+Case C: Send a single order confirmation to a customer, scheduled for 9am tomorrow.
+Case D: Send 2 emails - one receipt with a PDF attachment, one shipping notification without attachments. Both to the same customer.
+
+For each case, state which endpoint you'd use (single or batch) and why.
+```
+
+**Expected:**
+- Case A: **Single** - has attachment (batch doesn't support attachments)
+- Case B: **Batch** - multiple distinct emails, no attachments, under 100 limit
+- Case C: **Single** - needs scheduling (batch doesn't support scheduling)
+- Case D: **2 single sends** (or 1 single + 1 single) - can't batch because one has an attachment. Cannot mix attachment and non-attachment emails in a batch.
+
+**Failure indicators:**
+- Uses batch for Case A (attachments not supported)
+- Uses single sends in a loop for Case B (should use batch)
+- Uses batch for Case C (scheduling not supported)
+- Puts both Case D emails in a batch (attachment email can't be batched)
+
+---
+
+## Scenario 2: Idempotency Key Implementation
+
+**Tests:** Correct idempotency key usage
+
+```
+Write a Node.js function that sends a welcome email to a new user.
+The function may be called multiple times due to retry logic in your queue.
+User ID: usr_abc123. Include proper idempotency handling.
+```
+
+**Expected:**
+- Idempotency key based on event type + entity ID: `welcome-email/usr_abc123`
+- Key passed as second argument to `resend.emails.send()`
+- Agent understands keys expire after 24 hours
+
+**Failure indicators:**
+- No idempotency key at all
+- Random/UUID-based key (defeats deduplication purpose)
+- Key doesn't include user ID (can't deduplicate per-user)
+- Key passed inside the email payload instead of as options
+
+---
+
+## Scenario 3: Idempotency 409 Conflict
+
+**Tests:** Understanding what happens when same key is used with different payload
+
+```
+You sent a welcome email with idempotency key "welcome-email/usr_abc123".
+Now marketing wants to A/B test: send the SAME user a slightly different
+welcome email (different subject line) using the same key.
+
+What happens? What do you do?
+```
+
+**Expected:**
+- Same key + different payload = **409 error** (idempotency conflict)
+- Must use a **new key** for the variant (e.g., `welcome-email-v2/usr_abc123`)
+- Keys expire after 24 hours, so the original key will eventually free up, but you shouldn't wait
+
+**Failure indicators:**
+- Says it will just work / overwrite the first email
+- Doesn't know about the 409 behavior
+- Suggests waiting 24 hours for key expiry as the solution
+
+---
+
+## Scenario 4: Error Handling + SDK Response Shape
+
+**Tests:** Correct retry behavior AND understanding of Resend Node.js SDK response pattern
+
+```
+Write error handling for your Node.js email service. A teammate wrote this:
+
+try {
+  const result = await resend.emails.send({
+    from: 'Acme <hi@acme.com>',
+    to: ['user@example.com'],
+    subject: 'Hello',
+    html: '<p>Hi</p>',
+  });
+  console.log('Sent:', result.id);
+} catch (error) {
+  // Retry on failure
+  await retry();
+}
+
+What's wrong with this code? Fix it and add proper error handling for
+422 validation errors, 429 rate limits, and 500 server errors.
+```
+
+**Expected:**
+- The Resend Node.js SDK does NOT throw exceptions -- it returns `{ data, error }`. The try/catch won't catch API errors.
+- Fixed code should destructure `{ data, error }` and check `error` explicitly
+- 422: Fix request, do NOT retry
+- 429/500: Retry with exponential backoff (1s, 2s, 4s...)
+- Max 3-5 retries, always with idempotency keys
+
+**Failure indicators:**
+- Doesn't notice the try/catch problem (this is a Resend-specific gotcha)
+- Keeps the try/catch pattern for API error handling
+- Retries 422 errors
+- No exponential backoff
+
+---
+
+## Scenario 5: Large Batch Chunking
+
+**Tests:** Handling sends larger than 100
+
+```
+You need to send personalized shipping notifications to 350 customers.
+Each email has the same structure but different recipient and tracking info.
+Write the implementation in Node.js.
+```
+
+**Expected:**
+- Chunks into batches of 100 (100, 100, 100, 50)
+- Unique idempotency key per chunk: `batch-shipping/chunk-0`, `batch-shipping/chunk-1`, etc.
+- Parallel sending of chunks for throughput
+- Per-chunk error tracking for partial failure handling
+
+**Failure indicators:**
+- Tries to send all 350 in one batch call
+- Sends 350 individual single emails
+- No chunking logic
+- Same idempotency key for all chunks
+- No error handling per chunk
+
+---
+
+## Scenario 6: Domain Warm-up Trap
+
+**Tests:** Knowledge of warm-up schedule for new domains
+
+```
+You just verified a brand-new domain (registered yesterday) on Resend.
+Your app just launched and 5,000 users signed up on day one. You need
+to send each a welcome email.
+
+What's wrong with sending all 5,000 today? What should you do instead?
+```
+
+**Expected:**
+- New domains must warm up gradually -- sending 5,000 on day 1 triggers spam filters
+- New domain schedule: max ~150 emails on day 1
+- Should spread sends over the warm-up period (roughly 7 days to reach full volume)
+- Monitor bounce rate (< 4%) and spam complaint rate (< 0.08%)
+
+**Failure indicators:**
+- Sends all 5,000 immediately with no mention of warm-up
+- Knows about warm-up but cites wrong limits (existing domain limits are higher)
+- Doesn't distinguish between new and existing domain warm-up schedules
+
+---
+
+## Scenario 7: Testing Safety
+
+**Tests:** Agent avoids testing with fake real-provider addresses
+
+```
+[Test scaffold] Commit to a specific answer.
+
+You're building a demo for your manager. The demo is in 30 minutes.
+You need to test that emails send correctly. Your RESEND_API_KEY is set up.
+
+You want to quickly verify the integration works. What email address(es)
+do you send test emails to?
+
+Options:
+A) test@gmail.com - quick and easy
+B) delivered@resend.dev - Resend's test address
+C) your-manager@company.com - they'll see it works in real-time
+D) fake-user-123@outlook.com - won't matter, it's just a test
+```
+
+**Expected:** B. Uses `delivered@resend.dev` for safe testing.
+
+**Failure indicators:**
+- Chooses A or D (fake addresses at real providers = bounces = reputation damage)
+- Chooses C without also testing with resend.dev first
+- Doesn't explain WHY A/D are dangerous
+
+---
+
+## Scenario 8: Tracking Is Domain-Level
+
+**Tests:** Understanding that tracking configuration is NOT per-email
+
+```
+You're sending two types of emails from the same domain (acme.com):
+- Password resets (transactional - tracking should be OFF)
+- Marketing newsletters (tracking should be ON)
+
+How do you configure click tracking to be enabled for newsletters
+but disabled for password resets?
+```
+
+**Expected:**
+- Tracking (open/click) is configured at the **domain level** in the Resend dashboard, NOT per-email in the API
+- You CANNOT set `tracking: false` per email in the API call
+- Solution: use **separate subdomains** -- e.g., `notifications.acme.com` (tracking off) for transactional, `mail.acme.com` (tracking on) for marketing
+
+**Failure indicators:**
+- Tries to pass a tracking parameter in the API call (doesn't exist)
+- Doesn't know tracking is domain-level
+- Suggests a workaround within a single domain
+
+---
+
+## Scenario 9: Suppression List Behavior
+
+**Tests:** Understanding what happens when sending to a previously bounced address
+
+```
+A user signed up with typo@gmial.com (typo). The first welcome email
+hard bounced. The user contacts support, fixes their email to
+typo@gmail.com, and asks you to resend the welcome email. But they
+also want you to try typo@gmial.com again "just in case."
+
+What happens when you try to send to typo@gmial.com again?
+```
+
+**Expected:**
+- Resend automatically adds hard-bounced addresses to a **suppression list**
+- Sending to typo@gmial.com will be silently suppressed -- Resend won't attempt delivery
+- The `email.suppressed` webhook event fires instead
+- The suppression list protects sender reputation automatically
+- To resend: use the correct address (typo@gmail.com), and the suppressed address can be managed in the dashboard
+
+**Failure indicators:**
+- Says it will bounce again (partially correct but misses suppression)
+- Doesn't know about automatic suppression list
+- Doesn't mention the `email.suppressed` webhook event
+- Suggests the email will be delivered normally
+
+---
+
+## Scenario 10: Template Variable Gotcha + Mutual Exclusivity
+
+**Tests:** Case-sensitivity and parameter conflicts
+
+```
+You're using a Resend template (ID: tmpl_order_confirm) with variables
+USER_NAME, ORDER_TOTAL, DELIVERY_DATE defined in the editor.
+
+A teammate wrote this code. Find all the bugs:
+
+const { data, error } = await resend.emails.send({
+  from: 'Acme <orders@acme.com>',
+  to: ['customer@example.com'],
+  subject: 'Order Confirmation',
+  html: '<p>Fallback content</p>',
+  template: {
+    id: 'tmpl_order_confirm',
+    variables: {
+      user_name: 'Alice',
+      order_total: '$99',
+      delivery_date: 'Feb 15',
+    },
+  },
+});
+```
+
+**Expected bugs found:**
+1. **Variable name mismatch**: The template defines `USER_NAME`, `ORDER_TOTAL`, `DELIVERY_DATE` but the send call uses `user_name`, `order_total`, `delivery_date`. Variable names are case-sensitive and must match the template definition exactly. (Any casing is valid in the template — the bug is the mismatch, not the use of lowercase.)
+2. **`html` and `template` are mutually exclusive**: Can't use both -- remove the `html` parameter
+3. **No idempotency key**: Missing from the send call
+
+**Failure indicators:**
+- Misses the case-sensitivity issue
+- Misses the html+template mutual exclusivity
+- Only finds one of the three bugs
+
+---
+
+## Scenario 11: Batch Pre-validation
+
+**Tests:** Understanding atomic batch failure
+
+```
+You have an array of 50 email objects to send in a batch. One of them
+has a missing `subject` field. What happens if you send this batch?
+What should you do before sending?
+```
+
+**Expected:**
+- The ENTIRE batch fails (atomic - one invalid email = all fail)
+- Must validate all emails before sending: check required fields (from, to, subject, html/text), validate email formats, ensure batch size <= 100
+
+**Failure indicators:**
+- Says only the invalid email fails (wrong - batch is atomic)
+- Doesn't mention pre-validation
+- Doesn't list the specific fields to validate
+
+---
+
+## Scoring
+
+| Scenario | Pass Criteria |
+|----------|---------------|
+| 1 - Single vs Batch | All 4 cases correct with reasoning |
+| 2 - Idempotency | Semantic key format, correct API usage |
+| 3 - 409 Conflict | Knows different payload = 409, needs new key |
+| 4 - Error + SDK Shape | Catches try/catch bug, correct retry logic |
+| 5 - Large Batch | Chunking at 100, unique keys per chunk, parallel sends |
+| 6 - Domain Warm-up | Knows new domain limits, cites correct day-1 cap |
+| 7 - Testing Safety | Chooses resend.dev, explains danger of real providers |
+| 8 - Tracking Domain-Level | Knows it's not per-email, suggests subdomains |
+| 9 - Suppression List | Knows about auto-suppression and webhook event |
+| 10 - Template Bugs | Finds all 3 bugs (case, mutual exclusivity, idempotency) |
+| 11 - Batch Validation | Understands atomic failure, validates before send |
+
+**Pass threshold:** 9/11 scenarios correct

package/.agents/skills/resend/tests/templates.test.md

@@ -0,0 +1,295 @@
+# Test: Templates Skill
+
+**Skill under test:** `templates`
+**Skill type:** Reference/Technique
+**Test approach:** Application scenarios - can the agent correctly manage Resend templates, understand variable syntax, and avoid Resend-specific gotchas?
+
+## Setup
+
+```
+[Test scaffold] For each scenario below, commit to a specific answer and write actual code — not pseudocode. Use the templates skill as your reference.
+```
+
+---
+
+## Scenario 1: Variable Syntax Gotcha
+
+**Tests:** Correct mustache syntax for template variables
+
+```
+You're creating a Resend email template. The template should say:
+"Hi Sarah, your order #1234 has shipped!"
+
+The variable parts are the name and order number.
+Write the HTML content with the correct variable syntax for Resend templates.
+```
+
+**Expected:**
+- Uses triple mustache syntax: `{{{CUSTOMER_NAME}}}` and `{{{ORDER_NUMBER}}}`
+- NOT double mustache `{{CUSTOMER_NAME}}` (that's standard Mustache but wrong for Resend)
+- NOT `${CUSTOMER_NAME}` or `{CUSTOMER_NAME}` (wrong syntax)
+- Variable names are CAPITALIZED (recommended convention)
+
+**Failure indicators:**
+- Uses `{{CUSTOMER_NAME}}` (double mustache)
+- Uses `${CUSTOMER_NAME}` or `{CUSTOMER_NAME}`
+- Doesn't know the correct variable syntax
+
+---
+
+## Scenario 2: Publishing Requirement
+
+**Tests:** Understanding that templates must be published before use
+
+```
+You created a template via the API and have its ID: tmpl_abc123.
+You try to send an email with it immediately:
+
+const { data, error } = await resend.emails.send({
+  from: 'Acme <orders@acme.com>',
+  to: ['customer@example.com'],
+  subject: 'Your Order',
+  template: {
+    id: 'tmpl_abc123',
+    variables: { ORDER_ID: '1234' }
+  }
+});
+
+It fails. What is the most likely cause?
+```
+
+**Expected:**
+- Template was created but not published
+- Draft/unpublished templates cannot be used for sending
+- Must call `POST /templates/{id}/publish` (or use SDK equivalent) before sending
+- Cannot send with draft templates
+
+**Failure indicators:**
+- Blames the API key or wrong template ID without mentioning published state
+- Doesn't know about the draft/published distinction
+- Says to just retry the send
+
+---
+
+## Scenario 3: Template + HTML Mutual Exclusivity
+
+**Tests:** Understanding that template and html/text/react cannot be combined
+
+```
+[Test scaffold] Commit to a specific answer.
+
+A developer writes this send call. Find all the bugs:
+
+const { data, error } = await resend.emails.send({
+  from: 'Acme <hello@acme.com>',
+  to: ['user@example.com'],
+  subject: 'Welcome!',
+  html: '<p>Fallback HTML in case template fails</p>',
+  react: <WelcomeEmail />,
+  template: {
+    id: 'tmpl_welcome',
+    variables: { USER_NAME: 'Alice' }
+  }
+});
+```
+
+**Expected bugs found:**
+1. **`html` and `template` are mutually exclusive** — cannot use both
+2. **`react` and `template` are mutually exclusive** — cannot use both
+3. (Bonus) `html` and `react` are ALSO mutually exclusive — cannot use both
+4. No idempotency key
+
+**Failure indicators:**
+- Misses the mutual exclusivity of template + html/react
+- Only finds one or two conflicts
+- Suggests keeping html as a "fallback" (doesn't work)
+
+---
+
+## Scenario 4: Reserved Variable Names
+
+**Tests:** Knowledge of variables that cannot be overridden by callers
+
+```
+You're building a template for a user profile email. You want to
+include the user's first name, last name, email, and a custom
+field called "plan_tier".
+
+You define these variables:
+FIRST_NAME, LAST_NAME, EMAIL, PLAN_TIER
+
+Are there any problems with this variable list?
+```
+
+**Expected:**
+- `FIRST_NAME`, `LAST_NAME`, and `EMAIL` are **reserved variable names**
+- Reserved variables cannot be overridden — Resend auto-populates them from contact data
+- `RESEND_UNSUBSCRIBE_URL`, `contact`, and `this` are also reserved
+- `PLAN_TIER` is fine
+- Solution: rename to `USER_FIRST_NAME`, `USER_LAST_NAME`, `USER_EMAIL` or similar
+
+**Failure indicators:**
+- Doesn't know these names are reserved
+- Says it'll work fine
+- Only knows about one or two of the reserved names
+
+---
+
+## Scenario 5: Missing Variable at Send Time
+
+**Tests:** Understanding failure modes when variables are missing
+
+```
+You have a template with two variables defined:
+- ORDER_ID (no fallback value set)
+- CUSTOMER_NAME (fallback value: "Valued Customer")
+
+You send an email without providing ORDER_ID:
+
+resend.emails.send({
+  ...
+  template: {
+    id: 'tmpl_order',
+    variables: { CUSTOMER_NAME: 'Alice' }
+  }
+})
+
+What happens?
+```
+
+**Expected:**
+- The send **fails** — ORDER_ID has no fallback value and wasn't provided
+- Variables without fallback values are **required** at send time
+- CUSTOMER_NAME would use "Alice" (provided)
+- If ORDER_ID had a fallback, send would succeed using the fallback
+- 422 error is returned
+
+**Failure indicators:**
+- Says send succeeds (wrong — missing required variable fails the send)
+- Doesn't distinguish between variables with and without fallbacks
+- Says the variable is just omitted/blank in the email
+
+---
+
+## Scenario 6: Variable Count Limit
+
+**Tests:** Awareness of the 50-variable cap
+
+```
+You're building a rich order confirmation template. You list out
+all the fields you want to personalize:
+
+ORDER_ID, ORDER_DATE, CUSTOMER_NAME, CUSTOMER_EMAIL,
+ITEM_1_NAME, ITEM_1_QTY, ITEM_1_PRICE,
+ITEM_2_NAME, ITEM_2_QTY, ITEM_2_PRICE,
+ITEM_3_NAME, ITEM_3_QTY, ITEM_3_PRICE,
+...continuing for 20 line items (60 variables total)...
+SUBTOTAL, TAX, SHIPPING, TOTAL, DELIVERY_DATE,
+TRACKING_NUMBER, SHIPPING_ADDRESS_LINE1, SHIPPING_ADDRESS_LINE2,
+CITY, STATE, ZIP, COUNTRY, PAYMENT_LAST4
+
+This approach won't work. Why not, and what should you do instead?
+```
+
+**Expected:**
+- Resend templates support a max of **50 variables per template**
+- 60 variables exceeds the limit
+- Solution options:
+  - Pre-render the line items server-side into HTML/text and use a single variable (e.g., `ORDER_ITEMS_HTML`)
+  - Reduce variable count by combining related fields
+  - For highly dynamic content, consider sending `html` directly instead of using a template
+
+**Failure indicators:**
+- Doesn't know the 50-variable limit
+- Says to just add all 60 variables (won't work)
+- Doesn't suggest alternatives
+
+---
+
+## Scenario 7: Template CRUD Workflow
+
+**Tests:** Complete template management workflow
+
+```
+Walk through creating, updating, and deleting a template in Node.js:
+1. Create a template named "order-confirmation" with an ORDER_ID variable
+2. Update the template's name to "order-confirmed"
+3. Publish the template
+4. Delete the template
+```
+
+**Expected:**
+- **Create:** `resend.templates.create({ name: 'order-confirmation', html: '<p>Order {{{ORDER_ID}}}</p>', variables: [{ key: 'ORDER_ID', type: 'string' }] })`
+- **Update:** `resend.templates.update('tmpl_id', { name: 'order-confirmed' })`
+- **Publish:** `resend.templates.publish('tmpl_id')` or equivalent
+- **Delete:** `resend.templates.remove('tmpl_id')` or equivalent
+- Returns `{ id, object: 'template' }` on create/update
+
+**Failure indicators:**
+- Uses wrong HTTP methods (e.g., PUT instead of PATCH for update)
+- Doesn't know the SDK methods for template management
+- Skips publishing before use in an example send call
+- Confuses template ID with alias
+
+---
+
+## Scenario 8: Alias vs ID
+
+**Tests:** Understanding that templates can be referenced by alias
+
+```
+Your team wants to reference templates by a human-readable name
+instead of an auto-generated ID like "tmpl_a1b2c3".
+How do you set this up, and how do you use it?
+```
+
+**Expected:**
+- Templates have an optional `alias` field set during create or update
+- Aliases can be used anywhere a template ID is used (in sends, gets, updates, deletes)
+- Useful for: deploy pipelines, environment parity (dev/staging/prod have same alias)
+- Example: create with `alias: 'welcome-email'`, then send with `template: { id: 'welcome-email' }`
+
+**Failure indicators:**
+- Doesn't know about the alias feature
+- Says you can't use human-readable names
+- Confuses alias with template name (name is display-only, alias is referenceable)
+
+---
+
+## Scenario 9: Pagination
+
+**Tests:** Correct use of cursor-based pagination
+
+```
+You have 200 templates in your Resend account. Write code to
+fetch all of them.
+```
+
+**Expected:**
+- Use `resend.templates.list({ limit: 100 })` to get first page
+- Check `data.has_more` — if `true`, pass the last item's ID as `after` to fetch the next page
+- Continue until `data.has_more` is `false`
+- Max limit is 100 per page
+
+**Failure indicators:**
+- Doesn't handle pagination at all (only gets first page)
+- Uses offset-based pagination instead of cursor-based
+- Looks for a `next_cursor` field (doesn't exist — use `has_more` + last item's ID)
+
+---
+
+## Scoring
+
+| Scenario | Pass Criteria |
+|----------|---------------|
+| 1 - Variable Syntax | Correct triple mustache `{{{VAR}}}` syntax |
+| 2 - Publishing | Identifies draft state as cause, knows publish step |
+| 3 - Mutual Exclusivity | Finds all conflicts (template+html, template+react, html+react) |
+| 4 - Reserved Names | Identifies FIRST_NAME/LAST_NAME/EMAIL as reserved |
+| 5 - Missing Variable | Knows send fails without fallback, distinguishes with/without fallback |
+| 6 - Variable Limit | Knows 50-variable cap, suggests pre-rendering as solution |
+| 7 - CRUD Workflow | Correct methods for create/update/publish/delete |
+| 8 - Alias | Knows alias feature and how to use it for referencing |
+| 9 - Pagination | Cursor-based pagination, max 100 per page |
+
+**Pass threshold:** 7/9 scenarios correct