codehava-agent-kit 1.0.1 → 3.0.0

package/templates/.agent/skills/apify-actor-development/references/key-value-store-schema.md

@@ -0,0 +1,129 @@
+# Key-Value Store Schema Reference
+
+The key-value store schema organizes keys into logical groups called collections for easier data management.
+
+## Examples
+
+### JavaScript and TypeScript
+
+Consider an example Actor that calls `Actor.setValue()` to save records into the key-value store:
+
+```javascript
+import { Actor } from 'apify';
+// Initialize the JavaScript SDK
+await Actor.init();
+
+/**
+ * Actor code
+ */
+await Actor.setValue('document-1', 'my text data', { contentType: 'text/plain' });
+
+await Actor.setValue(`image-${imageID}`, imageBuffer, { contentType: 'image/jpeg' });
+
+// Exit successfully
+await Actor.exit();
+```
+
+### Python
+
+Consider an example Actor that calls `Actor.set_value()` to save records into the key-value store:
+
+```python
+# Key-Value Store set example (Python)
+import asyncio
+from apify import Actor
+
+async def main():
+    await Actor.init()
+
+    # Actor code
+    await Actor.set_value('document-1', 'my text data', content_type='text/plain')
+
+    image_id = '123'          # example placeholder
+    image_buffer = b'...'     # bytes buffer with image data
+    await Actor.set_value(f'image-{image_id}', image_buffer, content_type='image/jpeg')
+
+    # Exit successfully
+    await Actor.exit()
+
+if __name__ == '__main__':
+    asyncio.run(main())
+```
+
+## Configuration
+
+To configure the key-value store schema, reference a schema file in `.actor/actor.json`:
+
+```json
+{
+    "actorSpecification": 1,
+    "name": "data-collector",
+    "title": "Data Collector",
+    "version": "1.0.0",
+    "storages": {
+        "keyValueStore": "./key_value_store_schema.json"
+    }
+}
+```
+
+Then create the key-value store schema in `.actor/key_value_store_schema.json`:
+
+```json
+{
+    "actorKeyValueStoreSchemaVersion": 1,
+    "title": "Key-Value Store Schema",
+    "collections": {
+        "documents": {
+            "title": "Documents",
+            "description": "Text documents stored by the Actor",
+            "keyPrefix": "document-"
+        },
+        "images": {
+            "title": "Images",
+            "description": "Images stored by the Actor",
+            "keyPrefix": "image-",
+            "contentTypes": ["image/jpeg"]
+        }
+    }
+}
+```
+
+## Structure
+
+```json
+{
+    "actorKeyValueStoreSchemaVersion": 1,
+    "title": "string (required)",
+    "description": "string (optional)",
+    "collections": {
+        "<COLLECTION_NAME>": {
+            "title": "string (required)",
+            "description": "string (optional)",
+            "key": "string (conditional - use key OR keyPrefix)",
+            "keyPrefix": "string (conditional - use key OR keyPrefix)",
+            "contentTypes": ["string (optional)"],
+            "jsonSchema": "object (optional)"
+        }
+    }
+}
+```
+
+## Properties
+
+### Key-Value Store Schema Properties
+
+- `actorKeyValueStoreSchemaVersion` (integer, required) - Version of key-value store schema structure document (currently only version 1)
+- `title` (string, required) - Title of the schema
+- `description` (string, optional) - Description of the schema
+- `collections` (Object, required) - Object where each key is a collection ID and value is a Collection object
+
+### Collection Properties
+
+- `title` (string, required) - Collection title shown in UI tabs
+- `description` (string, optional) - Description appearing in UI tooltips
+- `key` (string, conditional) - Single specific key for this collection
+- `keyPrefix` (string, conditional) - Prefix for keys included in this collection
+- `contentTypes` (string[], optional) - Allowed content types for validation
+- `jsonSchema` (object, optional) - JSON Schema Draft 07 format for `application/json` content type validation
+
+Either `key` or `keyPrefix` must be specified for each collection, but not both.

package/templates/.agent/skills/apify-actor-development/references/logging.md

@@ -0,0 +1,50 @@
+# Actor Logging Reference
+
+## JavaScript and TypeScript
+
+**ALWAYS use the `apify/log` package for logging** - This package contains critical security logic including censoring sensitive data (Apify tokens, API keys, credentials) to prevent accidental exposure in logs.
+
+### Available Log Levels in `apify/log`
+
+The Apify log package provides the following methods for logging:
+
+- `log.debug()` - Debug level logs (detailed diagnostic information)
+- `log.info()` - Info level logs (general informational messages)
+- `log.warning()` - Warning level logs (warning messages for potentially problematic situations)
+- `log.warningOnce()` - Warning level logs (same warning message logged only once)
+- `log.error()` - Error level logs (error messages for failures)
+- `log.exception()` - Exception level logs (for exceptions with stack traces)
+- `log.perf()` - Performance level logs (performance metrics and timing information)
+- `log.deprecated()` - Deprecation level logs (warnings about deprecated code)
+- `log.softFail()` - Soft failure logs (non-critical failures that don't stop execution, e.g., input validation errors, skipped items)
+- `log.internal()` - Internal level logs (internal/system messages)
+
+### Best Practices
+
+- Use `log.debug()` for detailed operation-level diagnostics (inside functions)
+- Use `log.info()` for general informational messages (API requests, successful operations)
+- Use `log.warning()` for potentially problematic situations (validation failures, unexpected states)
+- Use `log.error()` for actual errors and failures
+- Use `log.exception()` for caught exceptions with stack traces
+
+## Python
+
+**ALWAYS use `Actor.log` for logging** - This logger contains critical security logic including censoring sensitive data (Apify tokens, API keys, credentials) to prevent accidental exposure in logs.
+
+### Available Log Levels
+
+The Apify Actor logger provides the following methods for logging:
+
+- `Actor.log.debug()` - Debug level logs (detailed diagnostic information)
+- `Actor.log.info()` - Info level logs (general informational messages)
+- `Actor.log.warning()` - Warning level logs (warning messages for potentially problematic situations)
+- `Actor.log.error()` - Error level logs (error messages for failures)
+- `Actor.log.exception()` - Exception level logs (for exceptions with stack traces)
+
+### Best Practices
+
+- Use `Actor.log.debug()` for detailed operation-level diagnostics (inside functions)
+- Use `Actor.log.info()` for general informational messages (API requests, successful operations)
+- Use `Actor.log.warning()` for potentially problematic situations (validation failures, unexpected states)
+- Use `Actor.log.error()` for actual errors and failures
+- Use `Actor.log.exception()` for caught exceptions with stack traces

package/templates/.agent/skills/apify-actor-development/references/output-schema.md

@@ -0,0 +1,49 @@
+# Output Schema Reference
+
+The Actor output schema builds upon the schemas for the dataset and key-value store. It specifies where an Actor stores its output and defines templates for accessing that output. Apify Console uses these output definitions to display run results.
+
+## Structure
+
+```json
+{
+    "actorOutputSchemaVersion": 1,
+    "title": "<OUTPUT-SCHEMA-TITLE>",
+    "properties": {
+        /* define your outputs here */
+    }
+}
+```
+
+## Example
+
+```json
+{
+    "actorOutputSchemaVersion": 1,
+    "title": "Output schema of the files scraper",
+    "properties": {
+        "files": {
+            "type": "string",
+            "title": "Files",
+            "template": "{{links.apiDefaultKeyValueStoreUrl}}/keys"
+        },
+        "dataset": {
+            "type": "string",
+            "title": "Dataset",
+            "template": "{{links.apiDefaultDatasetUrl}}/items"
+        }
+    }
+}
+```
+
+## Output Schema Template Variables
+
+- `links` (object) - Contains quick links to most commonly used URLs
+- `links.publicRunUrl` (string) - Public run url in format `https://console.apify.com/view/runs/:runId`
+- `links.consoleRunUrl` (string) - Console run url in format `https://console.apify.com/actors/runs/:runId`
+- `links.apiRunUrl` (string) - API run url in format `https://api.apify.com/v2/actor-runs/:runId`
+- `links.apiDefaultDatasetUrl` (string) - API url of default dataset in format `https://api.apify.com/v2/datasets/:defaultDatasetId`
+- `links.apiDefaultKeyValueStoreUrl` (string) - API url of default key-value store in format `https://api.apify.com/v2/key-value-stores/:defaultKeyValueStoreId`
+- `links.containerRunUrl` (string) - URL of a webserver running inside the run in format `https://<containerId>.runs.apify.net/`
+- `run` (object) - Contains information about the run same as it is returned from the `GET Run` API endpoint
+- `run.defaultDatasetId` (string) - ID of the default dataset
+- `run.defaultKeyValueStoreId` (string) - ID of the default key-value store

package/templates/.agent/skills/apify-actor-development/references/standby-mode.md

@@ -0,0 +1,61 @@
+# Actor Standby Mode Reference
+
+## JavaScript and TypeScript
+
+- **NEVER disable standby mode (`usesStandbyMode: false`) in `.actor/actor.json` without explicit permission** - Actor Standby mode solves this problem by letting you have the Actor ready in the background, waiting for the incoming HTTP requests. In a sense, the Actor behaves like a real-time web server or standard API server instead of running the logic once to process everything in batch. Always keep `usesStandbyMode: true` unless there is a specific documented reason to disable it
+- **ALWAYS implement readiness probe handler for standby Actors** - Handle the `x-apify-container-server-readiness-probe` header at GET / endpoint to ensure proper Actor lifecycle management
+
+You can recognize a standby Actor by checking the `usesStandbyMode` property in `.actor/actor.json`. Only implement the readiness probe if this property is set to `true`.
+
+### Readiness Probe Implementation Example
+
+```javascript
+// Apify standby readiness probe at root path
+app.get('/', (req, res) => {
+    res.writeHead(200, { 'Content-Type': 'text/plain' });
+    if (req.headers['x-apify-container-server-readiness-probe']) {
+        res.end('Readiness probe OK\n');
+    } else {
+        res.end('Actor is ready\n');
+    }
+});
+```
+
+Key points:
+
+- Detect the `x-apify-container-server-readiness-probe` header in incoming requests
+- Respond with HTTP 200 status code for both readiness probe and normal requests
+- This enables proper Actor lifecycle management in standby mode
+
+## Python
+
+- **NEVER disable standby mode (`usesStandbyMode: false`) in `.actor/actor.json` without explicit permission** - Actor Standby mode solves this problem by letting you have the Actor ready in the background, waiting for the incoming HTTP requests. In a sense, the Actor behaves like a real-time web server or standard API server instead of running the logic once to process everything in batch. Always keep `usesStandbyMode: true` unless there is a specific documented reason to disable it
+- **ALWAYS implement readiness probe handler for standby Actors** - Handle the `x-apify-container-server-readiness-probe` header at GET / endpoint to ensure proper Actor lifecycle management
+
+You can recognize a standby Actor by checking the `usesStandbyMode` property in `.actor/actor.json`. Only implement the readiness probe if this property is set to `true`.
+
+### Readiness Probe Implementation Example
+
+```python
+# Apify standby readiness probe
+from http.server import SimpleHTTPRequestHandler
+
+class GetHandler(SimpleHTTPRequestHandler):
+    def do_GET(self):
+        # Handle Apify standby readiness probe
+        if 'x-apify-container-server-readiness-probe' in self.headers:
+            self.send_response(200)
+            self.end_headers()
+            self.wfile.write(b'Readiness probe OK')
+            return
+
+        self.send_response(200)
+        self.end_headers()
+        self.wfile.write(b'Actor is ready')
+```
+
+Key points:
+
+- Detect the `x-apify-container-server-readiness-probe` header in incoming requests
+- Respond with HTTP 200 status code for both readiness probe and normal requests
+- This enables proper Actor lifecycle management in standby mode

package/templates/.agent/skills/apify-actorization/SKILL.md

@@ -0,0 +1,185 @@
+---
+name: apify-actorization
+description: "Actorization converts existing software into reusable serverless applications compatible with the Apify platform. Actors are programs packaged as Docker images that accept well-defined JSON input, perform an action, and optionally produce structured JSON output."
+risk: unknown
+source: community
+---
+
+# Apify Actorization
+
+Actorization converts existing software into reusable serverless applications compatible with the Apify platform. Actors are programs packaged as Docker images that accept well-defined JSON input, perform an action, and optionally produce structured JSON output.
+
+## Quick Start
+
+1. Run `apify init` in project root
+2. Wrap code with SDK lifecycle (see language-specific section below)
+3. Configure `.actor/input_schema.json`
+4. Test with `apify run --input '{"key": "value"}'`
+5. Deploy with `apify push`
+
+## When to Use This Skill
+
+- Converting an existing project to run on Apify platform
+- Adding Apify SDK integration to a project
+- Wrapping a CLI tool or script as an Actor
+- Migrating a Crawlee project to Apify
+
+## Prerequisites
+
+Verify `apify` CLI is installed:
+
+```bash
+apify --help
+```
+
+If not installed:
+
+```bash
+brew install apify-cli
+
+# Or: npm install -g apify-cli
+# Or install from an official release package that your OS package manager verifies
+```
+
+Verify CLI is logged in:
+
+```bash
+apify info  # Should return your username
+```
+
+If not logged in, check if `APIFY_TOKEN` environment variable is defined. If not, ask the user to generate one at https://console.apify.com/settings/integrations, add it to their shell or secret manager without putting the literal token in command history, then run:
+
+```bash
+apify login
+```
+
+## Actorization Checklist
+
+Copy this checklist to track progress:
+
+- [ ] Step 1: Analyze project (language, entry point, inputs, outputs)
+- [ ] Step 2: Run `apify init` to create Actor structure
+- [ ] Step 3: Apply language-specific SDK integration
+- [ ] Step 4: Configure `.actor/input_schema.json`
+- [ ] Step 5: Configure `.actor/output_schema.json` (if applicable)
+- [ ] Step 6: Update `.actor/actor.json` metadata
+- [ ] Step 7: Test locally with `apify run`
+- [ ] Step 8: Deploy with `apify push`
+
+## Step 1: Analyze the Project
+
+Before making changes, understand the project:
+
+1. **Identify the language** - JavaScript/TypeScript, Python, or other
+2. **Find the entry point** - The main file that starts execution
+3. **Identify inputs** - Command-line arguments, environment variables, config files
+4. **Identify outputs** - Files, console output, API responses
+5. **Check for state** - Does it need to persist data between runs?
+
+## Step 2: Initialize Actor Structure
+
+Run in the project root:
+
+```bash
+apify init
+```
+
+This creates:
+- `.actor/actor.json` - Actor configuration and metadata
+- `.actor/input_schema.json` - Input definition for the Apify Console
+- `Dockerfile` (if not present) - Container image definition
+
+## Step 3: Apply Language-Specific Changes
+
+Choose based on your project's language:
+
+- **JavaScript/TypeScript**: See [js-ts-actorization.md](references/js-ts-actorization.md)
+- **Python**: See [python-actorization.md](references/python-actorization.md)
+- **Other Languages (CLI-based)**: See [cli-actorization.md](references/cli-actorization.md)
+
+### Quick Reference
+
+| Language | Install | Wrap Code |
+|----------|---------|-----------|
+| JS/TS | `npm install apify` | `await Actor.init()` ... `await Actor.exit()` |
+| Python | `pip install apify` | `async with Actor:` |
+| Other | Use CLI in wrapper script | `apify actor:get-input` / `apify actor:push-data` |
+
+## Steps 4-6: Configure Schemas
+
+See [schemas-and-output.md](references/schemas-and-output.md) for detailed configuration of:
+- Input schema (`.actor/input_schema.json`)
+- Output schema (`.actor/output_schema.json`)
+- Actor configuration (`.actor/actor.json`)
+- State management (request queues, key-value stores)
+
+Validate schemas against `@apify/json_schemas` npm package.
+
+## Step 7: Test Locally
+
+Run the actor with inline input (for JS/TS and Python actors):
+
+```bash
+apify run --input '{"startUrl": "https://example.com", "maxItems": 10}'
+```
+
+Or use an input file:
+
+```bash
+apify run --input-file ./test-input.json
+```
+
+**Important:** Always use `apify run`, not `npm start` or `python main.py`. The CLI sets up the proper environment and storage.
+
+## Step 8: Deploy
+
+```bash
+apify push
+```
+
+This uploads and builds your actor on the Apify platform.
+
+## Monetization (Optional)
+
+After deploying, you can monetize your actor in the Apify Store. The recommended model is **Pay Per Event (PPE)**:
+
+- Per result/item scraped
+- Per page processed
+- Per API call made
+
+Configure PPE in the Apify Console under Actor > Monetization. Charge for events in your code with `await Actor.charge('result')`.
+
+Other options: **Rental** (monthly subscription) or **Free** (open source).
+
+## Pre-Deployment Checklist
+
+- [ ] `.actor/actor.json` exists with correct name and description
+- [ ] `.actor/actor.json` validates against `@apify/json_schemas` (`actor.schema.json`)
+- [ ] `.actor/input_schema.json` defines all required inputs
+- [ ] `.actor/input_schema.json` validates against `@apify/json_schemas` (`input.schema.json`)
+- [ ] `.actor/output_schema.json` defines output structure (if applicable)
+- [ ] `.actor/output_schema.json` validates against `@apify/json_schemas` (`output.schema.json`)
+- [ ] `Dockerfile` is present and builds successfully
+- [ ] `Actor.init()` / `Actor.exit()` wraps main code (JS/TS)
+- [ ] `async with Actor:` wraps main code (Python)
+- [ ] Inputs are read via `Actor.getInput()` / `Actor.get_input()`
+- [ ] Outputs use `Actor.pushData()` or key-value store
+- [ ] `apify run` executes successfully with test input
+- [ ] `generatedBy` is set in actor.json meta section
+
+## Apify MCP Tools
+
+If MCP server is configured, use these tools for documentation:
+
+- `search-apify-docs` - Search documentation
+- `fetch-apify-docs` - Get full doc pages
+
+Otherwise, the MCP Server url: `https://mcp.apify.com/?tools=docs`.
+
+## Resources
+
+- [Actorization Academy](https://docs.apify.com/academy/actorization) - Comprehensive guide
+- [Apify SDK for JavaScript](https://docs.apify.com/sdk/js) - Full SDK reference
+- [Apify SDK for Python](https://docs.apify.com/sdk/python) - Full SDK reference
+- [Apify CLI Reference](https://docs.apify.com/cli) - CLI commands
+- [Actor Specification](https://raw.githubusercontent.com/apify/actor-whitepaper/refs/heads/master/README.md) - Complete specification

package/templates/.agent/skills/apify-actorization/references/cli-actorization.md

@@ -0,0 +1,81 @@
+# CLI-Based Actorization
+
+For languages without an SDK (Go, Rust, Java, etc.), create a wrapper script that uses the Apify CLI.
+
+## Create Wrapper Script
+
+Create `start.sh` in project root:
+
+```bash
+#!/bin/bash
+set -e
+
+# Get input from Apify key-value store
+INPUT=$(apify actor:get-input)
+
+# Parse input values (adjust based on your input schema)
+MY_PARAM=$(echo "$INPUT" | jq -r '.myParam // "default"')
+
+# Run your application with the input
+./your-application --param "$MY_PARAM"
+
+# If your app writes to a file, push it to key-value store
+# apify actor:set-value OUTPUT --contentType application/json < output.json
+
+# Or push structured data to dataset
+# apify actor:push-data '{"result": "value"}'
+```
+
+## Update Dockerfile
+
+Reference the [cli-start template Dockerfile](https://github.com/apify/actor-templates/blob/master/templates/cli-start/Dockerfile) which includes the `ubi` utility for installing binaries from GitHub releases.
+
+```dockerfile
+FROM apify/actor-node:20
+
+# Install ubi from a package source or a verified release artifact
+# Example: use your base image package manager or vendor a pinned binary in the build context
+# RUN apt-get update && apt-get install -y ubi
+
+# Install your CLI tool from GitHub releases (example)
+# RUN install -m 0755 ./vendor/your-tool /usr/local/bin/your-tool
+
+# Or install apify-cli and jq manually
+RUN npm install -g apify-cli
+RUN apt-get update && apt-get install -y jq
+
+# Copy your application
+COPY . .
+
+# Build your application if needed
+# RUN ./build.sh
+
+# Make start script executable
+RUN chmod +x start.sh
+
+# Run the wrapper script
+CMD ["./start.sh"]
+```
+
+## Testing CLI-Based Actors
+
+For CLI-based actors (shell wrapper scripts), you may need to test the underlying application directly with mock input, as `apify run` requires a Node.js or Python entry point.
+
+Test your wrapper script locally:
+
+```bash
+# Set up mock input
+export INPUT='{"myParam": "test-value"}'
+
+# Run wrapper script
+./start.sh
+```
+
+## CLI Commands Reference
+
+| Command | Description |
+|---------|-------------|
+| `apify actor:get-input` | Get input JSON from key-value store |
+| `apify actor:set-value KEY` | Store value in key-value store |
+| `apify actor:push-data JSON` | Push data to dataset |
+| `apify actor:get-value KEY` | Retrieve value from key-value store |

package/templates/.agent/skills/apify-actorization/references/js-ts-actorization.md

@@ -0,0 +1,111 @@
+# JavaScript/TypeScript Actorization
+
+## Install the Apify SDK
+
+```bash
+npm install apify
+```
+
+## Wrap Main Code with Actor Lifecycle
+
+```javascript
+import { Actor } from 'apify';
+
+// Initialize connection to Apify platform
+await Actor.init();
+
+// ============================================
+// Your existing code goes here
+// ============================================
+
+// Example: Get input from Apify Console or API
+const input = await Actor.getInput();
+console.log('Input:', input);
+
+// Example: Your crawler or processing logic
+// const crawler = new PlaywrightCrawler({ ... });
+// await crawler.run([input.startUrl]);
+
+// Example: Push results to dataset
+// await Actor.pushData({ result: 'data' });
+
+// ============================================
+// End of your code
+// ============================================
+
+// Graceful shutdown
+await Actor.exit();
+```
+
+## Key Points
+
+- `Actor.init()` configures storage to use Apify API when running on platform
+- `Actor.exit()` handles graceful shutdown and cleanup
+- Both calls must be awaited
+- Local execution remains unchanged - the SDK automatically detects the environment
+
+## Crawlee Projects
+
+Crawlee projects require minimal changes - just wrap with Actor lifecycle:
+
+```javascript
+import { Actor } from 'apify';
+import { PlaywrightCrawler } from 'crawlee';
+
+await Actor.init();
+
+// Get and validate input
+const input = await Actor.getInput();
+const {
+    startUrl = 'https://example.com',
+    maxItems = 100,
+} = input ?? {};
+
+let itemCount = 0;
+
+const crawler = new PlaywrightCrawler({
+    requestHandler: async ({ page, request, pushData }) => {
+        if (itemCount >= maxItems) return;
+
+        const title = await page.title();
+        await pushData({ url: request.url, title });
+        itemCount++;
+    },
+});
+
+await crawler.run([startUrl]);
+
+await Actor.exit();
+```
+
+## Express/HTTP Servers
+
+For web servers, use standby mode in actor.json:
+
+```json
+{
+    "actorSpecification": 1,
+    "name": "my-api",
+    "usesStandbyMode": true
+}
+```
+
+Then implement readiness probe. See [standby-mode.md](../../apify-actor-development/references/standby-mode.md).
+
+## Batch Processing Scripts
+
+```javascript
+import { Actor } from 'apify';
+
+await Actor.init();
+
+const input = await Actor.getInput();
+const items = input.items || [];
+
+for (const item of items) {
+    const result = processItem(item);
+    await Actor.pushData(result);
+}
+
+await Actor.exit();
+```