@opendirectory.dev/skills 0.1.36 → 0.1.38

package/skills/sdk-adoption-tracker/evals/evals.json

@@ -0,0 +1,108 @@
+[
+  {
+    "id": "eval_001",
+    "name": "npm SDK with real company adopters",
+    "description": "Full pipeline run on a well-known npm SDK with 20+ public repos. Verifies company org detection, scoring, enrichment, and brief generation all work end-to-end.",
+    "input": {
+      "prompt": "Find who is using my SDK on GitHub: stripe",
+      "env": {
+        "GITHUB_TOKEN": "set"
+      }
+    },
+    "expected_behavior": [
+      "Detects ecosystem as npm from the SDK name",
+      "Builds 4 query patterns: require(\"stripe\"), require('stripe'), from \"stripe\", from 'stripe'",
+      "Fetches up to 100 results per pattern, deduplicates by repo full_name",
+      "Filters forks and repos whose name starts with 'stripe' as noise",
+      "Classifies repos with owner.type == Organization as company_org",
+      "Fetches full repo metadata, owner profile, and top contributors for non-noise repos",
+      "Computes adoption_score using formula from scoring-guide.md",
+      "AI generates adoption briefs for repos with score >= 80",
+      "Every repo in the brief exists in the raw code search results",
+      "Every company name comes from the GitHub org.name or user.company API field",
+      "Every contributor handle comes from the contributors API response",
+      "Saves output to docs/sdk-adopters/stripe-YYYY-MM-DD.md and .json"
+    ],
+    "expected_output": "Report with at least 1 company_org repo, adoption velocity metrics, and at least 1 adoption brief with company name, top contributor handle, and outreach message."
+  },
+  {
+    "id": "eval_002",
+    "name": "Python SDK ecosystem detection",
+    "description": "Verifies auto-detection of Python ecosystem from snake_case package name and correct query pattern construction.",
+    "input": {
+      "prompt": "Track adoption of my Python library: boto3",
+      "env": {
+        "GITHUB_TOKEN": "set"
+      }
+    },
+    "expected_behavior": [
+      "Detects ecosystem as python from snake_case name with no hyphens or slashes",
+      "Builds 3 query patterns: 'import boto3', 'from boto3 import', 'from boto3.'",
+      "Does NOT use npm-style require() patterns",
+      "Fetches results and deduplicates repos",
+      "Classifies repos by tier using the same scoring formula",
+      "Proceeds through Steps 4-8 normally"
+    ],
+    "expected_output": "Report showing Python repos importing boto3, with correct ecosystem label in the output header."
+  },
+  {
+    "id": "eval_003",
+    "name": "Very new SDK not yet indexed",
+    "description": "Handles the case where a brand-new SDK returns 0 results from GitHub code search because indexing takes 1-4 weeks.",
+    "input": {
+      "prompt": "Find who is using my new SDK: my-brand-new-sdk-2026",
+      "env": {
+        "GITHUB_TOKEN": "set"
+      }
+    },
+    "expected_behavior": [
+      "Runs code search queries successfully (no API error)",
+      "Receives 0 results from all pattern queries",
+      "Does NOT proceed to Steps 4-8",
+      "Stops with a clear message: 'No repos found importing my-brand-new-sdk-2026. GitHub code search indexing takes 1-4 weeks for new packages.'",
+      "Does not write any output files",
+      "Does not hallucinate any repo results"
+    ],
+    "expected_output": "Graceful stop message with indexing explanation. No output file created."
+  },
+  {
+    "id": "eval_004",
+    "name": "GITHUB_TOKEN missing",
+    "description": "Verifies the skill stops immediately at Step 1 when GITHUB_TOKEN is not set, with a clear actionable error message.",
+    "input": {
+      "prompt": "Who is using my SDK: express",
+      "env": {
+        "GITHUB_TOKEN": "not set"
+      }
+    },
+    "expected_behavior": [
+      "Step 1 setup check detects GITHUB_TOKEN is missing",
+      "Stops immediately with error: 'GITHUB_TOKEN is required for code search.'",
+      "Includes instruction: 'Add a token at github.com/settings/tokens (no scopes needed for public repos).'",
+      "Explains why: unauthenticated code search hits 3 req/min secondary rate limit",
+      "Does NOT proceed to Step 2 or any API calls",
+      "Exit code 1 (or equivalent failure signal)"
+    ],
+    "expected_output": "Error message at Step 1 with token setup instructions. No API calls made."
+  },
+  {
+    "id": "eval_005",
+    "name": "All results are tutorial noise",
+    "description": "Handles the case where code search returns results but all repos are tutorials, examples, or forks -- no production adopters.",
+    "input": {
+      "prompt": "Find who is using my SDK: todo-app",
+      "env": {
+        "GITHUB_TOKEN": "set"
+      }
+    },
+    "expected_behavior": [
+      "Runs code search and finds repos",
+      "Step 4 classification marks all repos as tutorial_noise (name/description contain tutorial keywords)",
+      "No repos remain after filtering tutorial_noise",
+      "Does NOT proceed to Step 5 enrichment or Step 6 AI brief generation",
+      "Stops with message: 'All repos found appear to be tutorials or examples. No production adopters detected in public GitHub.'",
+      "Does not hallucinate production users"
+    ],
+    "expected_output": "Graceful stop with explanation. Breakdown table shows tutorial_noise count but 0 company/solo repos."
+  }
+]

package/skills/sdk-adoption-tracker/references/import-patterns.md

@@ -0,0 +1,183 @@
+# Import Patterns Reference
+
+Used by SKILL.md Step 3 to build GitHub code search queries per ecosystem.
+
+---
+
+## Ecosystem Detection
+
+Auto-detect from the SDK name before falling back to user input:
+
+| Signal | Ecosystem |
+|---|---|
+| Starts with `@` (e.g. `@clerk/nextjs`) | npm |
+| Contains `-` and no `github.com/` (e.g. `react-query`) | npm |
+| snake_case only, no `-` or `/` (e.g. `requests`, `boto3`) | python |
+| Contains `github.com/` (e.g. `github.com/stripe/stripe-go`) | go |
+| Contains `.` with no other signals (e.g. `org.springframework.boot`) | java |
+| Ambiguous | ask the user |
+
+---
+
+## npm / Node.js
+
+**Search patterns (run each as a separate code search query):**
+
+```
+require("sdk-name")
+require('sdk-name')
+from "sdk-name"
+from 'sdk-name'
+```
+
+**Notes:**
+- `from "sdk-name"` catches ES module default and named imports: `import foo from "sdk-name"` and `import { bar } from "sdk-name"`
+- `require("sdk-name")` catches CommonJS usage
+- Use the full package name including `@org/` prefix for scoped packages
+- Do NOT search `"sdk-name"` as a bare string -- too many false positives in package.json files
+
+**Example for `@clerk/nextjs`:**
+```
+require("@clerk/nextjs")
+from "@clerk/nextjs"
+from '@clerk/nextjs'
+```
+
+---
+
+## Python
+
+**Search patterns:**
+
+```
+import sdk_name
+from sdk_name import
+from sdk_name.
+```
+
+**Notes:**
+- `from sdk_name.` catches submodule imports: `from requests.exceptions import`
+- Use the PyPI package name (snake_case), not the import name if they differ
+- Example: `Pillow` installs as `pillow` but imports as `PIL` -- search `from PIL import` not `import pillow`
+
+**Example for `requests`:**
+```
+import requests
+from requests import
+from requests.
+```
+
+---
+
+## Go
+
+**Search patterns:**
+
+```
+"github.com/org/sdk-name"
+```
+
+**Notes:**
+- Go imports use the full module path, not just the package name
+- Include the full `github.com/org/sdk-name` path in quotes
+- For major version suffixes, add the version: `"github.com/org/sdk-name/v2"`
+
+**Example for `github.com/stripe/stripe-go`:**
+```
+"github.com/stripe/stripe-go"
+```
+
+---
+
+## Ruby / RubyGems
+
+**Search patterns:**
+
+```
+require 'sdk-name'
+require "sdk-name"
+gem 'sdk-name'
+```
+
+**Notes:**
+- `gem 'sdk-name'` catches Gemfile declarations (projects listing the dependency)
+- `require 'sdk-name'` catches runtime usage
+
+---
+
+## Rust / Cargo
+
+**Search patterns:**
+
+```
+sdk-name = 
+sdk_name = 
+```
+
+**Notes:**
+- Search `Cargo.toml` files for the dependency declaration
+- Add `filename:Cargo.toml` to the query to narrow results
+- Cargo uses hyphens in package names but underscores in crate names -- search both
+
+**Example for `tokio`:**
+```
+tokio =
+```
+
+---
+
+## PHP / Composer
+
+**Search patterns:**
+
+```
+require 'vendor/sdk-name'
+use VendorName\SdkName
+```
+
+---
+
+## Java / Maven
+
+**Search patterns:**
+
+```
+import com.vendor.sdkname
+<artifactId>sdk-name</artifactId>
+```
+
+**Notes:**
+- Maven/Gradle projects declare dependencies in `pom.xml` or `build.gradle`
+- `<artifactId>sdk-name</artifactId>` catches Maven declarations
+- Add `filename:pom.xml` to the query to narrow to Maven projects
+
+---
+
+## Generic Fallback
+
+If the ecosystem is unclear, use the bare SDK name as the search query. This catches any file that mentions the name but produces more false positives. Always filter by file extension or language qualifier when possible:
+
+```
+sdk-name language:javascript
+sdk-name language:python
+sdk-name language:typescript
+```
+
+---
+
+## Query Limit Budget
+
+GitHub code search rate limit: 10 requests/minute (authenticated).
+
+- npm: 4 queries
+- Python: 3 queries
+- Go: 1 query
+- Ruby: 3 queries
+- Generic: 1-3 queries
+
+Sleep 6 seconds between queries to stay within 10 req/min.
+
+Total scan time per ecosystem:
+- npm: ~24 seconds (4 queries x 6s)
+- Python: ~18 seconds (3 queries x 6s)
+- Go: ~6 seconds (1 query)

package/skills/sdk-adoption-tracker/references/scoring-guide.md

@@ -0,0 +1,148 @@
+# Adoption Scoring Guide
+
+Used by SKILL.md Step 4 and scripts/fetch.py to classify repos by adoption signal strength.
+
+---
+
+## The Problem with Raw Search Results
+
+A GitHub code search for `require("stripe")` returns 1,000+ repos. Most are:
+- Tutorials: "learn-stripe-payments", "stripe-integration-example"
+- Forks: someone forked the official Stripe SDK
+- Abandoned side projects: last push was 2021
+- Test repos: "stripe-test-app", "payment-playground"
+
+The adoption score filters these out and surfaces the signal that matters: an active company repo using your SDK in production.
+
+---
+
+## Scoring Formula
+
+```python
+score = 0
+
+# Company signals (highest weight -- this is what you want to find)
+if owner_type == "Organization":          score += 50  # GitHub org = company or team
+if company_field and company_field.strip(): score += 20  # user has company listed on GitHub
+
+# Activity signals
+score += min(stars, 500) / 10            # up to 50 points; capped at 500 stars
+if days_since_push < 30:                 score += 30  # active in last month
+if days_since_push < 7:                  score += 20  # active this week (cumulative with above)
+
+# Quality signals
+if not is_fork:                          score += 10  # original project, not a fork
+if not is_archived:                      score += 10  # not abandoned
+if not is_tutorial:                      score += 20  # not a demo or example
+
+# Maximum possible score: 160+
+# Typical company_org with recent activity: 110-150
+# Typical active solo dev project: 60-90
+# Typical stale personal project: 20-40
+# Tutorial/noise: 0-20
+```
+
+---
+
+## Score Tiers
+
+| Score | Tier | What it means |
+|---|---|---|
+| >= 80 | High signal | Generate a full adoption brief and outreach message |
+| 40-79 | Medium signal | List in report, no full brief |
+| < 40 | Noise | Include in breakdown count only |
+
+---
+
+## Company Classification Tiers
+
+| Tier | Criteria | Confidence |
+|---|---|---|
+| `company_org` | `owner.type == "Organization"` | High -- GitHub org accounts are almost always a company, team, or serious open-source project |
+| `affiliated_dev` | `owner.type == "User"` AND `company` field populated on GitHub profile | Medium -- self-reported, but developers with a company listed are usually professionals |
+| `solo_dev` | `owner.type == "User"` AND no company field AND stars >= 10 | Low -- could be anyone; stars signal real usage |
+| `tutorial_noise` | Name or description matches tutorial keywords, OR repo is a fork, OR repo is archived | Excluded from lead output |
+
+---
+
+## Tutorial Noise Detection
+
+A repo is classified as `tutorial_noise` if its name or description contains any of these words:
+
+```
+example, tutorial, demo, learn, sample, starter, boilerplate,
+template, playground, test, course, workshop
+```
+
+Additional noise signals:
+- `fork == true`: the repo is a fork (could be a fork of the SDK itself)
+- `archived == true`: the repo is no longer maintained
+- Repo name matches the SDK name exactly (e.g. searching for `stripe` and finding a repo called `stripe`)
+- Repo name starts with the SDK name followed by a dash (e.g. `stripe-clone`, `stripe-copy`)
+
+---
+
+## Velocity Signals
+
+Velocity is inferred from `created_at` dates, not from direct historical data.
+
+**Primary velocity metric:** Count of repos with `created_at` in the last 7, 14, and 30 days.
+
+```
+New repos last 7 days:  3
+New repos last 30 days: 11
+```
+
+This tells you: 3 new teams started using the SDK in the last week.
+
+**Secondary velocity metric (requires previous snapshot):** When a previous JSON snapshot exists in `docs/sdk-adopters/`, compare the current repo list to the previous one. The difference is the exact set of new adopters since the last run.
+
+```
+New since last run (14 days ago): 5
+```
+
+**Velocity interpretation:**
+- 0 new in 30 days: stagnant adoption, possibly indexed months ago and no new users
+- 1-3 new in 7 days: healthy organic growth
+- 5+ new in 7 days: breakout adoption, consider proactive outreach
+
+---
+
+## The Top Contributor Signal
+
+For each enriched repo, the script fetches the top 3 contributors by commit count via:
+`GET /repos/{owner}/{repo}/contributors?per_page=3`
+
+The person with the most commits is the most likely person to have introduced the SDK to that project. They are the warmest possible outreach target because:
+1. They chose the SDK
+2. They understand what it does
+3. They have the context to evaluate an upgrade, integration, or partnership
+
+If you cannot identify the top contributor (rate limit, private contributors), fall back to the repo owner.
+
+---
+
+## Why Organization Repos Are Worth 50 Points
+
+A `owner.type == "Organization"` repo means:
+- A team created a GitHub org to publish this code
+- At minimum, multiple people are involved
+- The code is likely maintained beyond a single developer's interest
+- The org may represent an actual company (check `org.blog` for the website)
+
+Contrast with a user repo: one developer, may be a side project, no team ownership.
+
+The 50-point weight reflects that finding a company using your SDK is 2-3x more valuable for GTM purposes than finding an individual developer.
+
+---
+
+## Common Misclassifications
+
+**Large open-source projects classified as company_org:**
+A GitHub org doesn't always mean a company. The Linux Foundation, Apache, CNCF, and similar foundations use orgs. These are valuable signal (indicates enterprise-grade adoption) but the outreach angle differs -- reach out to the project maintainers, not a sales contact.
+
+**Tutorial repos with high stars:**
+A well-known tutorial like "stripe-payments-tutorial" may have 500 stars. The adoption score caps the star contribution at 50 points (max 500 stars / 10), but the tutorial detection should have already filtered it. If a high-star tutorial slips through, check `data_quality_flags`.
+
+**New repos with inflated scores:**
+A repo created this week by an organization (50 points) with active recent push (50 points) scores 100+ even with 0 stars. This is intentional: a company adopting the SDK this week is a higher-priority outreach target than a solo dev with 50 stars who hasn't pushed in 6 months.