soustack 0.1.2 → 0.1.3

package/README.md

@@ -46,9 +46,10 @@ npm install soustack
 - **Schema.org Conversion**:
   - `fromSchemaOrg()` (Schema.org JSON-LD → Soustack)
   - `toSchemaOrg()` (Soustack → Schema.org JSON-LD)
-- **Web Scraping**:
+- **Web Scraping**: 
   - `scrapeRecipe()` fetches a recipe page and extracts Schema.org recipe data (Node.js only)
-  - `extractRecipeFromHTML()` extracts recipe data from HTML string (browser & Node.js compatible)
+  - `extractRecipeFromHTML()` extracts recipe data from HTML string, returns Soustack format (browser & Node.js compatible)
+  - `extractSchemaOrgRecipeFromHTML()` extracts raw Schema.org recipe data from HTML string (browser & Node.js compatible)
   - Supports JSON-LD (`<script type="application/ld+json">`) and Microdata (`itemscope/itemtype`)
 
 ## Programmatic Usage
@@ -57,6 +58,7 @@ npm install soustack
 import {
   scrapeRecipe,
   extractRecipeFromHTML,
+  extractSchemaOrgRecipeFromHTML,
   fromSchemaOrg,
   toSchemaOrg,
   validateRecipe,
@@ -73,9 +75,16 @@ const computed = scaleRecipe(recipe, 2);
 const scraped = await scrapeRecipe('https://example.com/recipe');
 
 // Extract recipe from HTML string (browser & Node.js compatible)
+// Option 1: Get Soustack format directly
 const html = await fetch('https://example.com/recipe').then((r) => r.text());
 const recipe = extractRecipeFromHTML(html);
 
+// Option 2: Get Schema.org format first (for inspection/modification)
+const schemaOrgRecipe = extractSchemaOrgRecipeFromHTML(html);
+if (schemaOrgRecipe) {
+  const soustackRecipe = fromSchemaOrg(schemaOrgRecipe);
+}
+
 // Convert Schema.org → Soustack
 const soustack = fromSchemaOrg(schemaOrgJsonLd);
 
@@ -115,9 +124,11 @@ const recipe = await scrapeRecipe('https://example.com/recipe', {
 });
 ```
 
-### Browser: `extractRecipeFromHTML()`
+### Browser: `extractRecipeFromHTML()` and `extractSchemaOrgRecipeFromHTML()`
+
+#### `extractRecipeFromHTML()` - Returns Soustack Format
 
-`extractRecipeFromHTML(html)` extracts recipe data from an HTML string. **Works in both browser and Node.js**. Perfect for browser usage where you fetch HTML yourself (with cookies/session for authenticated content).
+`extractRecipeFromHTML(html)` extracts recipe data from an HTML string and returns it in Soustack format. **Works in both browser and Node.js**. Perfect for browser usage where you fetch HTML yourself (with cookies/session for authenticated content).
 
 ```ts
 import { extractRecipeFromHTML } from 'soustack';
@@ -125,15 +136,39 @@ import { extractRecipeFromHTML } from 'soustack';
 // In browser: fetch HTML yourself (bypasses CORS, uses your cookies/session)
 const response = await fetch('https://example.com/recipe');
 const html = await response.text();
-const recipe = extractRecipeFromHTML(html);
+const recipe = extractRecipeFromHTML(html); // Already in Soustack format
+```
+
+#### `extractSchemaOrgRecipeFromHTML()` - Returns Schema.org Format
+
+`extractSchemaOrgRecipeFromHTML(html)` extracts the raw Schema.org recipe data from HTML. Returns `null` if no recipe is found. Use this when you need to inspect, debug, or modify the Schema.org data before converting to Soustack format.
+
+```ts
+import { extractSchemaOrgRecipeFromHTML, fromSchemaOrg } from 'soustack';
+
+// In browser: fetch HTML yourself
+const response = await fetch('https://example.com/recipe');
+const html = await response.text();
+
+// Extract Schema.org format (for inspection/modification)
+const schemaOrgRecipe = extractSchemaOrgRecipeFromHTML(html);
+
+if (schemaOrgRecipe) {
+  // Inspect or modify Schema.org data before converting
+  console.log('Found recipe:', schemaOrgRecipe.name);
+  
+  // Convert to Soustack format when ready
+  const soustackRecipe = fromSchemaOrg(schemaOrgRecipe);
+}
 ```
 
-**Why use `extractRecipeFromHTML()` in browsers?**
+**Why use these functions in browsers?**
 
 - ✅ No CORS issues — you fetch HTML yourself
 - ✅ Works with authenticated/paywalled content — uses browser cookies
 - ✅ Smaller bundle — no Node.js dependencies
 - ✅ Universal — works in both browser and Node.js environments
+- ✅ Flexible — choose Schema.org format for inspection/modification, or Soustack format for direct use
 
 ### CLI
 

package/dist/cli/index.js

@@ -2006,14 +2006,30 @@ function extractRecipe(html) {
     return extractRecipeBrowser(html);
   }
   const jsonLdRecipe = extractJsonLd(html);
-  fetch("http://127.0.0.1:7243/ingest/7225c3b5-9ac2-4c94-b561-807ca9003b66", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ location: "scraper/extractors/index.ts:6", message: "JSON-LD extraction result", data: { hasJsonLd: !!jsonLdRecipe }, timestamp: Date.now(), sessionId: "debug-session", runId: "run1", hypothesisId: "C,D" }) }).catch(() => {
-  });
+  if (typeof process === "undefined" || process.env.NODE_ENV !== "test") {
+    try {
+      const globalFetch = typeof globalThis !== "undefined" && typeof globalThis.fetch !== "undefined" ? globalThis.fetch : null;
+      if (globalFetch) {
+        globalFetch("http://127.0.0.1:7243/ingest/7225c3b5-9ac2-4c94-b561-807ca9003b66", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ location: "scraper/extractors/index.ts:6", message: "JSON-LD extraction result", data: { hasJsonLd: !!jsonLdRecipe }, timestamp: Date.now(), sessionId: "debug-session", runId: "run1", hypothesisId: "C,D" }) }).catch(() => {
+        });
+      }
+    } catch {
+    }
+  }
   if (jsonLdRecipe) {
     return { recipe: jsonLdRecipe, source: "jsonld" };
   }
   const microdataRecipe = extractMicrodata(html);
-  fetch("http://127.0.0.1:7243/ingest/7225c3b5-9ac2-4c94-b561-807ca9003b66", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ location: "scraper/extractors/index.ts:12", message: "Microdata extraction result", data: { hasMicrodata: !!microdataRecipe }, timestamp: Date.now(), sessionId: "debug-session", runId: "run1", hypothesisId: "D" }) }).catch(() => {
-  });
+  if (typeof process === "undefined" || process.env.NODE_ENV !== "test") {
+    try {
+      const globalFetch = typeof globalThis !== "undefined" && typeof globalThis.fetch !== "undefined" ? globalThis.fetch : null;
+      if (globalFetch) {
+        globalFetch("http://127.0.0.1:7243/ingest/7225c3b5-9ac2-4c94-b561-807ca9003b66", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ location: "scraper/extractors/index.ts:12", message: "Microdata extraction result", data: { hasMicrodata: !!microdataRecipe }, timestamp: Date.now(), sessionId: "debug-session", runId: "run1", hypothesisId: "D" }) }).catch(() => {
+        });
+      }
+    } catch {
+    }
+  }
   if (microdataRecipe) {
     return { recipe: microdataRecipe, source: "microdata" };
   }
@@ -2022,20 +2038,52 @@ function extractRecipe(html) {
 
 // src/scraper/index.ts
 async function scrapeRecipe(url, options = {}) {
-  fetch("http://127.0.0.1:7243/ingest/7225c3b5-9ac2-4c94-b561-807ca9003b66", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ location: "scraper/index.ts:7", message: "scrapeRecipe entry", data: { url, hasOptions: !!options }, timestamp: Date.now(), sessionId: "debug-session", runId: "run1", hypothesisId: "A,B,C,D,E" }) }).catch(() => {
-  });
+  if (typeof process === "undefined" || process.env.NODE_ENV !== "test") {
+    try {
+      const globalFetch = typeof globalThis !== "undefined" && typeof globalThis.fetch !== "undefined" ? globalThis.fetch : null;
+      if (globalFetch) {
+        globalFetch("http://127.0.0.1:7243/ingest/7225c3b5-9ac2-4c94-b561-807ca9003b66", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ location: "scraper/index.ts:7", message: "scrapeRecipe entry", data: { url, hasOptions: !!options }, timestamp: Date.now(), sessionId: "debug-session", runId: "run1", hypothesisId: "A,B,C,D,E" }) }).catch(() => {
+        });
+      }
+    } catch {
+    }
+  }
   const html = await fetchPage(url, options);
-  fetch("http://127.0.0.1:7243/ingest/7225c3b5-9ac2-4c94-b561-807ca9003b66", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ location: "scraper/index.ts:9", message: "HTML fetched", data: { htmlLength: html?.length, htmlPreview: html?.substring(0, 200) }, timestamp: Date.now(), sessionId: "debug-session", runId: "run1", hypothesisId: "B" }) }).catch(() => {
-  });
+  if (typeof process === "undefined" || process.env.NODE_ENV !== "test") {
+    try {
+      const globalFetch = typeof globalThis !== "undefined" && typeof globalThis.fetch !== "undefined" ? globalThis.fetch : null;
+      if (globalFetch) {
+        globalFetch("http://127.0.0.1:7243/ingest/7225c3b5-9ac2-4c94-b561-807ca9003b66", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ location: "scraper/index.ts:9", message: "HTML fetched", data: { htmlLength: html?.length, htmlPreview: html?.substring(0, 200) }, timestamp: Date.now(), sessionId: "debug-session", runId: "run1", hypothesisId: "B" }) }).catch(() => {
+        });
+      }
+    } catch {
+    }
+  }
   const { recipe } = extractRecipe(html);
-  fetch("http://127.0.0.1:7243/ingest/7225c3b5-9ac2-4c94-b561-807ca9003b66", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ location: "scraper/index.ts:11", message: "extractRecipe result", data: { hasRecipe: !!recipe, recipeType: recipe?.["@type"], recipeName: recipe?.name }, timestamp: Date.now(), sessionId: "debug-session", runId: "run1", hypothesisId: "A,C,D" }) }).catch(() => {
-  });
+  if (typeof process === "undefined" || process.env.NODE_ENV !== "test") {
+    try {
+      const globalFetch = typeof globalThis !== "undefined" && typeof globalThis.fetch !== "undefined" ? globalThis.fetch : null;
+      if (globalFetch) {
+        globalFetch("http://127.0.0.1:7243/ingest/7225c3b5-9ac2-4c94-b561-807ca9003b66", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ location: "scraper/index.ts:11", message: "extractRecipe result", data: { hasRecipe: !!recipe, recipeType: recipe?.["@type"], recipeName: recipe?.name }, timestamp: Date.now(), sessionId: "debug-session", runId: "run1", hypothesisId: "A,C,D" }) }).catch(() => {
+        });
+      }
+    } catch {
+    }
+  }
   if (!recipe) {
     throw new Error("No Schema.org recipe data found in page");
   }
   const soustackRecipe = fromSchemaOrg(recipe);
-  fetch("http://127.0.0.1:7243/ingest/7225c3b5-9ac2-4c94-b561-807ca9003b66", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ location: "scraper/index.ts:17", message: "fromSchemaOrg result", data: { hasSoustackRecipe: !!soustackRecipe, soustackRecipeName: soustackRecipe?.name }, timestamp: Date.now(), sessionId: "debug-session", runId: "run1", hypothesisId: "A" }) }).catch(() => {
-  });
+  if (typeof process === "undefined" || process.env.NODE_ENV !== "test") {
+    try {
+      const globalFetch = typeof globalThis !== "undefined" && typeof globalThis.fetch !== "undefined" ? globalThis.fetch : null;
+      if (globalFetch) {
+        globalFetch("http://127.0.0.1:7243/ingest/7225c3b5-9ac2-4c94-b561-807ca9003b66", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ location: "scraper/index.ts:17", message: "fromSchemaOrg result", data: { hasSoustackRecipe: !!soustackRecipe, soustackRecipeName: soustackRecipe?.name }, timestamp: Date.now(), sessionId: "debug-session", runId: "run1", hypothesisId: "A" }) }).catch(() => {
+        });
+      }
+    } catch {
+    }
+  }
   if (!soustackRecipe) {
     throw new Error("Schema.org data did not include a valid recipe");
   }