@adverant/nexus-memory-skill 2.1.0 → 2.2.0

package/SKILL.md

@@ -418,6 +418,145 @@ curl -X POST "https://api.adverant.ai/fileprocess/api/process" \
 
 ---
 
+## Upload Document Hook (Full Knowledge Extraction)
+
+The `upload-document.sh` hook provides a streamlined way to upload files with **full auto-discovery and knowledge extraction** enabled by default.
+
+### Auto-Discovery Features (All Enabled by Default)
+
+When you upload a document, the system automatically:
+
+| Feature | Description | Accuracy |
+|---------|-------------|----------|
+| **Smart File Detection** | Magic byte detection for accurate MIME type | 100% |
+| **Intelligent Routing** | Auto-routes to MageAgent, VideoAgent, or CyberAgent | Automatic |
+| **3-Tier OCR Cascade** | Tesseract → GPT-4o Vision → Claude Opus | Auto-escalates |
+| **Layout Analysis** | Document structure preservation | 99.2% |
+| **Table Extraction** | Tables converted to structured data | 97.9% |
+| **Document DNA** | Triple-layer storage (semantic + structural + original) | Full fidelity |
+| **Entity Extraction** | People, places, organizations → Neo4j | Automatic |
+| **Vector Embeddings** | VoyageAI embeddings → Qdrant | Automatic |
+
+### Basic Usage
+
+```bash
+# Upload a single file
+~/.claude/hooks/upload-document.sh ./document.pdf
+
+# Upload and wait for processing to complete
+~/.claude/hooks/upload-document.sh ./book.pdf --wait
+
+# Upload with custom tags for easier recall
+~/.claude/hooks/upload-document.sh ./research.pdf --wait --tags=research,ai,papers
+```
+
+### Batch Upload (Multiple Files)
+
+```bash
+# Upload 3 books at once
+~/.claude/hooks/upload-document.sh book1.pdf book2.pdf book3.pdf --batch --wait
+
+# Upload entire directory of PDFs
+~/.claude/hooks/upload-document.sh ./docs/*.pdf --batch --wait --tags=documentation
+```
+
+### Processing Options
+
+| Flag | Description |
+|------|-------------|
+| `--wait` | Wait for processing to complete and show results |
+| `--batch` | Enable batch mode for multiple files |
+| `--tags=a,b,c` | Add custom tags for easier recall |
+| `--no-entities` | Skip entity extraction (faster, less rich) |
+| `--prefer-speed` | Use faster OCR (may reduce accuracy) |
+| `--poll-interval=N` | Poll interval in seconds (default: 5) |
+
+### Supported File Types
+
+The upload hook supports **ALL file types** through intelligent routing:
+
+- **Documents**: PDF, DOCX, DOC, TXT, MD, HTML, CSV, XML, JSON
+- **Images**: JPEG, PNG, GIF, TIFF, WebP (with OCR)
+- **Videos**: MP4, MOV, AVI, MKV, WebM, FLV
+- **Archives**: ZIP, RAR, 7z, TAR, TAR.GZ, TAR.BZ2
+- **Geospatial**: GeoJSON, Shapefile, GeoTIFF, KML
+- **Point Cloud**: LAS, LAZ, PLY, PCD, E57
+- **Code**: Any programming language
+- **Any binary format** (routed to CyberAgent if suspicious)
+
+### Example Output (--wait mode)
+
+```
+[upload-document] Uploading: research-paper.pdf (12MB)
+Job ID: abc123-def456-ghi789
+
+╔══════════════════════════════════════════════════════════════╗
+║              PROCESSING COMPLETE: research-paper.pdf
+╚══════════════════════════════════════════════════════════════╝
+
+📄 Document Type:     academic_paper
+📑 Pages:             42
+📝 Words:             15,230
+
+🔍 Auto-Discovery Results:
+   • OCR Tier Used:   tesseract (text-based PDF)
+   • Tables Found:    8
+   • Entities:        127
+   • GraphRAG:        true
+
+🏷️  Extracted Entities:
+   • Dr. Emily Chen (person)
+   • Stanford University (organization)
+   • NeurIPS 2024 (event)
+   • Transformer Architecture (concept)
+   ... and 123 more
+
+💡 To recall this content:
+   echo '{"query": "<your search>"}' | recall-memory.sh
+```
+
+### Recalling Uploaded Content
+
+After upload, documents are immediately searchable:
+
+```bash
+# Search by content
+echo '{"query": "transformer architecture research"}' | ~/.claude/hooks/recall-memory.sh
+
+# Search by entity
+echo '{"query": "papers by Dr. Emily Chen"}' | ~/.claude/hooks/recall-memory.sh
+
+# Search by tag
+echo '{"query": "research papers tagged ai"}' | ~/.claude/hooks/recall-memory.sh
+```
+
+### Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `NEXUS_API_KEY` | (required) | API key for authentication |
+| `NEXUS_API_URL` | `https://api.adverant.ai` | API endpoint |
+| `NEXUS_COMPANY_ID` | `adverant` | Company identifier |
+| `NEXUS_APP_ID` | `claude-code` | Application identifier |
+| `NEXUS_VERBOSE` | `0` | Set to `1` for debug output |
+
+### Troubleshooting
+
+**File too large:**
+Maximum file size is 5GB. For larger files, consider splitting or using the direct API.
+
+**Processing takes too long:**
+- Large PDFs with many images trigger OCR cascade
+- Use `--prefer-speed` for faster (but less accurate) processing
+- Increase `--poll-interval` for large batches
+
+**Entities not extracted:**
+- Ensure you didn't use `--no-entities`
+- Check NEXUS_VERBOSE=1 for detailed logs
+- Some file types don't support entity extraction
+
+---
+
 ## Store Memory
 
 ```bash

package/hooks/upload-document.sh

@@ -1,7 +1,19 @@
 #!/bin/bash
 #
-# Nexus Memory - Upload Document Hook
-# Uploads documents to FileProcessAgent for intelligent processing.
+# Nexus Memory - Upload Document Hook (v2.2.0)
+# Uploads documents to FileProcessAgent for intelligent processing with
+# FULL KNOWLEDGE EXTRACTION enabled by default.
+#
+# Auto-Discovery Features (enabled automatically):
+# - Smart file type detection via magic bytes
+# - Intelligent routing: MageAgent (docs), VideoAgent (video), CyberAgent (binaries)
+# - 3-tier OCR cascade: Tesseract → GPT-4o Vision → Claude Opus (auto-escalates)
+# - Layout analysis: 99.2% accuracy (Dockling-level)
+# - Table extraction: 97.9% accuracy
+# - Document DNA: Triple-layer storage (semantic + structural + original)
+# - Entity extraction → Neo4j knowledge graph
+# - Vector embeddings → Qdrant for semantic search
+# - Content findable via recall-memory.sh
 #
 # Supports ALL file types including:
 # - Documents: PDF, DOCX, TXT, MD, HTML, etc.
@@ -10,15 +22,21 @@
 # - Archives: ZIP, RAR, 7z, TAR, GZIP
 # - Geospatial: GeoJSON, Shapefile, GeoTIFF, KML (via intelligent routing)
 # - Point Cloud: LAS, LAZ, PLY, PCD, E57 (via intelligent routing)
+# - Code repositories: Automatically detected and processed
 # - Any other binary format (routed to appropriate processor)
 #
 # Usage:
-#   upload-document.sh <file_path> [--wait] [--poll-interval=5]
+#   upload-document.sh <file_path> [options]
+#   upload-document.sh <file1> <file2> ... --batch [options]
 #
 # Arguments:
-#   file_path      Path to the file to upload (required)
-#   --wait         Wait for processing to complete and return results
-#   --poll-interval=N  Poll interval in seconds (default: 5)
+#   file_path         Path to the file(s) to upload (required)
+#   --wait            Wait for processing to complete and return results
+#   --poll-interval=N Poll interval in seconds (default: 5)
+#   --batch           Process multiple files (list files before this flag)
+#   --tags=a,b,c      Add custom tags for recall (comma-separated)
+#   --no-entities     Skip entity extraction to knowledge graph
+#   --prefer-speed    Use faster OCR (may reduce accuracy for scanned docs)
 #
 # Environment Variables:
 #   NEXUS_API_KEY     - API key for authentication (REQUIRED)
@@ -29,9 +47,10 @@
 #
 # Examples:
 #   upload-document.sh ./document.pdf
-#   upload-document.sh ./large-dataset.csv --wait
+#   upload-document.sh ./book.pdf --wait
+#   upload-document.sh ./data.csv --wait --tags=dataset,sales
+#   upload-document.sh book1.pdf book2.pdf book3.pdf --batch --wait
 #   upload-document.sh ./video.mp4 --wait --poll-interval=10
-#   upload-document.sh ./pointcloud.las --wait
 #
 
 set -o pipefail
@@ -63,12 +82,27 @@ log_info() {
 }
 
 print_usage() {
-  echo "Usage: upload-document.sh <file_path> [--wait] [--poll-interval=N]"
+  echo "Usage: upload-document.sh <file_path> [options]"
+  echo "       upload-document.sh <file1> <file2> ... --batch [options]"
   echo ""
   echo "Arguments:"
-  echo "  file_path         Path to the file to upload (required)"
+  echo "  file_path         Path to the file(s) to upload (required)"
   echo "  --wait            Wait for processing to complete"
   echo "  --poll-interval=N Poll interval in seconds (default: 5)"
+  echo "  --batch           Process multiple files (list files before this flag)"
+  echo "  --tags=a,b,c      Add custom tags for recall (comma-separated)"
+  echo "  --no-entities     Skip entity extraction to knowledge graph"
+  echo "  --prefer-speed    Use faster OCR (may reduce accuracy)"
+  echo ""
+  echo "Auto-Discovery Features (enabled by default):"
+  echo "  • Smart file type detection via magic bytes"
+  echo "  • Intelligent routing: MageAgent, VideoAgent, CyberAgent"
+  echo "  • 3-tier OCR cascade (auto-escalates for quality)"
+  echo "  • Layout analysis (99.2% accuracy)"
+  echo "  • Table extraction (97.9% accuracy)"
+  echo "  • Entity extraction → Knowledge graph"
+  echo "  • Vector embeddings → Semantic search"
+  echo "  • Content findable via recall-memory.sh"
   echo ""
   echo "Supported file types:"
   echo "  Documents:  PDF, DOCX, DOC, TXT, MD, HTML, CSV, XML, JSON"
@@ -77,13 +111,16 @@ print_usage() {
   echo "  Archives:   ZIP, RAR, 7z, TAR, TAR.GZ, TAR.BZ2"
   echo "  Geospatial: GeoJSON, Shapefile, GeoTIFF, KML"
   echo "  Point Cloud: LAS, LAZ, PLY, PCD, E57"
+  echo "  Code:       Any programming language"
   echo "  Any other binary format"
   echo ""
   echo "Maximum file size: 5GB"
   echo ""
   echo "Examples:"
   echo "  upload-document.sh ./document.pdf"
-  echo "  upload-document.sh ./data.csv --wait"
+  echo "  upload-document.sh ./book.pdf --wait"
+  echo "  upload-document.sh ./data.csv --wait --tags=dataset,sales"
+  echo "  upload-document.sh book1.pdf book2.pdf book3.pdf --batch --wait"
   echo "  upload-document.sh ./video.mp4 --wait --poll-interval=10"
 }
 
@@ -109,9 +146,13 @@ if ! command -v jq &> /dev/null; then
 fi
 
 # Parse arguments
-FILE_PATH=""
+FILES=()
 WAIT_FOR_COMPLETION=0
 POLL_INTERVAL=5
+BATCH_MODE=0
+CUSTOM_TAGS=""
+EXTRACT_ENTITIES=1
+PREFER_SPEED=0
 
 while [[ $# -gt 0 ]]; do
   case $1 in
@@ -123,6 +164,22 @@ while [[ $# -gt 0 ]]; do
       POLL_INTERVAL="${1#*=}"
       shift
       ;;
+    --batch)
+      BATCH_MODE=1
+      shift
+      ;;
+    --tags=*)
+      CUSTOM_TAGS="${1#*=}"
+      shift
+      ;;
+    --no-entities)
+      EXTRACT_ENTITIES=0
+      shift
+      ;;
+    --prefer-speed)
+      PREFER_SPEED=1
+      shift
+      ;;
     --help|-h)
       print_usage
       exit 0
@@ -133,153 +190,348 @@ while [[ $# -gt 0 ]]; do
       exit 1
       ;;
     *)
-      if [[ -z "$FILE_PATH" ]]; then
-        FILE_PATH="$1"
-      else
-        log_error "Unexpected argument: $1"
-        print_usage
-        exit 1
-      fi
+      # Collect file paths
+      FILES+=("$1")
       shift
       ;;
   esac
 done
 
-# Validate file path
-if [[ -z "$FILE_PATH" ]]; then
-  log_error "File path is required"
+# Validate files
+if [[ ${#FILES[@]} -eq 0 ]]; then
+  log_error "At least one file path is required"
   print_usage
   exit 1
 fi
 
-if [[ ! -f "$FILE_PATH" ]]; then
-  log_error "File not found: $FILE_PATH"
+# If not batch mode but multiple files provided, error
+if [[ "$BATCH_MODE" == "0" ]] && [[ ${#FILES[@]} -gt 1 ]]; then
+  log_error "Multiple files require --batch flag"
+  log_error "Usage: upload-document.sh file1.pdf file2.pdf --batch"
   exit 1
 fi
 
-# Get file info
-FILE_NAME=$(basename "$FILE_PATH")
-FILE_SIZE=$(wc -c < "$FILE_PATH" | tr -d ' ')
-FILE_SIZE_MB=$((FILE_SIZE / 1024 / 1024))
+# Validate all files exist
+for file in "${FILES[@]}"; do
+  if [[ ! -f "$file" ]]; then
+    log_error "File not found: $file"
+    exit 1
+  fi
+done
 
-# Check file size (max 5GB)
-MAX_SIZE=$((5 * 1024 * 1024 * 1024))
-if [[ "$FILE_SIZE" -gt "$MAX_SIZE" ]]; then
-  log_error "File too large. Maximum size is 5GB. Your file: ${FILE_SIZE_MB}MB"
-  exit 1
-fi
+# Build processing metadata JSON with hints for aggressive extraction
+build_metadata() {
+  local file_name="$1"
+  local tags_json="[]"
 
-log "File: $FILE_PATH"
-log "Size: $FILE_SIZE bytes (${FILE_SIZE_MB}MB)"
+  # Convert comma-separated tags to JSON array
+  if [[ -n "$CUSTOM_TAGS" ]]; then
+    tags_json=$(echo "$CUSTOM_TAGS" | tr ',' '\n' | jq -R . | jq -s .)
+  fi
 
-# Display upload info
-if [[ "$FILE_SIZE_MB" -gt 100 ]]; then
-  log_info "Uploading large file: $FILE_NAME (${FILE_SIZE_MB}MB) - this may take a while..."
-else
-  log_info "Uploading: $FILE_NAME (${FILE_SIZE_MB}MB)"
-fi
+  # Determine OCR preference
+  local prefer_accuracy="true"
+  if [[ "$PREFER_SPEED" == "1" ]]; then
+    prefer_accuracy="false"
+  fi
 
-# Upload file via multipart form
-log "Uploading to $FILEPROCESS_URL"
+  # Determine entity extraction
+  local extract_entities="true"
+  if [[ "$EXTRACT_ENTITIES" == "0" ]]; then
+    extract_entities="false"
+  fi
 
-RESPONSE=$(curl -s -w "\n%{http_code}" -X POST "$FILEPROCESS_URL" \
-  -H "Authorization: Bearer $NEXUS_API_KEY" \
-  -H "X-Company-ID: $COMPANY_ID" \
-  -H "X-App-ID: $APP_ID" \
-  -H "X-User-ID: ${USER:-unknown}" \
-  -F "file=@${FILE_PATH}" \
-  -F "userId=${USER:-unknown}" \
-  --max-time 600 2>&1)
+  cat <<EOF
+{
+  "source": "nexus-memory-skill",
+  "version": "2.2.0",
+  "preferAccuracy": ${prefer_accuracy},
+  "forceEntityExtraction": ${extract_entities},
+  "storeInKnowledgeGraph": ${extract_entities},
+  "enableDocumentDNA": true,
+  "tags": ${tags_json},
+  "uploadedBy": "${USER:-unknown}",
+  "uploadedAt": "$(date -u +"%Y-%m-%dT%H:%M:%SZ")"
+}
+EOF
+}
 
-# Parse response
-HTTP_CODE=$(echo "$RESPONSE" | tail -n1)
-BODY=$(echo "$RESPONSE" | sed '$d')
+# Function to upload a single file
+upload_file() {
+  local FILE_PATH="$1"
+  local FILE_NAME=$(basename "$FILE_PATH")
+  local FILE_SIZE=$(wc -c < "$FILE_PATH" | tr -d ' ')
+  local FILE_SIZE_MB=$((FILE_SIZE / 1024 / 1024))
+
+  # Check file size (max 5GB)
+  local MAX_SIZE=$((5 * 1024 * 1024 * 1024))
+  if [[ "$FILE_SIZE" -gt "$MAX_SIZE" ]]; then
+    log_error "File too large: $FILE_NAME (${FILE_SIZE_MB}MB). Maximum size is 5GB."
+    return 1
+  fi
 
-log "Response code: $HTTP_CODE"
+  log "File: $FILE_PATH"
+  log "Size: $FILE_SIZE bytes (${FILE_SIZE_MB}MB)"
 
-# Check for upload errors (200, 201, 202 are all success codes)
-if [[ "$HTTP_CODE" != "200" ]] && [[ "$HTTP_CODE" != "201" ]] && [[ "$HTTP_CODE" != "202" ]]; then
-  log_error "Failed to upload document (HTTP $HTTP_CODE)"
-  if [[ -n "$BODY" ]]; then
+  # Display upload info
+  if [[ "$FILE_SIZE_MB" -gt 100 ]]; then
+    log_info "Uploading large file: $FILE_NAME (${FILE_SIZE_MB}MB) - this may take a while..."
+  else
+    log_info "Uploading: $FILE_NAME (${FILE_SIZE_MB}MB)"
+  fi
+
+  # Build metadata with processing hints
+  local METADATA=$(build_metadata "$FILE_NAME")
+  log "Metadata: $METADATA"
+
+  # Upload file via multipart form with metadata hints
+  log "Uploading to $FILEPROCESS_URL"
+
+  local RESPONSE=$(curl -s -w "\n%{http_code}" -X POST "$FILEPROCESS_URL" \
+    -H "Authorization: Bearer $NEXUS_API_KEY" \
+    -H "X-Company-ID: $COMPANY_ID" \
+    -H "X-App-ID: $APP_ID" \
+    -H "X-User-ID: ${USER:-unknown}" \
+    -F "file=@${FILE_PATH}" \
+    -F "userId=${USER:-unknown}" \
+    -F "metadata=${METADATA}" \
+    --max-time 600 2>&1)
+
+  # Parse response
+  local HTTP_CODE=$(echo "$RESPONSE" | tail -n1)
+  local BODY=$(echo "$RESPONSE" | sed '$d')
+
+  log "Response code: $HTTP_CODE"
+
+  # Check for upload errors (200, 201, 202 are all success codes)
+  if [[ "$HTTP_CODE" != "200" ]] && [[ "$HTTP_CODE" != "201" ]] && [[ "$HTTP_CODE" != "202" ]]; then
+    log_error "Failed to upload document (HTTP $HTTP_CODE)"
+    if [[ -n "$BODY" ]]; then
+      echo "$BODY" | jq . 2>/dev/null || echo "$BODY"
+    fi
+    return 1
+  fi
+
+  # Parse job ID from response
+  local JOB_ID=$(echo "$BODY" | jq -r '.jobId // empty')
+
+  if [[ -z "$JOB_ID" ]]; then
+    log_error "No job ID returned from upload"
     echo "$BODY" | jq . 2>/dev/null || echo "$BODY"
+    return 1
   fi
-  exit 1
-fi
 
-# Parse job ID from response
-JOB_ID=$(echo "$BODY" | jq -r '.jobId // empty')
+  log_info "Document queued for processing"
+  echo "Job ID: $JOB_ID"
 
-if [[ -z "$JOB_ID" ]]; then
-  log_error "No job ID returned from upload"
-  echo "$BODY" | jq . 2>/dev/null || echo "$BODY"
-  exit 1
-fi
+  # Return job ID for tracking
+  echo "$JOB_ID"
+}
 
-log_info "Document queued for processing"
-echo "Job ID: $JOB_ID"
+# Function to display detailed results
+display_results() {
+  local STATUS_RESPONSE="$1"
+  local FILE_NAME="$2"
 
-# If not waiting, exit here
-if [[ "$WAIT_FOR_COMPLETION" == "0" ]]; then
   echo ""
-  echo "To check status: curl -s \"$JOBS_URL/$JOB_ID\" | jq ."
-  exit 0
-fi
-
-# Wait for processing to complete
-log_info "Waiting for processing to complete (polling every ${POLL_INTERVAL}s)..."
+  echo "╔══════════════════════════════════════════════════════════════╗"
+  echo "║              PROCESSING COMPLETE: $FILE_NAME"
+  echo "╚══════════════════════════════════════════════════════════════╝"
+  echo ""
 
-MAX_WAIT=3600  # 1 hour max wait
-WAITED=0
+  # Extract key metrics from response
+  local ENTITY_COUNT=$(echo "$STATUS_RESPONSE" | jq -r '.result.entities // .entities // [] | length' 2>/dev/null)
+  local TABLE_COUNT=$(echo "$STATUS_RESPONSE" | jq -r '.result.tables // .tables // [] | length' 2>/dev/null)
+  local OCR_TIER=$(echo "$STATUS_RESPONSE" | jq -r '.result.ocrTier // .ocrTier // "auto"' 2>/dev/null)
+  local PAGE_COUNT=$(echo "$STATUS_RESPONSE" | jq -r '.result.pageCount // .pageCount // "N/A"' 2>/dev/null)
+  local WORD_COUNT=$(echo "$STATUS_RESPONSE" | jq -r '.result.wordCount // .wordCount // "N/A"' 2>/dev/null)
+  local DOC_TYPE=$(echo "$STATUS_RESPONSE" | jq -r '.result.documentType // .documentType // "unknown"' 2>/dev/null)
+  local GRAPHRAG_STORED=$(echo "$STATUS_RESPONSE" | jq -r '.result.storedInGraphRAG // .storedInGraphRAG // false' 2>/dev/null)
+
+  echo "📄 Document Type:     $DOC_TYPE"
+  echo "📑 Pages:             $PAGE_COUNT"
+  echo "📝 Words:             $WORD_COUNT"
+  echo ""
+  echo "🔍 Auto-Discovery Results:"
+  echo "   • OCR Tier Used:   $OCR_TIER"
+  echo "   • Tables Found:    $TABLE_COUNT"
+  echo "   • Entities:        $ENTITY_COUNT"
+  echo "   • GraphRAG:        $GRAPHRAG_STORED"
+  echo ""
 
-while [[ "$WAITED" -lt "$MAX_WAIT" ]]; do
-  sleep "$POLL_INTERVAL"
-  WAITED=$((WAITED + POLL_INTERVAL))
+  # Show extracted entities if any
+  if [[ "$ENTITY_COUNT" != "0" ]] && [[ "$ENTITY_COUNT" != "null" ]]; then
+    echo "🏷️  Extracted Entities:"
+    echo "$STATUS_RESPONSE" | jq -r '.result.entities // .entities // [] | .[:10][] | "   • \(.name // .text) (\(.type // "entity"))"' 2>/dev/null
+    if [[ "$ENTITY_COUNT" -gt 10 ]]; then
+      echo "   ... and $((ENTITY_COUNT - 10)) more"
+    fi
+    echo ""
+  fi
 
-  # Check job status
-  STATUS_RESPONSE=$(curl -s "$JOBS_URL/$JOB_ID" \
-    -H "Authorization: Bearer $NEXUS_API_KEY" \
-    -H "X-Company-ID: $COMPANY_ID" \
-    -H "X-App-ID: $APP_ID" \
-    -H "X-User-ID: ${USER:-unknown}" \
-    --max-time 30 2>/dev/null)
+  # Show recall command
+  echo "💡 To recall this content:"
+  echo "   echo '{\"query\": \"<your search>\"}' | recall-memory.sh"
+  echo ""
 
-  if [[ -z "$STATUS_RESPONSE" ]]; then
-    log "Waiting... (${WAITED}s elapsed)"
-    continue
+  # Show full JSON if verbose
+  if [[ "$VERBOSE" == "1" ]]; then
+    echo "=== FULL RESPONSE ==="
+    echo "$STATUS_RESPONSE" | jq .
   fi
+}
 
-  JOB_STATE=$(echo "$STATUS_RESPONSE" | jq -r '.state // .status // empty')
+# Function to wait for job completion
+wait_for_job() {
+  local JOB_ID="$1"
+  local FILE_NAME="$2"
+
+  log_info "Waiting for processing to complete (polling every ${POLL_INTERVAL}s)..."
+
+  local MAX_WAIT=3600  # 1 hour max wait
+  local WAITED=0
+
+  while [[ "$WAITED" -lt "$MAX_WAIT" ]]; do
+    sleep "$POLL_INTERVAL"
+    WAITED=$((WAITED + POLL_INTERVAL))
+
+    # Check job status
+    local STATUS_RESPONSE=$(curl -s "$JOBS_URL/$JOB_ID" \
+      -H "Authorization: Bearer $NEXUS_API_KEY" \
+      -H "X-Company-ID: $COMPANY_ID" \
+      -H "X-App-ID: $APP_ID" \
+      -H "X-User-ID: ${USER:-unknown}" \
+      --max-time 30 2>/dev/null)
+
+    if [[ -z "$STATUS_RESPONSE" ]]; then
+      log "Waiting... (${WAITED}s elapsed)"
+      continue
+    fi
+
+    local JOB_STATE=$(echo "$STATUS_RESPONSE" | jq -r '.state // .status // empty')
+
+    case "$JOB_STATE" in
+      "completed"|"finished"|"success")
+        display_results "$STATUS_RESPONSE" "$FILE_NAME"
+        return 0
+        ;;
+      "failed"|"error")
+        log_error "Processing failed for: $FILE_NAME"
+        echo ""
+        echo "=== ERROR DETAILS ==="
+        echo "$STATUS_RESPONSE" | jq .
+        return 1
+        ;;
+      "waiting"|"active"|"processing"|"pending")
+        local PROGRESS=$(echo "$STATUS_RESPONSE" | jq -r '.progress // empty')
+        local STAGE=$(echo "$STATUS_RESPONSE" | jq -r '.stage // empty')
+        if [[ -n "$PROGRESS" ]] && [[ -n "$STAGE" ]]; then
+          log "[$FILE_NAME] ${STAGE}: ${PROGRESS}% (${WAITED}s elapsed)"
+        elif [[ -n "$PROGRESS" ]]; then
+          log "[$FILE_NAME] Processing... ${PROGRESS}% (${WAITED}s elapsed)"
+        else
+          log "[$FILE_NAME] Processing... (${WAITED}s elapsed)"
+        fi
+        ;;
+      *)
+        log "[$FILE_NAME] Status: $JOB_STATE (${WAITED}s elapsed)"
+        ;;
+    esac
+  done
+
+  log_error "Timeout waiting for processing (${MAX_WAIT}s)"
+  echo "Job may still be processing. Check status manually:"
+  echo "curl -s \"$JOBS_URL/$JOB_ID\" | jq ."
+  return 1
+}
 
-  case "$JOB_STATE" in
-    "completed"|"finished"|"success")
-      log_info "Processing completed!"
-      echo ""
-      echo "=== PROCESSING RESULT ==="
-      echo "$STATUS_RESPONSE" | jq .
-      exit 0
-      ;;
-    "failed"|"error")
-      log_error "Processing failed!"
-      echo ""
-      echo "=== ERROR DETAILS ==="
-      echo "$STATUS_RESPONSE" | jq .
-      exit 1
-      ;;
-    "waiting"|"active"|"processing"|"pending")
-      PROGRESS=$(echo "$STATUS_RESPONSE" | jq -r '.progress // empty')
-      if [[ -n "$PROGRESS" ]]; then
-        log "Processing... ${PROGRESS}% (${WAITED}s elapsed)"
-      else
-        log "Processing... (${WAITED}s elapsed)"
+# ============================================================================
+# MAIN EXECUTION
+# ============================================================================
+
+# Track all job IDs for batch mode
+JOB_IDS=()
+FILE_NAMES=()
+FAILED_UPLOADS=0
+
+# Display batch info
+if [[ "$BATCH_MODE" == "1" ]]; then
+  log_info "Batch mode: uploading ${#FILES[@]} files"
+  echo ""
+fi
+
+# Upload all files
+for file in "${FILES[@]}"; do
+  FILE_NAME=$(basename "$file")
+  FILE_NAMES+=("$FILE_NAME")
+
+  # Upload file and capture job ID (last line of output)
+  UPLOAD_OUTPUT=$(upload_file "$file" 2>&1)
+  UPLOAD_EXIT_CODE=$?
+
+  if [[ $UPLOAD_EXIT_CODE -eq 0 ]]; then
+    # Extract job ID from output (last non-empty line that looks like a job ID)
+    JOB_ID=$(echo "$UPLOAD_OUTPUT" | grep -E '^[a-f0-9-]+$' | tail -1)
+    if [[ -n "$JOB_ID" ]]; then
+      JOB_IDS+=("$JOB_ID")
+    else
+      # Try to extract from "Job ID: xxx" format
+      JOB_ID=$(echo "$UPLOAD_OUTPUT" | grep "Job ID:" | sed 's/Job ID: //' | tr -d ' ')
+      if [[ -n "$JOB_ID" ]]; then
+        JOB_IDS+=("$JOB_ID")
       fi
-      ;;
-    *)
-      log "Status: $JOB_STATE (${WAITED}s elapsed)"
-      ;;
-  esac
+    fi
+    echo "$UPLOAD_OUTPUT"
+  else
+    log_error "Failed to upload: $FILE_NAME"
+    echo "$UPLOAD_OUTPUT"
+    FAILED_UPLOADS=$((FAILED_UPLOADS + 1))
+  fi
+
+  # Add spacing between files in batch mode
+  if [[ "$BATCH_MODE" == "1" ]]; then
+    echo ""
+  fi
 done
 
-log_error "Timeout waiting for processing (${MAX_WAIT}s)"
-echo "Job may still be processing. Check status manually:"
-echo "curl -s \"$JOBS_URL/$JOB_ID\" | jq ."
-exit 1
+# Summary for batch mode
+if [[ "$BATCH_MODE" == "1" ]]; then
+  echo "╔══════════════════════════════════════════════════════════════╗"
+  echo "║                    UPLOAD SUMMARY                            ║"
+  echo "╚══════════════════════════════════════════════════════════════╝"
+  echo "Total files:     ${#FILES[@]}"
+  echo "Uploaded:        ${#JOB_IDS[@]}"
+  echo "Failed:          $FAILED_UPLOADS"
+  echo ""
+fi
+
+# If not waiting, exit here
+if [[ "$WAIT_FOR_COMPLETION" == "0" ]]; then
+  if [[ ${#JOB_IDS[@]} -gt 0 ]]; then
+    echo "To check status:"
+    for i in "${!JOB_IDS[@]}"; do
+      echo "  curl -s \"$JOBS_URL/${JOB_IDS[$i]}\" | jq .  # ${FILE_NAMES[$i]}"
+    done
+  fi
+  exit $FAILED_UPLOADS
+fi
+
+# Wait for all jobs to complete
+FAILED_JOBS=0
+for i in "${!JOB_IDS[@]}"; do
+  JOB_ID="${JOB_IDS[$i]}"
+  FILE_NAME="${FILE_NAMES[$i]}"
+
+  if ! wait_for_job "$JOB_ID" "$FILE_NAME"; then
+    FAILED_JOBS=$((FAILED_JOBS + 1))
+  fi
+done
+
+# Final exit code
+TOTAL_FAILURES=$((FAILED_UPLOADS + FAILED_JOBS))
+if [[ "$TOTAL_FAILURES" -gt 0 ]]; then
+  log_error "$TOTAL_FAILURES file(s) failed to process"
+  exit 1
+fi
+
+exit 0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@adverant/nexus-memory-skill",
-  "version": "2.1.0",
+  "version": "2.2.0",
   "description": "Claude Code skill for persistent memory via Nexus GraphRAG - store and recall memories across all sessions and projects",
   "main": "SKILL.md",
   "type": "module",