sentiment_insights 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ab2970402e6ee32bfa46d70caaafa135d17e780528c76ab7d42c5a6d9a62acb5
-  data.tar.gz: ca8fcc08f054c4f348d53697c3a4fed6a9b36485d628c7b89b0112cc081b332f
+  metadata.gz: 74729added1e999a0e6277b89a95fb2ee934103f18df50b8187f6a86d04c833b
+  data.tar.gz: 3be0d99ae0ecc2148110d64bf7030d988e9a03810774d338e1c1ac58d735ed48
 SHA512:
-  metadata.gz: 1af62b7158097b907df3609521c75997f2919294c7a725f12bc7243d9b89721b66d1a0c982fc3daab1ab7e5623ed140db5360355b84e67b309d1392aaf515d21
-  data.tar.gz: e42bf88ffd443546a20617cfcab9818982c19180425047306e8ee59e022bc0d0de50b3908d2d3f2fed587fb373d31e6f4d7d7d799b79c40c10cbd33cb09bc964
+  metadata.gz: 563aecf46e7af1300bb84854f8af8c30eeaa874bf73eb49f18c44c5b2af075084b4fe8eaaf1503b43dad87e48225601f358c99caf658abdad5fd30423d799e22
+  data.tar.gz: 5f017d18b1a447e22399d161f493e3f01784eb917790b846548c0d63aef13f0b8df58101e197c0625c4dbfc209e34e5072a533f41026238468ad6e9fa1bb90ad

data/EXPORT_USAGE.md

@@ -0,0 +1,325 @@
+# SentimentInsights Export Feature Guide
+
+The SentimentInsights gem now includes comprehensive export functionality for all analysis results. Export your sentiment analysis, entity extraction, and key phrase results to CSV and Excel formats with advanced filtering and customization options.
+
+## 🚀 Quick Start
+
+```ruby
+# Run your analysis as usual
+result = SentimentInsights::Insights::Sentiment.new.analyze(entries)
+
+# Export to CSV (simplest usage)
+result.to_csv("my_analysis.csv")
+
+# Export to Excel
+result.to_excel("my_analysis.xlsx")
+
+# Auto-detect format from filename
+result.export("my_analysis.csv")  # Creates CSV
+result.export("my_analysis.xlsx") # Creates Excel
+```
+
+## 📊 CSV Export Structure
+
+### Sentiment Analysis CSV
+```csv
+response_id,text,sentiment_label,sentiment_score,segment_age_group,segment_region,timestamp
+r_1,"I love this product!",positive,0.9,18-25,North,2024-06-28T10:30:00Z
+r_2,"Service was terrible",negative,-0.8,26-35,South,2024-06-28T10:31:00Z
+
+SUMMARY STATISTICS
+Total Responses,150
+Positive Count,90
+Positive Percentage,60.0%
+Net Sentiment Score,40.0
+
+SEGMENT ANALYSIS  
+Segment Type,Segment Value,Total Count,Positive %,Neutral %,Negative %,Net Score
+Age Group,18-25,75,65.3%,18.7%,16.0%,49.3
+Age Group,26-35,45,53.3%,22.2%,24.4%,28.9
+```
+
+### Entity Extraction CSV
+```csv
+response_id,text,entities_found,segment_age_group,segment_region,timestamp
+r_1,"Apple released iPhone",apple:ORGANIZATION,iphone:PRODUCT,18-25,North,2024-06-28T10:30:00Z
+
+SUMMARY STATISTICS
+Total Unique Entities,25
+Organizations Mentioned,8
+Most Mentioned Entity,apple (5 times)
+
+ENTITY DETAILS
+Entity,Type,Total Mentions,Response IDs,Segment Distribution
+apple,ORGANIZATION,5,"r_1,r_3,r_7",age_group_18-25:3,age_group_26-35:2
+```
+
+### Key Phrases CSV
+```csv
+response_id,text,sentiment,sentiment_score,key_phrases_found,segment_age_group,timestamp
+r_1,"Great customer service!",positive,0.8,"customer service,great experience",18-25,2024-06-28T10:30:00Z
+
+PHRASE DETAILS
+Phrase,Total Mentions,Response IDs,Sentiment Distribution,Segment Distribution
+customer service,8,"r_1,r_4,r_7",positive:5,neutral:2,negative:1,age_group_18-25:4
+```
+
+## 🎛️ Advanced Export Options
+
+### Basic Configuration
+```ruby
+# Export with custom options
+result.to_csv("analysis.csv", {
+  include_summary: true,      # Include summary statistics
+  include_segments: true,     # Include segment analysis  
+  include_timestamp: true,    # Add timestamp column
+  timestamp_format: "%Y-%m-%d %H:%M:%S"
+})
+
+# Skip summary and segments (responses only)
+result.to_csv("responses_only.csv", {
+  include_summary: false,
+  include_segments: false
+})
+```
+
+### Dynamic Segment Handling
+The export system automatically detects and flattens all segment structures:
+
+```ruby
+entries = [
+  { 
+    answer: "Great product!", 
+    segment: { 
+      demographics: { age: "25-34", income: "high" },
+      location: { country: "USA", state: "CA" },
+      simple_field: "value"
+    }
+  }
+]
+
+result = analyzer.analyze(entries)
+result.to_csv("complex_segments.csv")
+
+# CSV headers will be:
+# response_id,text,sentiment_label,sentiment_score,
+# segment_demographics_age,segment_demographics_income,
+# segment_location_country,segment_location_state,
+# segment_simple_field,timestamp
+```
+
+## 🔍 Filtering and Custom Exports
+
+### Fluent Interface
+```ruby
+# Method chaining for custom exports
+result
+  .responses_only                    # Skip summary sections
+  .filter_sentiment(:positive)       # Only positive responses
+  .filter_segments(age_group: ["18-25", "26-35"])  # Specific age groups
+  .to_csv("young_positive_feedback.csv")
+```
+
+### Quick Filter Methods
+```ruby
+# Export only positive responses
+result.export_positive(:csv, "positive_feedback.csv")
+
+# Export only negative responses  
+result.export_negative(:excel, "issues_to_address.xlsx")
+
+# Export specific segments
+result.export_by_segment(:age_group, ["18-25"], :csv, "young_users.csv")
+result.export_by_segment(:region, ["North", "South"], :excel, "regional_analysis.xlsx")
+```
+
+### Advanced Filtering
+```ruby
+# Complex filter combinations
+result.exporter({
+  filter: {
+    sentiment: [:positive, :neutral],  # Exclude negative
+    segments: { 
+      age_group: ["18-25", "26-35"],
+      region: ["North", "West"] 
+    }
+  },
+  include_summary: false
+}).to_csv("filtered_analysis.csv")
+```
+
+## 📈 Excel Export Features
+
+Excel exports include multiple worksheets:
+
+1. **Responses** - Individual response data
+2. **Summary** - Statistical summaries  
+3. **Segment Analysis** - Breakdown by segments
+4. **Entity Details** - Entity-specific analysis (entities only)
+5. **Phrase Details** - Phrase-specific analysis (key phrases only)
+
+```ruby
+# Basic Excel export
+result.to_excel("comprehensive_report.xlsx")
+
+# Excel with custom options
+result.to_excel("detailed_report.xlsx", {
+  include_summary: true,
+  include_segments: true
+})
+```
+
+## 🚀 Batch Export
+
+Export to multiple formats at once:
+
+```ruby
+# Export to both CSV and Excel
+files = result.export_all("quarterly_analysis")
+# Returns: { csv: "quarterly_analysis.csv", excel: "quarterly_analysis.xlsx" }
+
+files.each do |format, filename|
+  puts "#{format.upcase} exported to: #{filename}"
+end
+```
+
+## 🔧 Advanced Usage with Exporter Class
+
+For maximum control, use the Exporter class directly:
+
+```ruby
+exporter = result.exporter({
+  include_summary: true,
+  include_segments: true,
+  filter: { sentiment: [:positive] }
+})
+
+# Multiple export operations
+csv_file = exporter.to_csv("positive_analysis.csv")
+excel_file = exporter.to_excel("positive_analysis.xlsx")
+
+# Specialized exports
+summary_only = exporter.to_csv_summary_only("summary.csv")
+responses_only = exporter.to_csv_responses_only("responses.csv")
+```
+
+## 🎯 Use Cases
+
+### Customer Feedback Analysis
+```ruby
+feedback = analyzer.analyze(customer_responses)
+
+# Export all feedback
+feedback.export_all("customer_feedback_analysis")
+
+# Export issues that need attention
+feedback.export_negative(:excel, "issues_to_resolve.xlsx")
+
+# Export by customer segment
+feedback.export_by_segment(:customer_tier, ["premium", "enterprise"], :csv, "premium_feedback.csv")
+```
+
+### Survey Response Processing
+```ruby
+survey_results = analyzer.analyze(survey_responses)
+
+# Comprehensive report for stakeholders
+survey_results.to_excel("survey_report.xlsx", {
+  include_summary: true,
+  include_segments: true
+})
+
+# Quick CSV for data team
+survey_results
+  .responses_only
+  .to_csv("raw_survey_data.csv")
+```
+
+### Regional Analysis
+```ruby
+regional_analysis = analyzer.analyze(regional_data)
+
+# Export by region
+%w[North South East West].each do |region|
+  regional_analysis.export_by_segment(:region, region, :csv, "#{region.downcase}_analysis.csv")
+end
+```
+
+## 📋 File Naming
+
+Auto-generated filenames include timestamps:
+- `sentiment_analysis_20240628_103045.csv`
+- `entities_analysis_20240628_103045.xlsx`
+- `key_phrases_analysis_20240628_103045.csv`
+
+## 🛠️ Installation Requirements
+
+Add to your Gemfile:
+```ruby
+gem 'sentiment_insights', '~> 0.3.0'
+gem 'rubyXL', '~> 3.4'  # For Excel export support
+```
+
+## 🔍 Error Handling
+
+```ruby
+begin
+  result.to_excel("report.xlsx")
+rescue => e
+  puts "Excel export failed: #{e.message}"
+  # Fallback to CSV
+  result.to_csv("report.csv")
+end
+```
+
+## 🎉 Integration Examples
+
+### Rails Controller
+```ruby
+class AnalyticsController < ApplicationController
+  def export_feedback
+    entries = FeedbackResponse.includes(:customer).map do |response|
+      {
+        answer: response.comment,
+        segment: {
+          customer_tier: response.customer.tier,
+          region: response.customer.region,
+          product: response.product_name
+        }
+      }
+    end
+
+    result = SentimentInsights::Insights::Sentiment.new.analyze(entries)
+    
+    filename = result.to_excel("feedback_analysis_#{Date.current}.xlsx")
+    
+    send_file filename, 
+      filename: "customer_feedback_analysis.xlsx",
+      type: "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
+  end
+end
+```
+
+### Background Job
+```ruby
+class AnalysisExportJob < ApplicationJob
+  def perform(survey_id, email)
+    survey = Survey.find(survey_id)
+    entries = survey.responses.map { |r| { answer: r.text, segment: r.metadata } }
+    
+    result = SentimentInsights::Insights::Sentiment.new.analyze(entries)
+    files = result.export_all("survey_#{survey_id}_analysis")
+    
+    AnalysisMailer.send_results(email, files).deliver_now
+  end
+end
+```
+
+## 🚀 Performance Tips
+
+1. **Use filtering** to reduce export size for large datasets
+2. **Skip unnecessary sections** with `responses_only` for faster exports
+3. **Batch processing** - split large datasets before analysis
+4. **Background jobs** for large exports in web applications
+
+The export system is designed to handle dynamic segment structures, large datasets, and provides maximum flexibility for different use cases while maintaining excellent performance.

data/Gemfile.lock

@@ -1,8 +1,9 @@
 PATH
   remote: .
   specs:
-    sentiment_insights (0.3.0)
+    sentiment_insights (0.4.0)
       aws-sdk-comprehend (>= 1.98.0)
+      rubyXL (~> 3.4)
       sentimental (~> 1.4.0)
 
 GEM
@@ -27,6 +28,9 @@ GEM
     dotenv (2.8.1)
     jmespath (1.6.2)
     logger (1.7.0)
+    nokogiri (1.15.7-arm64-darwin)
+      racc (~> 1.4)
+    racc (1.8.1)
     rake (13.2.1)
     rspec (3.13.0)
       rspec-core (~> 3.13.0)
@@ -41,6 +45,10 @@ GEM
       diff-lcs (>= 1.2.0, < 2.0)
       rspec-support (~> 3.13.0)
     rspec-support (3.13.2)
+    rubyXL (3.4.33)
+      nokogiri (>= 1.10.8)
+      rubyzip (>= 1.3.0)
+    rubyzip (2.4.1)
     sentimental (1.4.1)
 
 PLATFORMS

data/README.md

@@ -12,6 +12,7 @@
     - [Sentiment Analysis](#sentiment-analysis)
     - [Key Phrase Extraction](#key-phrase-extraction)
     - [Entity Extraction](#entity-extraction)
+- [Export & Output Formats](#export--output-formats)
 - [Provider Options & Custom Prompts](#provider-options--custom-prompts)
 - [Full Example](#full-example)
 - [Contributing](#contributing)
@@ -313,6 +314,87 @@ result = insight.extract(
          "the response was copy-paste and didn't address my issue directly.",
      :segment=>{:age=>"45-54", :region=>"Midwest"}}]}
 ```
+
+---
+
+## Export & Output Formats
+
+SentimentInsights provides flexible export options for all analysis results. Export to CSV, Excel, or JSON formats with advanced filtering and customization.
+
+### Quick Export Examples
+
+```ruby
+result = SentimentInsights::Insights::Sentiment.new.analyze(entries)
+
+# Direct format methods
+result.to_csv("analysis.csv")           # CSV file
+result.to_json("analysis.json")         # JSON file  
+result.to_excel("analysis.xlsx")        # Excel file
+
+# Auto-detect format from filename
+result.export("analysis.csv")           # Creates CSV
+result.export("analysis.json")          # Creates JSON
+result.export("analysis.xlsx")          # Creates Excel
+
+# API-friendly formats (no files)
+api_data = result.to_hash               # Structured Hash
+json_string = result.to_json_string     # JSON string
+```
+
+### Advanced Export Options
+
+```ruby
+# Filtered exports
+result.export_positive(:csv, "positive_feedback.csv")
+result.export_negative(:excel, "issues_to_address.xlsx")
+
+# Fluent interface with filtering
+result
+  .filter_sentiment(:positive)
+  .filter_segments(age_group: ["18-25", "26-35"])
+  .to_excel("young_positive_feedback.xlsx")
+
+# Export all formats at once
+files = result.export_all("comprehensive_analysis")
+# Returns: { csv: "file.csv", json: "file.json", excel: "file.xlsx" }
+```
+
+### CSV Export Structure
+
+```csv
+response_id,text,sentiment_label,sentiment_score,segment_age_group,segment_region,timestamp
+r_1,"I love this product!",positive,0.9,18-25,North,2024-06-28T10:30:00Z
+
+SUMMARY STATISTICS
+Total Responses,150
+Positive Percentage,60.0%
+Net Sentiment Score,40.0
+
+SEGMENT ANALYSIS
+Segment Type,Segment Value,Total Count,Positive %,Net Score
+Age Group,18-25,75,65.3%,49.3
+```
+
+### JSON Export Structure
+
+```json
+{
+  "metadata": {
+    "export_timestamp": "2024-06-28T10:30:00Z",
+    "analysis_type": "sentiment",
+    "total_responses": 150,
+    "provider_used": "claude"
+  },
+  "analysis": {
+    "responses": [...],
+    "global_summary": {...},
+    "segment_summary": {...}
+  }
+}
+```
+
+**📋 For complete export documentation, see [EXPORT_USAGE.md](EXPORT_USAGE.md)**
+
 ---
 
 ## Provider Options & Custom Prompts
@@ -365,8 +447,8 @@ bundle exec rspec
 - [x] Sentiment Analysis
 - [x] Key Phrase Extraction
 - [x] Entity Recognition
+- [x] Export Functionality (CSV, Excel, JSON)
 - [ ] Topic Modeling
-- [ ] CSV/JSON Export Helpers
 - [ ] Visual Dashboard Add-on
 
 ---