@botlearn/sentiment-analyzer 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 BotLearn
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,35 @@
+# @botlearn/sentiment-analyzer
+
+> Fine-grained sentiment recognition and opinion mining with aspect-based analysis, sarcasm detection, and polarity classification for OpenClaw Agent
+
+## Installation
+
+```bash
+# via npm
+npm install @botlearn/sentiment-analyzer
+
+# via clawhub
+clawhub install @botlearn/sentiment-analyzer
+```
+
+## Category
+
+content-processing
+
+## Dependencies
+
+None
+
+## Files
+
+| File | Description |
+|------|-------------|
+| `manifest.json` | Skill metadata and configuration |
+| `skill.md` | Role definition and activation rules |
+| `knowledge/` | Domain knowledge documents |
+| `strategies/` | Behavioral strategy definitions |
+| `tests/` | Smoke and benchmark tests |
+
+## License
+
+MIT

package/knowledge/anti-patterns.md

@@ -0,0 +1,86 @@
+---
+domain: sentiment-analyzer
+topic: anti-patterns
+priority: medium
+ttl: 30d
+---
+
+# Sentiment Analysis — Anti-Patterns
+
+## Polarity Classification Anti-Patterns
+
+### 1. Binary Sentiment Trap
+- **Problem**: Reducing all sentiment to just positive or negative, losing nuance and gradation
+- **Fix**: Always use a fine-grained scale (7-point: strongly negative through strongly positive). "Okay" is not positive. "Not terrible" is not the same as "good". Capture the full spectrum including slightly positive/negative and neutral.
+
+### 2. Ignoring Negation and Valence Shifters
+- **Problem**: Treating "not good" as positive because "good" is in the sentiment lexicon, or ignoring "barely", "hardly", "never" as modifiers
+- **Fix**: Always parse for negation markers (not, no, never, neither, un-, in-, -less) BEFORE assigning polarity. Apply negation scope rules: negation inverts the next sentiment expression within the same clause. Process valence shifters in order: negation first, then intensifiers/diminishers.
+
+### 3. Ignoring Intensifiers and Diminishers
+- **Problem**: Treating "excellent" and "somewhat good" as the same strength of positive sentiment
+- **Fix**: Apply intensity multipliers. Intensifiers (very, extremely, incredibly) amplify by 1.25-1.5x. Diminishers (somewhat, slightly, a bit) reduce by 0.5-0.75x. Stack multipliers when multiple modifiers are present ("really very good" = double intensification).
+
+### 4. Context-Free Lexicon Lookup
+- **Problem**: Assigning fixed sentiment scores from a general lexicon without considering domain context. "Sick" is negative in health context but positive in slang ("that's sick!").
+- **Fix**: Detect the text domain first. Apply domain-specific sentiment adjustments. Maintain override lists for domain-dependent terms. When domain is uncertain, flag the term and provide both interpretations.
+
+### 5. Averaging Away Mixed Sentiment
+- **Problem**: A review that says "Amazing camera but horrible battery" gets averaged to "neutral", losing the actionable insight that two distinct aspects have opposite sentiments
+- **Fix**: Report aspect-level sentiments separately. Document-level should be labeled "mixed" rather than averaged to neutral. Always preserve the aspect-level detail that reveals what specifically is positive or negative.
+
+## Sarcasm and Irony Anti-Patterns
+
+### 6. Missing Sarcasm and Irony
+- **Problem**: Taking sarcastic statements at face value. "Great, another update that broke everything" is classified as positive because of "great"
+- **Fix**: Check for incongruity between sentiment words and context (complaints, low ratings, negative events). Look for hyperbole, excessive punctuation, and known sarcasm markers. When sarcasm is detected, invert the polarity. Flag uncertainty when sarcasm detection is ambiguous.
+
+### 7. Over-Detecting Sarcasm
+- **Problem**: Flagging every positive statement in a negative context as sarcastic, even when the author is genuinely acknowledging a positive aspect
+- **Fix**: Require multiple sarcasm signals before inverting. A single positive word in a negative review is not necessarily sarcastic ("The packaging was nice but the product was broken" — "nice" is genuine). Only invert when there is strong incongruity evidence (e.g., star rating contradicts text, extreme hyperbole).
+
+### 8. Ignoring Quoted or Reported Sarcasm
+- **Problem**: Treating sarcasm in reported speech the same as the author's own sarcasm. "My friend said 'oh how wonderful' when the flight was cancelled" — the author is reporting, not being sarcastic themselves.
+- **Fix**: Identify the opinion holder. Apply sarcasm detection only to the expressed opinion of the correct holder. The author's sentiment about the reported sarcasm may differ from the sarcasm itself.
+
+## Scope and Granularity Anti-Patterns
+
+### 9. Sentence-Level Assumption
+- **Problem**: Assuming each sentence has exactly one sentiment, when a single sentence can contain multiple aspects with different sentiments: "The food was delicious but overpriced"
+- **Fix**: Perform clause-level or aspect-level analysis within sentences. Split on conjunctions (but, however, although, yet, while) that signal sentiment shifts. Each clause may have an independent polarity.
+
+### 10. Ignoring Implicit Aspects
+- **Problem**: Only extracting explicit aspect nouns ("battery", "screen") and missing implicit aspects. "It's too heavy" implies the WEIGHT aspect without naming it.
+- **Fix**: Maintain an implicit aspect mapping: adjective/property to aspect category. "Heavy" maps to WEIGHT, "slow" maps to PERFORMANCE, "expensive" maps to PRICE. Train on domain-specific implicit aspect patterns.
+
+### 11. Entity Confusion
+- **Problem**: Misattributing sentiment to the wrong entity. "I returned the Samsung because the Apple was better" — sentiment toward Samsung is negative, Apple is positive, but a naive approach might reverse this.
+- **Fix**: Parse sentence structure to correctly bind sentiment expressions to their targets using dependency parsing or proximity rules. Track entity mentions and their co-occurring sentiment expressions separately.
+
+## Confidence and Output Anti-Patterns
+
+### 12. Overconfident Neutral
+- **Problem**: Assigning high confidence (>0.90) to "neutral" when the text is actually ambiguous, mixed, or contains sentiment that was not detected
+- **Fix**: Reserve high-confidence neutral for genuinely objective/factual text ("The meeting is at 3 PM"). For ambiguous text, assign lower confidence neutral (0.40-0.60) and flag as "potentially ambiguous". Run a second pass specifically looking for implicit sentiment.
+
+### 13. Ignoring Opinion Holder
+- **Problem**: Treating all sentiment in a text as the author's opinion, even when some is reported speech, quotes, or hypothetical
+- **Fix**: Distinguish between the author's own sentiment, reported sentiment from others, and hypothetical/conditional sentiment. Tag each opinion with its holder: author, quoted source, or hypothetical.
+
+### 14. No Uncertainty Communication
+- **Problem**: Always presenting a single definitive sentiment label without indicating when the analysis is uncertain or when multiple interpretations are plausible
+- **Fix**: Always include confidence scores. When confidence is below 0.60, present multiple possible interpretations ranked by likelihood. Explicitly flag when sarcasm detection is uncertain, when domain is ambiguous, or when signals conflict.
+
+### 15. Temporal Sentiment Blindness
+- **Problem**: Ignoring how sentiment evolves within a text. A review might start positive ("I was excited to receive this") and end negative ("but after a week it completely fell apart"). The final sentiment matters most.
+- **Fix**: Track sentiment trajectory across the document. Weight later sentences more heavily for document-level sentiment (recency bias aligns with the author's final judgment). Flag sentiment shifts and report them explicitly.
+
+## Data Quality Anti-Patterns
+
+### 16. Emoji and Emoticon Blindness
+- **Problem**: Ignoring emoji or emoticons that carry significant sentiment signal, especially in social media and messaging
+- **Fix**: Include emoji sentiment mappings in the lexicon. Common mappings: positive (thumbs up, heart, laughing, fire), negative (thumbs down, angry, crying, broken heart), sarcastic (eye-roll, upside-down smile). Emoji can override or reinforce textual sentiment.
+
+### 17. Treating All Text Sources Equally
+- **Problem**: Applying the same analysis parameters to a tweet, a product review, a news article, and a legal document
+- **Fix**: Detect text source/register and adjust analysis accordingly. Tweets: expect abbreviations, emoji, hashtags. Product reviews: expect star ratings as anchors. News: distinguish editorial from reported content. Legal: formal negation, not sentiment-bearing.

package/knowledge/best-practices.md

@@ -0,0 +1,148 @@
+---
+domain: sentiment-analyzer
+topic: multi-level-analysis-and-calibration
+priority: high
+ttl: 30d
+---
+
+# Sentiment Analysis — Best Practices
+
+## Multi-Level Analysis
+
+### 1. Document-Level vs. Sentence-Level vs. Aspect-Level
+Always analyze at the most granular level appropriate for the task, then aggregate upward:
+
+- **Aspect-level** — Most granular: identifies sentiment toward specific features or entities
+- **Sentence-level** — Each sentence gets an independent polarity label
+- **Document-level** — Overall sentiment, aggregated from lower levels
+
+**Aggregation rule**: Document sentiment is NOT simply the average of sentence sentiments. Weight by:
+1. **Position** — Concluding sentences carry more weight (recency effect)
+2. **Emphasis** — Sentences with intensifiers or exclamation carry more weight
+3. **Aspect importance** — Core aspects (e.g., food in a restaurant review) weigh more than peripheral ones (e.g., parking)
+4. **Explicit summary signals** — Phrases like "overall", "in summary", "all in all" anchor the document-level sentiment
+
+### 2. Sentence-Level Analysis Pipeline
+For each sentence:
+1. Identify opinion targets (explicit or implicit aspects)
+2. Locate sentiment expressions (adjectives, adverbs, verbs, phrases)
+3. Detect valence shifters (negation, intensifiers, diminishers)
+4. Compute modified polarity: `base_polarity * shifter_multiplier`
+5. Assign confidence based on signal strength
+
+### 3. Cross-Sentence Sentiment Flow
+Sentiment can span multiple sentences:
+- "The screen is gorgeous. Colors are vibrant and the resolution is sharp." — all three sentences express positive sentiment toward the DISPLAY aspect
+- Track aspect continuity across sentences using coreference: "The laptop... It... This device..."
+- Contrastive conjunctions signal sentiment shifts: "The food was great. However, the service was terrible."
+
+## Sarcasm and Irony Detection
+
+### Indicators of Sarcasm
+Sarcasm detection requires looking beyond lexical sentiment. Key signals:
+
+1. **Hyperbole** — Excessive praise or criticism that doesn't match context:
+   - "Oh sure, waiting 3 hours for a table is just DELIGHTFUL"
+2. **Incongruity** — Positive words in objectively negative situations:
+   - "Love how my flight got cancelled for the third time this month"
+3. **Punctuation patterns** — Ellipsis, excessive exclamation, quotes:
+   - "The 'premium' service was truly... something"
+4. **Contextual mismatch** — Sentiment contradicts known facts:
+   - Rating: 1/5 stars. Text: "Best experience ever!"
+5. **Universal quantifiers with negative context** — "Everything" + "perfect" when listing complaints:
+   - "Everything about this product is absolutely perfect (if you enjoy things that break immediately)"
+6. **Hashtags and tags** (social media): `#sarcasm`, `#not`, `/s`
+
+### Sarcasm Handling Strategy
+- IF sarcasm is detected THEN invert the surface sentiment polarity
+- Assign lower confidence (0.60-0.75) to sarcasm-inverted sentiments since sarcasm detection is inherently uncertain
+- Flag the output as sarcasm-detected so downstream consumers are aware
+- When uncertain whether text is sarcastic, report BOTH literal and sarcastic interpretations with respective confidence scores
+
+### Irony vs. Sarcasm
+- **Sarcasm** — Intentionally saying the opposite of what is meant, usually to mock or criticize
+- **Irony** — A broader concept where reality contradicts expectations (not always mocking)
+- **Situational irony** in text: "The fire station burned down" — neutral/objective sentiment despite ironic content
+- Only invert sentiment for **verbal irony/sarcasm**, not situational irony
+
+## Domain Calibration
+
+### Why Domain Matters
+The same word carries different sentiment weight depending on context:
+
+| Word | Product Reviews | Medical Text | Financial Text | Sports |
+|------|----------------|-------------|----------------|--------|
+| aggressive | -0.4 | -0.2 | +0.3 | +0.4 |
+| stable | +0.3 | +0.6 | +0.5 | +0.1 |
+| volatile | -0.5 | -0.3 | -0.6 | +0.2 |
+| critical | -0.5 | -0.7 | -0.4 | +0.1 |
+| positive | +0.5 | +0.8 | +0.6 | +0.3 |
+| sharp | +0.3 | -0.3 | -0.4 | +0.4 |
+
+### Domain Detection Heuristics
+Before starting analysis, identify the text domain:
+1. Check for domain-specific vocabulary (medical terms, financial jargon, product categories)
+2. Identify the text source if available (Amazon review, tweet, news article, clinical note)
+3. Look for structural cues (star ratings, review headings, formal structure)
+4. Apply the appropriate domain-specific sentiment adjustments from the lexicon
+
+### Calibration Rules
+- **Product reviews** — Star ratings (if available) serve as ground truth anchors; calibrate text analysis to align with explicit ratings
+- **Social media** — Short, informal, heavy slang/emoji usage; expand lexicon to include platform-specific terms
+- **News articles** — More objective tone; distinguish between reported sentiment (what sources said) and editorial sentiment
+- **Academic/technical text** — Hedged language is standard, not an indicator of negativity; "may", "could", "suggests" are neutral
+- **Legal/regulatory text** — Formal negation patterns; "shall not" is a directive, not sentiment
+
+## Confidence Scoring
+
+### Calibrated Confidence Guidelines
+
+| Signal Strength | Confidence Range | Example |
+|----------------|-----------------|---------|
+| Strong explicit + consistent | 0.90 - 1.00 | "Absolutely love it, best purchase ever!" |
+| Clear sentiment, single aspect | 0.80 - 0.89 | "The battery life is disappointing" |
+| Sentiment with hedging | 0.65 - 0.79 | "It seems fairly good, though I'm not sure yet" |
+| Mixed/conflicting signals | 0.50 - 0.64 | "Great features but terrible reliability" |
+| Sarcasm-inverted | 0.55 - 0.75 | "Oh wonderful, another crash" (detected as sarcasm) |
+| Ambiguous/insufficient data | 0.30 - 0.49 | "It is what it is" |
+
+### Confidence Penalties
+Apply confidence reductions for:
+- **Short text** (< 10 words): -0.10 (less context for disambiguation)
+- **No explicit sentiment words**: -0.15 (relying on implicit sentiment)
+- **Domain mismatch** (uncertain domain): -0.10
+- **Sarcasm uncertainty**: -0.15 to -0.25
+- **Multiple conflicting aspects**: -0.05 per conflicting pair (for document-level confidence only)
+
+## Output Formatting
+
+### Structured Sentiment Report
+Always output in a structured format:
+
+```
+Document-Level Summary:
+  Overall Polarity: [label] ([score])
+  Confidence: [0.00 - 1.00]
+  Dominant Emotion: [if applicable]
+
+Aspect-Level Detail:
+  [Aspect 1]:
+    Polarity: [label] ([score])
+    Opinion Expression: "[quoted text]"
+    Valence Shifters: [none | negation | intensifier | diminisher]
+    Confidence: [0.00 - 1.00]
+  [Aspect 2]:
+    ...
+
+Flags:
+  Sarcasm Detected: [yes/no]
+  Mixed Sentiment: [yes/no]
+  Domain: [detected domain]
+  Low Confidence Aspects: [list]
+```
+
+### Handling Edge Cases
+- **Purely factual text**: Label as "neutral/objective" with a note that no opinion was expressed
+- **Mixed sentiment**: Report both the positive and negative aspects separately; document-level is "mixed" not an average
+- **Non-English text**: Note the language limitation; apply analysis only if the language is within capability
+- **Very short text** (< 5 words): Analyze but flag low confidence; avoid over-interpreting

package/knowledge/domain.md

@@ -0,0 +1,152 @@
+---
+domain: sentiment-analyzer
+topic: sentiment-polarity-and-opinion-mining
+priority: high
+ttl: 30d
+---
+
+# Sentiment Analysis — Polarity, ABSA, Opinion Mining & Valence Shifters
+
+## Sentiment Polarity Scale
+
+### Fine-Grained Polarity Levels
+Sentiment is classified on a 7-point scale rather than a binary positive/negative split:
+
+| Level | Label | Score Range | Example |
+|-------|-------|-------------|---------|
+| 7 | Strongly Positive | +0.75 to +1.00 | "Absolutely phenomenal — the best I've ever used" |
+| 6 | Positive | +0.50 to +0.74 | "I really like this product, it works well" |
+| 5 | Slightly Positive | +0.25 to +0.49 | "It's decent, does what it's supposed to" |
+| 4 | Neutral | -0.24 to +0.24 | "The device weighs 200 grams and ships in a blue box" |
+| 3 | Slightly Negative | -0.25 to -0.49 | "It's okay but nothing special" |
+| 2 | Negative | -0.50 to -0.74 | "Disappointed — it broke after a week" |
+| 1 | Strongly Negative | -0.75 to -1.00 | "Completely unusable, waste of money, worst purchase ever" |
+
+### Polarity Signals
+- **Lexical cues**: Sentiment-bearing words (e.g., "excellent", "terrible", "mediocre")
+- **Syntactic patterns**: Comparative structures ("better than", "worse than"), superlatives ("the best", "the worst")
+- **Pragmatic cues**: Exclamation marks, ALL CAPS, emoji, rhetorical questions
+- **Contextual signals**: Domain-specific terms that carry sentiment only in context (e.g., "unpredictable" is negative for software, neutral for weather description)
+
+## Aspect-Based Sentiment Analysis (ABSA)
+
+### Core Components
+ABSA decomposes a text into structured opinion tuples:
+
+```
+(aspect_term, aspect_category, sentiment_polarity, opinion_expression, confidence)
+```
+
+**Example**:
+- Input: "The camera quality is stunning but the battery life is disappointing"
+- Tuple 1: ("camera quality", QUALITY, positive, "stunning", 0.92)
+- Tuple 2: ("battery life", PERFORMANCE, negative, "disappointing", 0.89)
+
+### Aspect Extraction Methods
+1. **Explicit aspects** — Directly mentioned noun phrases: "screen", "battery", "customer service"
+2. **Implicit aspects** — Inferred from context: "It's too heavy" implies WEIGHT aspect without naming it
+3. **Composite aspects** — Multi-word aspect terms: "noise cancellation", "build quality", "user interface"
+
+### Common Aspect Categories
+
+| Domain | Typical Aspect Categories |
+|--------|--------------------------|
+| Product Reviews | Quality, Price, Design, Performance, Durability, Usability, Customer Service |
+| Restaurant Reviews | Food, Service, Ambiance, Price, Cleanliness, Location, Wait Time |
+| Software Reviews | Functionality, Performance, UI/UX, Reliability, Documentation, Support, Pricing |
+| Hotel Reviews | Room, Cleanliness, Location, Staff, Amenities, Value, Food |
+
+### Aspect-Sentiment Pairing Rules
+- Each aspect can have exactly one polarity (if a text says contradictory things about the same aspect, split into sub-aspects)
+- Aspects without a clear sentiment expression are tagged as neutral with low confidence
+- An opinion expression can modify multiple aspects: "Both the food and service were excellent"
+
+## Opinion Mining Components
+
+### Opinion Holder Identification
+The entity expressing the opinion:
+- **First-person**: "I think this is great" — opinion holder is the author
+- **Reported speech**: "My colleague says the software is buggy" — opinion holder is the colleague
+- **Impersonal**: "This product is considered reliable" — opinion holder is general consensus
+
+### Opinion Expression Types
+1. **Direct opinions** — Explicitly stated: "The battery life is excellent"
+2. **Comparative opinions** — Relative judgments: "Camera A is better than Camera B"
+3. **Suggestive opinions** — Implied through suggestions: "They should improve the packaging"
+4. **Conditional opinions** — Dependent on context: "Great if you don't mind the weight"
+
+### Opinion Strength Indicators
+- **Intensifiers** (+): very, extremely, incredibly, absolutely, utterly, remarkably
+- **Diminishers** (-): somewhat, slightly, a bit, fairly, kind of, sort of, rather
+- **Modal hedges** (~): might be, could be, seems, appears to, arguably
+- **Superlatives** (++): best, worst, most, least, greatest, finest
+
+## Valence Shifters
+
+Valence shifters are linguistic constructs that alter the base sentiment of a word or phrase.
+
+### Negation
+Negation inverts or significantly reduces the polarity of a sentiment expression:
+
+| Negation Type | Examples | Effect |
+|---------------|----------|--------|
+| Simple negation | not, no, never, neither, nor | Inverts polarity |
+| Morphological negation | un-, in-, im-, dis-, -less | Inverts polarity |
+| Implicit negation | fail to, lack, absence of, devoid of | Inverts polarity |
+| Double negation | "not uncommon", "not without merit" | Weakened positive (litotes) |
+| Negation scope | "not only good but great" | Scope ends at "but" — positive |
+
+### Negation Scope Rules
+- Negation typically scopes over the **next sentiment-bearing word or phrase**
+- Conjunctions (`but`, `however`, `although`) reset negation scope
+- Clause boundaries terminate negation scope
+- Example: "I don't think the camera is bad" — negation applies to "bad", result is weakly positive
+
+### Intensifiers and Diminishers
+
+**Intensifiers** amplify the existing polarity:
+```
+"very good"     → more positive than "good"
+"very bad"      → more negative than "bad"
+"absolutely terrible" → more negative than "terrible"
+```
+
+**Diminishers** reduce the magnitude of the existing polarity:
+```
+"somewhat good"  → less positive than "good"
+"slightly bad"   → less negative than "bad"
+"fairly decent"  → weakly positive
+```
+
+### Irrealis Markers
+Words or constructions that indicate the sentiment is hypothetical, wished-for, or counterfactual rather than asserted:
+
+| Marker Type | Examples | Effect |
+|-------------|----------|--------|
+| Conditional | "if it were better", "would be nice if" | Sentiment is hypothetical, not current |
+| Subjunctive | "I wish it were faster" | Implies negative current state |
+| Questions | "Is this any good?" | Uncertain — do not classify as asserted sentiment |
+| Future/Hope | "hopefully it improves" | Implies negative current state, positive desired state |
+
+## Sentiment Lexicons
+
+### General-Purpose Lexicons
+- **VADER** — Valence Aware Dictionary for social media, includes emoji and slang, scores -4 to +4
+- **SentiWordNet** — Synset-level scores for WordNet entries (positivity, negativity, objectivity)
+- **AFINN** — 2,477 words scored -5 to +5, good for short informal text
+- **NRC Emotion Lexicon** — 14,182 words annotated for 8 emotions + positive/negative
+
+### Domain-Specific Adjustments
+Words change sentiment valence across domains:
+- "unpredictable" — negative (software), neutral/positive (thriller novel), positive (sports)
+- "aggressive" — negative (customer service), positive (investment strategy), neutral (medical treatment)
+- "volatile" — negative (software), neutral (chemistry), negative (financial markets)
+- "intense" — positive (workout), negative (headache), positive (flavor), neutral (light)
+
+### Slang and Informal Expressions
+Social media and informal text require expanded lexicons:
+- "fire" / "lit" — strongly positive (slang)
+- "salty" — negative (slang, meaning bitter or upset)
+- "slaps" — strongly positive (slang, meaning excellent)
+- "mid" — negative (slang, meaning mediocre)
+- "/s" or "not" (sarcasm markers in online text)

package/manifest.json

@@ -0,0 +1,26 @@
+{
+  "name": "@botlearn/sentiment-analyzer",
+  "version": "0.1.0",
+  "description": "Fine-grained sentiment recognition and opinion mining with aspect-based analysis, sarcasm detection, and polarity classification for OpenClaw Agent",
+  "category": "content-processing",
+  "author": "BotLearn",
+  "benchmarkDimension": "content-understanding",
+  "expectedImprovement": 30,
+  "dependencies": {},
+  "compatibility": {
+    "openclaw": ">=0.5.0"
+  },
+  "files": {
+    "skill": "skill.md",
+    "knowledge": [
+      "knowledge/domain.md",
+      "knowledge/best-practices.md",
+      "knowledge/anti-patterns.md"
+    ],
+    "strategies": [
+      "strategies/main.md"
+    ],
+    "smokeTest": "tests/smoke.json",
+    "benchmark": "tests/benchmark.json"
+  }
+}

package/package.json

@@ -0,0 +1,35 @@
+{
+  "name": "@botlearn/sentiment-analyzer",
+  "version": "0.1.0",
+  "description": "Fine-grained sentiment recognition and opinion mining with aspect-based analysis, sarcasm detection, and polarity classification for OpenClaw Agent",
+  "type": "module",
+  "main": "manifest.json",
+  "files": [
+    "manifest.json",
+    "skill.md",
+    "knowledge/",
+    "strategies/",
+    "tests/",
+    "README.md"
+  ],
+  "keywords": [
+    "botlearn",
+    "openclaw",
+    "skill",
+    "content-processing"
+  ],
+  "author": "BotLearn",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/readai-team/botlearn-awesome-skills.git",
+    "directory": "packages/skills/sentiment-analyzer"
+  },
+  "homepage": "https://github.com/readai-team/botlearn-awesome-skills/tree/main/packages/skills/sentiment-analyzer",
+  "bugs": {
+    "url": "https://github.com/readai-team/botlearn-awesome-skills/issues"
+  },
+  "publishConfig": {
+    "access": "public"
+  }
+}

package/skill.md

@@ -0,0 +1,47 @@
+---
+name: sentiment-analyzer
+role: Sentiment Analysis Specialist
+version: 1.0.0
+triggers:
+  - "sentiment"
+  - "opinion mining"
+  - "analyze tone"
+  - "polarity"
+  - "sentiment analysis"
+  - "emotional tone"
+  - "opinion detection"
+  - "feeling analysis"
+---
+
+# Role
+
+You are a Sentiment Analysis Specialist. When activated, you perform fine-grained sentiment recognition and opinion mining on text, identifying polarity at document, sentence, and aspect levels. You detect nuanced sentiment cues including sarcasm, irony, hedging, and intensification, and produce structured sentiment assessments with confidence scores achieving >85% accuracy.
+
+# Capabilities
+
+1. Classify sentiment polarity at multiple granularities: document-level, sentence-level, and aspect-level (ABSA)
+2. Identify and extract opinion targets (aspects) and their associated sentiment expressions using opinion mining techniques
+3. Detect valence shifters including negation, intensifiers, diminishers, and irrealis markers that modify base sentiment
+4. Recognize sarcasm, irony, and implicit sentiment that contradicts surface-level lexical cues
+5. Produce calibrated confidence scores for each sentiment judgment, reflecting genuine uncertainty when signals are mixed
+6. Aggregate aspect-level sentiments into a coherent document-level summary with weighted rollup
+
+# Constraints
+
+1. Never assign sentiment without identifying the specific opinion target — every sentiment must be anchored to an aspect or entity
+2. Never treat sentiment as purely binary (positive/negative) — always use a fine-grained scale (e.g., strongly negative, negative, slightly negative, neutral, slightly positive, positive, strongly positive)
+3. Never ignore negation or valence shifters — "not good" is not positive, "not bad" is not negative
+4. Never assume literal interpretation when sarcasm or irony markers are present (hyperbole, contradiction, context mismatch)
+5. Never present high-confidence scores when the text contains genuinely ambiguous or conflicting sentiment signals
+6. Always calibrate sentiment interpretation to the domain context — product reviews, social media, and formal reports use different sentiment conventions
+
+# Activation
+
+WHEN the user requests sentiment analysis, opinion mining, or tone assessment:
+1. Segment the input text into analyzable units following strategies/main.md
+2. Identify opinion targets (aspects) and sentiment expressions using knowledge/domain.md
+3. Detect valence shifters, sarcasm markers, and contextual modifiers
+4. Classify polarity on a fine-grained scale with calibrated confidence
+5. Verify against knowledge/anti-patterns.md to avoid common sentiment analysis errors
+6. Apply knowledge/best-practices.md for multi-level aggregation and domain calibration
+7. Output structured sentiment assessment with aspect-level detail and document-level summary

package/strategies/main.md

@@ -0,0 +1,110 @@
+---
+strategy: sentiment-analyzer
+version: 1.0.0
+steps: 6
+---
+
+# Sentiment Analysis Strategy
+
+## Step 1: Segmentation
+- Receive the input text and determine its scope: single sentence, paragraph, multi-paragraph document, or structured data (e.g., review with star rating)
+- Detect the text **domain** using vocabulary and structural cues:
+  - Product review → look for product terms, star ratings, pros/cons structure
+  - Social media → short text, emoji, hashtags, informal language, slang
+  - News/editorial → formal structure, quotations, byline
+  - Technical/academic → hedged language, citations, formal tone
+- Segment the text into analyzable units:
+  - Split into sentences using punctuation and structural boundaries
+  - Within sentences, split on contrastive conjunctions (`but`, `however`, `although`, `yet`, `while`, `on the other hand`) into clauses
+  - IF the text contains bullet points, numbered lists, or structured sections THEN treat each item as a separate unit
+- Record metadata: total sentence count, detected domain, text register (formal/informal), presence of ratings or structured signals
+
+## Step 2: Aspect Identification
+- For each sentence/clause, extract **opinion targets** (aspects):
+  - **Explicit aspects**: Noun phrases that are the subject of opinion expressions ("the battery life", "customer support", "build quality")
+  - **Implicit aspects**: Infer from adjectives or properties without a named target ("too heavy" → WEIGHT aspect, "overpriced" → PRICE aspect, "slow" → PERFORMANCE aspect)
+  - **Composite aspects**: Multi-word targets that should not be split ("noise cancellation", "user interface", "customer service")
+- Categorize each aspect into a domain-appropriate category using the taxonomy in knowledge/domain.md
+- IF no aspects are identified THEN treat the entire sentence as a single-aspect unit targeting the overall subject
+- IF the text mentions multiple entities (e.g., comparing products) THEN tag each aspect with its entity to prevent misattribution
+- Build an **aspect inventory**: list all unique aspects found across the document with their source sentences
+
+## Step 3: Sentiment Cue Detection
+- For each clause/aspect pair, identify all sentiment-bearing expressions:
+  - **Sentiment words**: Adjectives ("excellent", "terrible"), adverbs ("poorly", "beautifully"), verbs ("love", "hate", "enjoy", "struggle"), nouns ("disaster", "delight", "nightmare")
+  - **Phrases and idioms**: "fell short of expectations", "blew me away", "left a lot to be desired", "hit it out of the park"
+  - **Comparative expressions**: "better than", "worse than", "not as good as", "far superior to"
+  - **Emoji and emoticons**: Map to sentiment scores using knowledge/domain.md lexicon entries
+- Detect all **valence shifters** in scope:
+  - **Negation markers**: not, no, never, neither, hardly, barely, nobody, nothing, un-, in-, im-, dis-, -less
+  - **Intensifiers**: very, extremely, incredibly, absolutely, utterly, remarkably, thoroughly, deeply
+  - **Diminishers**: somewhat, slightly, a bit, fairly, kind of, sort of, rather, marginally
+  - **Irrealis markers**: if, would, could, might, wish, hope, should (indicates hypothetical, not asserted sentiment)
+- Determine **negation scope**: negation applies to the next sentiment expression within the same clause; conjunction boundaries and clause boundaries terminate scope
+- IF a sentiment word appears but has no clear aspect target THEN check previous sentences for aspect continuity via coreference ("The laptop... It... This device...")
+
+## Step 4: Polarity Classification
+- For each aspect-sentiment pair, compute the polarity score:
+  1. Start with the **base polarity** of the sentiment expression (from lexicon or contextual analysis)
+  2. Apply **domain calibration** — adjust the base score if the word has domain-specific sentiment (see knowledge/domain.md domain-specific adjustments)
+  3. Apply **valence shifters** in order:
+     - Negation: multiply by -0.8 to -1.0 (not full inversion for nuanced negation like "not bad")
+     - Intensifiers: multiply magnitude by 1.25-1.5x
+     - Diminishers: multiply magnitude by 0.5-0.75x
+     - Double negation: result is weakly positive ("not uncommon" ≈ +0.2 to +0.3)
+  4. Apply **irrealis discount**: IF the sentiment is hypothetical/conditional THEN reduce confidence by 0.20 and tag as "non-asserted"
+- Map the final score to the 7-point polarity label using the scale in knowledge/domain.md
+- IF sarcasm indicators are present (see Step 3 and knowledge/best-practices.md):
+  - Check for incongruity: positive words + negative context, or hyperbole + complaint pattern
+  - IF sarcasm is confirmed THEN invert polarity and apply confidence penalty (see Step 5)
+  - IF sarcasm is uncertain THEN report BOTH literal and inverted interpretations
+
+## Step 5: Confidence Assessment
+- Assign a confidence score (0.00 - 1.00) to each aspect-level sentiment based on signal strength:
+  - **Start at 0.85** as baseline for clear, unambiguous sentiment
+  - **Adjust upward** (+0.05 to +0.15):
+    - Multiple reinforcing sentiment cues for the same aspect
+    - Star rating aligns with textual sentiment
+    - Strong intensifiers with clear targets
+  - **Adjust downward** (-0.05 to -0.25):
+    - Short text (< 10 words): -0.10
+    - No explicit sentiment words (relying on implicit): -0.15
+    - Sarcasm detected but uncertain: -0.15 to -0.25
+    - Domain uncertain: -0.10
+    - Hedging language present: -0.10
+    - Conflicting signals within the same aspect: -0.15
+- Cap confidence at 1.00 and floor at 0.30
+- SELF-CHECK against knowledge/anti-patterns.md:
+  - Is any neutral label assigned with confidence > 0.85? → Verify the text is genuinely factual
+  - Is any sentiment assigned without an identified aspect? → Reassign to an aspect or flag
+  - Has negation been properly accounted for in every case?
+  - Has the domain calibration been applied?
+
+## Step 6: Aggregation & Output
+- **Aspect-level output**: For each aspect, output the structured tuple:
+  ```
+  Aspect: [aspect_term] ([category])
+  Polarity: [label] ([score])
+  Opinion Expression: "[quoted text from source]"
+  Valence Shifters: [list or "none"]
+  Sarcasm: [detected/not detected/uncertain]
+  Confidence: [0.00-1.00]
+  ```
+- **Sentence-level aggregation**: For sentences with multiple aspects:
+  - IF all aspects agree in polarity THEN sentence polarity = the shared direction with averaged magnitude
+  - IF aspects disagree THEN sentence polarity = "mixed" with the dominant direction noted
+- **Document-level aggregation**: Compute overall sentiment using weighted rollup:
+  1. Weight aspects by **importance** (core domain aspects weigh 1.5x; peripheral aspects weigh 0.75x)
+  2. Weight by **position** (concluding sentences weigh 1.25x)
+  3. Weight by **emphasis** (intensified sentiments weigh 1.15x)
+  4. IF explicit summary phrases exist ("overall", "in summary", "all in all") THEN anchor document sentiment to those phrases
+  5. IF sentiment is mixed across aspects THEN label document as "mixed" with a breakdown, NOT an averaged neutral
+- **Final output**: Produce the complete structured sentiment report following the format in knowledge/best-practices.md:
+  - Document-level summary with polarity, confidence, and dominant emotion
+  - Aspect-level detail table
+  - Flags: sarcasm, mixed sentiment, domain, low-confidence aspects
+- SELF-CHECK the complete output:
+  - Does the document-level sentiment coherently represent the aspect-level findings?
+  - Are all flagged anti-patterns from knowledge/anti-patterns.md avoided?
+  - Is every sentiment anchored to a specific aspect or entity?
+  - IF any check fails THEN loop back to the relevant step and reprocess

package/tests/benchmark.json

@@ -0,0 +1,476 @@
+{
+  "version": "0.0.1",
+  "dimension": "content-understanding",
+  "tasks": [
+    {
+      "id": "bench-easy-01",
+      "difficulty": "easy",
+      "description": "Simple positive sentiment with clear opinion target",
+      "input": "Analyze the sentiment of this text:\n\n\"I absolutely love this coffee maker. It brews a perfect cup every morning and the built-in grinder is incredibly convenient. Best kitchen purchase I've made in years.\"",
+      "rubric": [
+        {
+          "criterion": "Polarity Accuracy",
+          "weight": 0.4,
+          "scoring": {
+            "5": "Correctly identifies strongly positive sentiment; detects intensifiers ('absolutely', 'incredibly', 'best') and maps to the high end of the positive scale",
+            "3": "Identifies positive sentiment but does not capture the intensity or differentiate from mildly positive",
+            "1": "Identifies sentiment direction but labels it as neutral or mixed",
+            "0": "Incorrect polarity"
+          }
+        },
+        {
+          "criterion": "Aspect Extraction",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Extracts at least 3 aspects: overall product, brewing quality, built-in grinder/convenience; all tagged as positive",
+            "3": "Extracts 1-2 aspects or correctly identifies overall sentiment without aspect detail",
+            "1": "No aspect extraction, only document-level label",
+            "0": "No usable analysis"
+          }
+        },
+        {
+          "criterion": "Output Quality",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Structured output with polarity label, score, confidence (high, >0.90), and aspect breakdown",
+            "3": "Provides polarity and some structure but missing confidence or aspect detail",
+            "1": "Single-word or single-sentence output",
+            "0": "No usable output"
+          }
+        }
+      ],
+      "expectedScoreWithout": 45,
+      "expectedScoreWith": 85
+    },
+    {
+      "id": "bench-easy-02",
+      "difficulty": "easy",
+      "description": "Simple negative sentiment with explicit complaints",
+      "input": "Analyze the sentiment of this text:\n\n\"Terrible experience at this restaurant. The food was cold, the waiter was rude, and we waited over an hour for our order. Never coming back.\"",
+      "rubric": [
+        {
+          "criterion": "Polarity Accuracy",
+          "weight": 0.4,
+          "scoring": {
+            "5": "Correctly identifies strongly negative sentiment across all aspects; detects 'terrible', 'cold', 'rude', 'never' as negative signals",
+            "3": "Identifies negative sentiment but misses intensity or treats some aspects as neutral",
+            "1": "Gets direction right but classifies as mildly negative",
+            "0": "Incorrect polarity"
+          }
+        },
+        {
+          "criterion": "Aspect Extraction",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Extracts 3+ aspects: food (cold/negative), service/waiter (rude/negative), wait time (negative); all correctly negative",
+            "3": "Extracts 1-2 aspects, misses some",
+            "1": "Only document-level analysis",
+            "0": "No analysis"
+          }
+        },
+        {
+          "criterion": "Output Quality",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Structured output with aspect-level detail, high confidence scores, domain detected as restaurant review",
+            "3": "Basic structure with polarity labels but missing domain detection or confidence",
+            "1": "Minimal output",
+            "0": "No usable output"
+          }
+        }
+      ],
+      "expectedScoreWithout": 45,
+      "expectedScoreWith": 85
+    },
+    {
+      "id": "bench-easy-03",
+      "difficulty": "easy",
+      "description": "Neutral/factual text with no opinion expressed",
+      "input": "Analyze the sentiment of this text:\n\n\"The company was founded in 2015 and is headquartered in Austin, Texas. It currently employs approximately 500 people across three offices. The annual revenue report will be published next quarter.\"",
+      "rubric": [
+        {
+          "criterion": "Polarity Accuracy",
+          "weight": 0.4,
+          "scoring": {
+            "5": "Correctly identifies the text as neutral/objective with no opinion expressed; does not force a positive or negative label",
+            "3": "Identifies as mostly neutral but incorrectly finds weak sentiment in factual statements",
+            "1": "Misclassifies as positive or negative",
+            "0": "Completely wrong classification"
+          }
+        },
+        {
+          "criterion": "Objectivity Detection",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Explicitly notes the text is factual/informational with no opinion holder or sentiment expressions; provides appropriate low-confidence note",
+            "3": "Labels as neutral but does not explain why or note the absence of opinion",
+            "1": "Does not distinguish between neutral-opinion and factual-objective",
+            "0": "No objectivity assessment"
+          }
+        },
+        {
+          "criterion": "Output Quality",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Structured output noting neutral classification, no aspects with sentiment, and a note that the text is purely factual",
+            "3": "Basic neutral label with some explanation",
+            "1": "Minimal output",
+            "0": "No usable output"
+          }
+        }
+      ],
+      "expectedScoreWithout": 40,
+      "expectedScoreWith": 80
+    },
+    {
+      "id": "bench-med-01",
+      "difficulty": "medium",
+      "description": "Mixed sentiment review with contrasting aspects requiring aspect-level analysis",
+      "input": "Analyze the sentiment of this text:\n\n\"The laptop has an incredible display — the OLED panel is vibrant and the 120Hz refresh rate makes scrolling buttery smooth. Build quality is also top-notch with a solid aluminum chassis. However, the keyboard is mushy and imprecise, the trackpad is too small for comfortable use, and the fan noise under any load is distractingly loud. For the price they're asking, the input devices should be much better. The webcam is passable but nothing special.\"",
+      "rubric": [
+        {
+          "criterion": "Aspect-Level Accuracy",
+          "weight": 0.35,
+          "scoring": {
+            "5": "Correctly identifies 6+ aspects (display, refresh rate, build quality, keyboard, trackpad, fan noise, price/value, webcam) with correct individual polarities: display/build positive, keyboard/trackpad/fan negative, webcam slightly positive/neutral, price negative",
+            "3": "Identifies 3-4 aspects with mostly correct polarities but misses some",
+            "1": "Only 1-2 aspects identified or several misclassified",
+            "0": "No aspect-level analysis"
+          }
+        },
+        {
+          "criterion": "Mixed Sentiment Handling",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Document-level sentiment labeled as 'mixed' with clear breakdown of positive aspects (display, build) vs. negative aspects (input, noise, value); does not average to neutral",
+            "3": "Labels as mixed but aggregation is simplistic (e.g., averaged to slightly negative)",
+            "1": "Forces a single polarity label (positive or negative) ignoring the contrast",
+            "0": "Completely mischaracterizes the overall sentiment"
+          }
+        },
+        {
+          "criterion": "Valence Shifter and Comparative Handling",
+          "weight": 0.2,
+          "scoring": {
+            "5": "Detects 'however' as sentiment pivot, 'too small' as negative intensification, 'nothing special' as diminished neutral, 'should be much better' as negative expectation gap",
+            "3": "Handles 'however' pivot but misses nuanced shifters",
+            "1": "Misses most valence shifters and comparatives",
+            "0": "No shifter handling"
+          }
+        },
+        {
+          "criterion": "Output Structure",
+          "weight": 0.15,
+          "scoring": {
+            "5": "Full structured report with aspect table, confidence per aspect, document summary, domain detected as product/laptop review",
+            "3": "Aspect list with polarities but missing confidence or structure",
+            "1": "Unstructured text output",
+            "0": "No usable output"
+          }
+        }
+      ],
+      "expectedScoreWithout": 30,
+      "expectedScoreWith": 75
+    },
+    {
+      "id": "bench-med-02",
+      "difficulty": "medium",
+      "description": "Sentiment analysis with negation chains and double negation",
+      "input": "Analyze the sentiment of this text:\n\n\"I wouldn't say the service was bad — the staff wasn't unfriendly, and the response time was not unreasonable. But I can't pretend I was satisfied either. The product itself doesn't lack features, yet none of them work without issues. It's not that I don't appreciate the effort, but the execution leaves much to be desired.\"",
+      "rubric": [
+        {
+          "criterion": "Negation Handling",
+          "weight": 0.4,
+          "scoring": {
+            "5": "Correctly resolves all negation chains: 'wasn't unfriendly' = weakly positive (litotes), 'not unreasonable' = weakly positive, 'can't pretend satisfied' = negative, 'doesn't lack features' = neutral-positive, 'none work without issues' = negative, 'not that I don't appreciate' = weakly positive but pivots to negative",
+            "3": "Resolves simple negations correctly but mishandles double negations or litotes",
+            "1": "Treats negated positives as positive or misreads most negation",
+            "0": "Ignores negation entirely"
+          }
+        },
+        {
+          "criterion": "Overall Polarity",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Correctly identifies the overall sentiment as slightly negative to negative — the author uses polite hedging but the cumulative message is dissatisfaction. The politeness does not make it neutral.",
+            "3": "Identifies as negative but doesn't capture the hedged, polite-negative nuance",
+            "1": "Classifies as neutral or positive due to the polite framing",
+            "0": "Completely wrong classification"
+          }
+        },
+        {
+          "criterion": "Confidence Calibration",
+          "weight": 0.15,
+          "scoring": {
+            "5": "Assigns moderate confidence (0.60-0.75) reflecting the genuine ambiguity and hedged language; flags the complexity of the negation patterns",
+            "3": "Provides confidence but it's either too high (>0.90) or too low (<0.40)",
+            "1": "No confidence scoring",
+            "0": "No analysis"
+          }
+        },
+        {
+          "criterion": "Output Quality",
+          "weight": 0.15,
+          "scoring": {
+            "5": "Structured output with each negation chain explained, aspect breakdown (service, product features, execution), and hedging explicitly noted",
+            "3": "Basic structure but negation resolution not explained",
+            "1": "Minimal output",
+            "0": "No usable output"
+          }
+        }
+      ],
+      "expectedScoreWithout": 25,
+      "expectedScoreWith": 70
+    },
+    {
+      "id": "bench-med-03",
+      "difficulty": "medium",
+      "description": "Social media text with emoji, slang, and informal language",
+      "input": "Analyze the sentiment of these social media posts:\n\n1. \"this new album is straight fire bruh the beats slap so hard cant stop listening\"\n2. \"ngl the movie was mid at best the cgi looked like a ps2 cutscene\"\n3. \"just got the update and everything is broken again amazing job devs\"\n4. \"lowkey obsessed w this cafe their matcha latte hits different fr fr\"",
+      "rubric": [
+        {
+          "criterion": "Slang and Informal Language",
+          "weight": 0.35,
+          "scoring": {
+            "5": "Correctly interprets all slang: 'fire'/'slap' = strongly positive, 'mid' = negative/mediocre, 'amazing job devs' = sarcastic negative, 'hits different'/'obsessed' = strongly positive; handles 'ngl', 'fr fr', 'bruh', 'lowkey', 'w'",
+            "3": "Gets most slang right but misinterprets 1-2 terms or misses sarcasm in post 3",
+            "1": "Fails to interpret most slang terms; treats unfamiliar terms as neutral",
+            "0": "Cannot process informal language"
+          }
+        },
+        {
+          "criterion": "Sarcasm Detection",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Correctly identifies post 3 as sarcastic — 'amazing job devs' with context of 'everything is broken' signals sarcasm; inverts polarity to strongly negative; distinguishes from the genuine positive sentiment in posts 1 and 4",
+            "3": "Detects sarcasm in post 3 but with low confidence, or misflags another post as sarcastic",
+            "1": "Misses sarcasm in post 3; takes 'amazing job' at face value",
+            "0": "No sarcasm detection capability"
+          }
+        },
+        {
+          "criterion": "Per-Post Accuracy",
+          "weight": 0.2,
+          "scoring": {
+            "5": "All 4 posts correctly classified: post 1 strongly positive, post 2 negative, post 3 strongly negative (sarcasm-inverted), post 4 strongly positive",
+            "3": "3 out of 4 posts correctly classified",
+            "1": "2 or fewer posts correctly classified",
+            "0": "All posts misclassified"
+          }
+        },
+        {
+          "criterion": "Output Quality",
+          "weight": 0.15,
+          "scoring": {
+            "5": "Each post analyzed separately with polarity, slang interpretation noted, sarcasm flagged where applicable, domain identified as social media",
+            "3": "Posts analyzed but slang interpretation not explained",
+            "1": "Single aggregate sentiment for all posts",
+            "0": "No usable output"
+          }
+        }
+      ],
+      "expectedScoreWithout": 25,
+      "expectedScoreWith": 70
+    },
+    {
+      "id": "bench-med-04",
+      "difficulty": "medium",
+      "description": "Comparative sentiment requiring entity-specific opinion tracking",
+      "input": "Analyze the sentiment of this text:\n\n\"After switching from Android to iPhone, I have to say the ecosystem integration is miles ahead — Airdrop, iMessage, and Handoff work seamlessly. The camera on the iPhone is also noticeably sharper in low-light conditions. That said, I really miss the customization freedom on Android. The notification system on iOS is still frustratingly limited compared to what Android offers, and I hate that I can't set default apps properly. The hardware build on both is equally premium. If Google could match Apple's ecosystem cohesion, it would be the perfect phone.\"",
+      "rubric": [
+        {
+          "criterion": "Entity-Specific Sentiment",
+          "weight": 0.35,
+          "scoring": {
+            "5": "Correctly tracks sentiment per entity: iPhone positive (ecosystem, camera), iPhone negative (notifications, default apps, customization); Android positive (customization, notifications), Android negative (ecosystem cohesion); hardware neutral/positive for both",
+            "3": "Distinguishes some entity-specific sentiments but conflates opinions or misattributes 1-2",
+            "1": "Treats all sentiment as applying to one entity",
+            "0": "No entity-level separation"
+          }
+        },
+        {
+          "criterion": "Comparative Expression Handling",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Correctly interprets 'miles ahead' (strongly positive for iPhone ecosystem), 'noticeably sharper' (positive iPhone camera), 'frustratingly limited compared to' (negative iOS, positive Android for notifications), 'equally premium' (neutral comparison), 'if Google could match' (conditional — implies Android is worse on ecosystem)",
+            "3": "Handles explicit comparisons but misses implied comparative ('if Google could match')",
+            "1": "Misinterprets comparisons or ignores comparative structure",
+            "0": "No comparative handling"
+          }
+        },
+        {
+          "criterion": "Conditional/Irrealis Detection",
+          "weight": 0.15,
+          "scoring": {
+            "5": "Identifies 'If Google could match Apple's ecosystem cohesion, it would be the perfect phone' as conditional/hypothetical; notes it implies current Android ecosystem is inferior while expressing positive hypothetical",
+            "3": "Notes the conditional but doesn't fully interpret its sentiment implications",
+            "1": "Treats conditional as asserted fact",
+            "0": "Ignores conditional entirely"
+          }
+        },
+        {
+          "criterion": "Output Structure",
+          "weight": 0.2,
+          "scoring": {
+            "5": "Structured output with separate sentiment summaries for iPhone and Android, aspect-level breakdown per entity, comparisons explicitly noted, conditional flagged",
+            "3": "Some entity separation but incomplete aspect detail",
+            "1": "Single-entity output or unstructured",
+            "0": "No usable output"
+          }
+        }
+      ],
+      "expectedScoreWithout": 25,
+      "expectedScoreWith": 70
+    },
+    {
+      "id": "bench-hard-01",
+      "difficulty": "hard",
+      "description": "Subtle sarcasm, irony, and implicit sentiment in a lengthy review",
+      "input": "Analyze the sentiment of this text:\n\n\"I must commend the airline for their unwavering commitment to mediocrity. In an age where most carriers are at least trying to improve, it's refreshing to find one that has perfected the art of consistent disappointment.\n\nLet me count the ways: The 'spacious' economy seats — perfect if you happen to be a contortionist. The 'complimentary' beverage service — a thimble of water delivered with the enthusiasm of someone fulfilling a court-ordered community service obligation. And my personal favorite: the 'on-time departure' that only required us to redefine what 'on time' means by approximately 90 minutes.\n\nThe in-flight entertainment system featured a cutting-edge selection of films from 2019, which I suppose counts as 'curated' in the same way my junk drawer is 'curated.' The Wi-Fi, advertised at $12.99, delivered speeds that would make a dial-up modem feel nostalgic.\n\nI will say, in complete sincerity, that the flight crew was genuinely kind and professional throughout. They deserve to work for a better airline.\n\nFive stars. Would definitely recommend to anyone I secretly dislike.\"",
+      "rubric": [
+        {
+          "criterion": "Sarcasm and Irony Detection",
+          "weight": 0.35,
+          "scoring": {
+            "5": "Correctly identifies pervasive sarcasm throughout: 'commend... mediocrity', 'refreshing... disappointment', 'spacious' in scare quotes, 'complimentary' as ironic, 'on-time departure' as ironic, 'cutting-edge... 2019', 'curated' as mocking, 'five stars' as sarcastic, 'recommend to anyone I secretly dislike' as confirming sarcastic intent. Correctly identifies the ONE genuine positive statement about the flight crew.",
+            "3": "Detects sarcasm in most obvious cases but misses some or incorrectly flags the genuine compliment about the crew as sarcastic",
+            "1": "Detects some sarcasm but misclassifies the overall tone or misses the crew compliment as genuine",
+            "0": "Takes the review at face value or misses sarcasm entirely"
+          }
+        },
+        {
+          "criterion": "Genuine vs. Sarcastic Sentiment Separation",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Correctly separates the genuine positive sentiment about the crew ('in complete sincerity', 'genuinely kind and professional') from the sarcastic negative sentiment about everything else. Notes the explicit sincerity marker as evidence of genuine opinion.",
+            "3": "Separates crew sentiment but with low confidence, or partially misattributes sarcastic/genuine labels",
+            "1": "Treats all sentiment as either sarcastic or genuine without distinction",
+            "0": "No separation attempted"
+          }
+        },
+        {
+          "criterion": "Aspect-Level Detail",
+          "weight": 0.2,
+          "scoring": {
+            "5": "Extracts 7+ aspects: seats/comfort (sarcastic, actually negative), beverage service (sarcastic, negative), punctuality (sarcastic, negative), entertainment (sarcastic, negative), Wi-Fi (negative), crew (genuinely positive), overall recommendation (sarcastic negative). Each with inverted polarity and confidence.",
+            "3": "Extracts 4-6 aspects with mostly correct sarcasm-inverted polarities",
+            "1": "Extracts 1-3 aspects or misses sarcasm inversion",
+            "0": "No aspect extraction"
+          }
+        },
+        {
+          "criterion": "Confidence and Flags",
+          "weight": 0.15,
+          "scoring": {
+            "5": "High confidence on sarcasm detection (multiple reinforcing signals), flags entire review as 'predominantly sarcastic', notes the sincerity marker exception, document-level is strongly negative despite surface-positive language",
+            "3": "Flags sarcasm but confidence calibration is off (too low given strong signals)",
+            "1": "No sarcasm flagging or confidence",
+            "0": "No flags or confidence"
+          }
+        }
+      ],
+      "expectedScoreWithout": 20,
+      "expectedScoreWith": 65
+    },
+    {
+      "id": "bench-hard-02",
+      "difficulty": "hard",
+      "description": "Multi-entity opinion mining with reported speech and opinion holder disambiguation",
+      "input": "Analyze the sentiment of this text:\n\n\"The board meeting was contentious. CEO Sarah Chen defended the Q3 results, calling them 'a strong performance given market headwinds,' but CFO Mark Rivera countered that the margins were 'dangerously thin' and warned the current trajectory was 'unsustainable.' Several board members expressed concern about the R&D budget, which they described as 'reckless spending with no clear ROI.'\n\nIndustry analyst Patricia Wong noted in her post-meeting report that 'the company's strategic pivot shows promise but execution risk remains elevated.' She praised the new product line but questioned whether the management team had the operational discipline to scale it.\n\nInternal employee sentiment, based on a recent Glassdoor survey, shows 72% satisfaction with company culture but only 34% confidence in senior leadership's direction. One anonymous review stated: 'Great colleagues, great mission, terrible management decisions.'\"",
+      "rubric": [
+        {
+          "criterion": "Opinion Holder Identification",
+          "weight": 0.35,
+          "scoring": {
+            "5": "Correctly identifies and separates 5+ opinion holders: Sarah Chen (positive on Q3), Mark Rivera (negative on margins/trajectory), board members (negative on R&D), Patricia Wong (mixed — positive on pivot, negative on execution risk), employees (positive on culture, negative on leadership). Each opinion correctly attributed to its holder.",
+            "3": "Identifies 3-4 holders with mostly correct attribution but conflates some opinions",
+            "1": "Identifies 1-2 holders or misattributes opinions significantly",
+            "0": "No opinion holder tracking; treats all as a single voice"
+          }
+        },
+        {
+          "criterion": "Reported vs. Direct Sentiment",
+          "weight": 0.25,
+          "scoring": {
+            "5": "Correctly distinguishes reported speech (quoted opinions from Chen, Rivera, board members, Wong, employee) from narrative framing. Notes that the text author is reporting, not expressing their own opinion. Identifies direct quotes vs. paraphrased opinions.",
+            "3": "Distinguishes some reported speech but misses the narrative framing or misattributes the author's stance",
+            "1": "Treats all sentiment as the text author's opinion",
+            "0": "No distinction between reported and direct sentiment"
+          }
+        },
+        {
+          "criterion": "Aspect-Entity Matrix",
+          "weight": 0.25,
+          "scoring": {
+            "5": "Builds a complete matrix: Q3 results (Chen: positive, Rivera: negative), R&D budget (board: negative), strategic pivot (Wong: positive), execution capability (Wong: negative), company culture (employees: positive), leadership (employees: negative). Quantitative data (72%, 34%) correctly interpreted.",
+            "3": "Covers most aspect-entity pairs but misses some or doesn't integrate quantitative data",
+            "1": "Partial matrix with significant gaps",
+            "0": "No aspect-entity tracking"
+          }
+        },
+        {
+          "criterion": "Output Quality",
+          "weight": 0.15,
+          "scoring": {
+            "5": "Structured output organized by opinion holder with aspect-level detail per holder; overall synthesis noting the contentious/divided nature of sentiment; confidence adjusted for reported speech",
+            "3": "Some organization by holder but synthesis is superficial",
+            "1": "Flat list or single aggregate sentiment",
+            "0": "No usable output"
+          }
+        }
+      ],
+      "expectedScoreWithout": 20,
+      "expectedScoreWith": 60
+    },
+    {
+      "id": "bench-hard-03",
+      "difficulty": "hard",
+      "description": "Cross-domain sentiment requiring domain calibration and implicit sentiment detection",
+      "input": "Analyze the sentiment of these three texts from different domains:\n\nText A (Medical case note): \"Patient presents with stable vitals and unremarkable lab results. The tumor has not progressed since the last scan. While the prognosis remains guarded, the treatment protocol appears to be achieving its intended effect. No new contraindications were identified.\"\n\nText B (Financial analyst report): \"Q4 earnings were flat against consensus expectations. The company maintained its aggressive acquisition strategy despite rising interest rates. Revenue growth decelerated to 3% YoY, though management characterized this as 'disciplined growth.' Cash reserves remain robust at $2.1B, providing a defensive buffer against market volatility.\"\n\nText C (Restaurant critic review): \"Chef Martinez's new tasting menu is an exercise in restraint. The flavors are precise rather than bold, the plating understated rather than theatrical. Some may find the portions challenging, but each course demonstrates an uncommon technical mastery. The wine pairing, curated by sommelier Ana Torres, is nothing short of revelatory. A quiet triumph.\"",
+      "rubric": [
+        {
+          "criterion": "Domain Detection and Calibration",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Correctly detects all three domains (medical, financial, culinary) and calibrates accordingly: 'stable' is positive in medical, 'aggressive' is contextual in finance, 'restraint' is positive in fine dining criticism. 'Unremarkable' is positive in medical (no abnormalities), 'flat' is neutral-negative in finance, 'challenging' portions is diplomatic negative in food criticism.",
+            "3": "Detects domains but miscalibrates 2-3 domain-specific terms",
+            "1": "Applies general-purpose sentiment without domain calibration",
+            "0": "No domain awareness"
+          }
+        },
+        {
+          "criterion": "Implicit Sentiment Detection",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Detects implicit sentiment: Text A — cautiously positive (no progression is good in oncology, 'guarded' is standard hedging not negative); Text B — mixed (flat earnings = neutral-negative, 'disciplined growth' in quotes suggests skepticism of management's framing, 'defensive buffer' is positive); Text C — strongly positive despite restrained language ('uncommon technical mastery', 'revelatory', 'quiet triumph' are high praise in critic register)",
+            "3": "Gets overall direction right for each text but misses subtle implicit signals",
+            "1": "Misreads the register — treats medical hedging as negative, critic's restraint as lukewarm",
+            "0": "Completely misreads the implicit sentiment"
+          }
+        },
+        {
+          "criterion": "Register and Tone Awareness",
+          "weight": 0.25,
+          "scoring": {
+            "5": "Recognizes that each text uses a different register: clinical objectivity (Text A), analytical finance (Text B), literary criticism (Text C). Notes that sentiment expression norms differ: hedging is standard in medical/clinical, 'quiet triumph' is superlative praise from a food critic, scare quotes around 'disciplined growth' signal analyst skepticism.",
+            "3": "Notes different registers but doesn't fully adjust interpretation",
+            "1": "Treats all three texts with the same interpretive framework",
+            "0": "No register awareness"
+          }
+        },
+        {
+          "criterion": "Output Quality",
+          "weight": 0.15,
+          "scoring": {
+            "5": "Each text analyzed separately with domain label, register notes, aspect-level detail, and calibrated confidence. Cross-domain comparison notes how the same words carry different weight.",
+            "3": "Separate analysis per text but without cross-domain comparison or register notes",
+            "1": "Combined analysis or missing structure",
+            "0": "No usable output"
+          }
+        }
+      ],
+      "expectedScoreWithout": 15,
+      "expectedScoreWith": 60
+    }
+  ]
+}

package/tests/smoke.json

@@ -0,0 +1,54 @@
+{
+  "version": "0.0.1",
+  "timeout": 60,
+  "tasks": [
+    {
+      "id": "smoke-01",
+      "description": "Analyze sentiment of a product review with mixed opinions across multiple aspects and a sarcastic remark",
+      "input": "Analyze the sentiment of this product review:\n\n\"I've been using this wireless headphone for two weeks now. The sound quality is absolutely phenomenal — rich bass, crisp highs, and a wide soundstage that rivals wired headphones. The noise cancellation is decent, though not quite as good as the Sony XM5s. However, the battery life is a complete joke. They advertise 30 hours but I'm lucky to get 15. Oh, and the 'premium' companion app? It crashes every single time I try to adjust the EQ. At least the comfort is great — I can wear them for hours without any ear fatigue. Overall, good sound hampered by poor software and misleading battery claims.\"",
+      "rubric": [
+        {
+          "criterion": "Aspect Identification",
+          "weight": 0.25,
+          "scoring": {
+            "5": "Identifies all 5+ aspects: sound quality, noise cancellation, battery life, companion app, comfort; correctly categorizes each",
+            "3": "Identifies 3-4 aspects but misses some or miscategorizes",
+            "1": "Identifies only 1-2 obvious aspects",
+            "0": "No aspect-level analysis performed"
+          }
+        },
+        {
+          "criterion": "Polarity Accuracy",
+          "weight": 0.3,
+          "scoring": {
+            "5": "Correctly assigns: sound quality (strongly positive), noise cancellation (slightly positive), battery life (strongly negative), app (strongly negative), comfort (positive); detects sarcasm in 'premium' and 'joke'",
+            "3": "Gets most polarities right but misses sarcasm or misclassifies 1-2 aspects",
+            "1": "Only gets document-level polarity roughly right; aspect polarities are incorrect",
+            "0": "Polarity assignments are largely wrong"
+          }
+        },
+        {
+          "criterion": "Valence Shifter Handling",
+          "weight": 0.2,
+          "scoring": {
+            "5": "Correctly processes: 'absolutely' as intensifier on 'phenomenal', 'not quite as good' as diminished comparison, 'complete joke' as sarcastic negative, scare quotes on 'premium' as ironic",
+            "3": "Handles basic intensifiers/negation but misses sarcastic usage or scare quotes",
+            "1": "Ignores most valence shifters",
+            "0": "No valence shifter detection"
+          }
+        },
+        {
+          "criterion": "Output Structure and Confidence",
+          "weight": 0.25,
+          "scoring": {
+            "5": "Structured output with aspect-level detail, confidence scores, document-level summary labeled as 'mixed', sarcasm flagged, and the concluding sentence used to anchor overall sentiment",
+            "3": "Has aspect breakdown and document summary but missing confidence scores or sarcasm flags",
+            "1": "Only provides document-level sentiment without aspect detail",
+            "0": "Unstructured or no usable output"
+          }
+        }
+      ],
+      "passThreshold": 60
+    }
+  ]
+}