ragaai-catalyst 2.0.7.2b0__py3-none-any.whl → 2.1__py3-none-any.whl

ragaai_catalyst/tracers/agentic_tracing/examples/FinancialAnalysisSystem.ipynb

@@ -0,0 +1,536 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Financial Analysis System with AgentNeo Integration\n",
+    "\n",
+    "This Jupyter notebook demonstrates the integration of AgentNeo, a powerful tracing and monitoring tool, with a financial analysis system. AgentNeo provides seamless tracing capabilities for both function calls and AI model interactions, allowing for comprehensive analysis and debugging of complex systems."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setup and Imports\n",
+    "\n",
+    "First, let's import the necessary libraries and set up our environment."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:httpx:HTTP Request: GET https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json \"HTTP/1.1 200 OK\"\n"
+     ]
+    }
+   ],
+   "source": [
+    "import os\n",
+    "os.chdir('..')\n",
+    "\n",
+    "import random\n",
+    "from textblob import TextBlob\n",
+    "import openai\n",
+    "from dotenv import load_dotenv\n",
+    "from agentneo import AgentNeo, Tracer, Evaluation\n",
+    "\n",
+    "# Load environment variables\n",
+    "load_dotenv(\"YOUR_ENV_FILE\")\n",
+    "\n",
+    "# Initialize OpenAI API\n",
+    "openai.api_key = os.getenv(\"OPENAI_API_KEY\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## FinancialAnalysisSystem Class\n",
+    "\n",
+    "Now, let's define our `FinancialAnalysisSystem` class with AgentNeo integration."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Project 'financial_analysis_project2' found.\n",
+      "Tracing Started.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Initialize AgentNeo session\n",
+    "neo_session = AgentNeo(session_name=\"financial_analysis_session2\")\n",
+    "\n",
+    "# Create project\n",
+    "neo_session.create_project(project_name=\"financial_analysis_project2\")\n",
+    "\n",
+    "# Start tracing\n",
+    "tracer = Tracer(session=neo_session)\n",
+    "tracer.start()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class FinancialAnalysisSystem:\n",
+    "    def __init__(self):\n",
+    "        self.stock_data = {}\n",
+    "        self.news_sentiment = {}\n",
+    "        self.economic_indicators = {}\n",
+    "\n",
+    "    @tracer.trace_tool(name=\"fetch_stock_data\")\n",
+    "    def fetch_stock_data(self, symbol):\n",
+    "        return {\n",
+    "            \"symbol\": symbol,\n",
+    "            \"price\": round(random.uniform(50, 500), 2),\n",
+    "            \"change\": round(random.uniform(-5, 5), 2),\n",
+    "        }\n",
+    "\n",
+    "    @tracer.trace_tool(name=\"fetch_news_articles\")\n",
+    "    def fetch_news_articles(self, company):\n",
+    "        return [\n",
+    "            f\"{company} announces new product line\",\n",
+    "            f\"{company} reports quarterly earnings\",\n",
+    "            f\"{company} faces regulatory scrutiny\",\n",
+    "        ]\n",
+    "\n",
+    "    @tracer.trace_tool(name=\"analyze_sentiment\")\n",
+    "    def analyze_sentiment(self, text):\n",
+    "        return TextBlob(text).sentiment.polarity\n",
+    "\n",
+    "    @tracer.trace_tool(name=\"fetch_economic_indicators\")\n",
+    "    def fetch_economic_indicators(self):\n",
+    "        return {\n",
+    "            \"gdp_growth\": round(random.uniform(-2, 5), 2),\n",
+    "            \"unemployment_rate\": round(random.uniform(3, 10), 2),\n",
+    "            \"inflation_rate\": round(random.uniform(0, 5), 2),\n",
+    "        }\n",
+    "\n",
+    "    @tracer.trace_llm(name=\"analyze_market_conditions\")\n",
+    "    def analyze_market_conditions(self, stock_data, sentiment, economic_indicators):\n",
+    "        prompt = f\"\"\"\n",
+    "        Analyze the following market conditions and provide a brief market outlook:\n",
+    "        Stock: {stock_data['symbol']} at ${stock_data['price']} (change: {stock_data['change']}%)\n",
+    "        News Sentiment: {sentiment}\n",
+    "        Economic Indicators:\n",
+    "        - GDP Growth: {economic_indicators['gdp_growth']}%\n",
+    "        - Unemployment Rate: {economic_indicators['unemployment_rate']}%\n",
+    "        - Inflation Rate: {economic_indicators['inflation_rate']}%\n",
+    "        \"\"\"\n",
+    "        response = openai.chat.completions.create(\n",
+    "            model=\"gpt-4-0125-preview\",\n",
+    "            messages=[{\"role\": \"user\", \"content\": prompt}],\n",
+    "            max_tokens=150,\n",
+    "        )\n",
+    "        return response.choices[0].message.content.strip()\n",
+    "\n",
+    "    @tracer.trace_llm(name=\"generate_investment_recommendation\")\n",
+    "    def generate_investment_recommendation(self, market_outlook, risk_tolerance):\n",
+    "        prompt = f\"\"\"\n",
+    "        Based on the following market outlook and investor risk tolerance,\n",
+    "        provide a specific investment recommendation:\n",
+    "        Market Outlook: {market_outlook}\n",
+    "        Investor Risk Tolerance: {risk_tolerance}\n",
+    "        \"\"\"\n",
+    "        response = openai.chat.completions.create(\n",
+    "            model=\"gpt-4-0125-preview\",\n",
+    "            messages=[{\"role\": \"user\", \"content\": prompt}],\n",
+    "            max_tokens=200,\n",
+    "        )\n",
+    "        return response.choices[0].message.content.strip()\n",
+    "\n",
+    "    @tracer.trace_agent(name=\"FinancialAdvisorAgent\")\n",
+    "    def financial_advisor_agent(self, stock_symbol, risk_tolerance):\n",
+    "        self.stock_data = self.fetch_stock_data(stock_symbol)\n",
+    "        news_articles = self.fetch_news_articles(stock_symbol)\n",
+    "        sentiment_scores = [self.analyze_sentiment(article) for article in news_articles]\n",
+    "        self.news_sentiment = sum(sentiment_scores) / len(sentiment_scores)\n",
+    "        self.economic_indicators = self.fetch_economic_indicators()\n",
+    "        market_outlook = self.analyze_market_conditions(\n",
+    "            self.stock_data, self.news_sentiment, self.economic_indicators\n",
+    "        )\n",
+    "        recommendation = self.generate_investment_recommendation(market_outlook, risk_tolerance)\n",
+    "        return recommendation\n",
+    "\n",
+    "    def run_analysis(self, stock_symbol, risk_tolerance):\n",
+    "        recommendation = self.financial_advisor_agent(stock_symbol, risk_tolerance)\n",
+    "        print(f\"\\nAnalysis for {stock_symbol}:\")\n",
+    "        print(f\"Stock Data: {self.stock_data}\")\n",
+    "        print(f\"News Sentiment: {self.news_sentiment}\")\n",
+    "        print(f\"Economic Indicators: {self.economic_indicators}\")\n",
+    "        print(f\"\\nInvestment Recommendation:\\n{recommendation}\")\n",
+    "        if \"buy\" in recommendation.lower():\n",
+    "            self.execute_buy_order(stock_symbol)\n",
+    "        elif \"sell\" in recommendation.lower():\n",
+    "            self.execute_sell_order(stock_symbol)\n",
+    "        else:\n",
+    "            print(\"No action taken based on the current recommendation.\")\n",
+    "\n",
+    "    @tracer.trace_tool(name=\"execute_buy_order\")\n",
+    "    def execute_buy_order(self, symbol):\n",
+    "        print(f\"Executing buy order for {symbol}\")\n",
+    "\n",
+    "    @tracer.trace_tool(name=\"execute_sell_order\")\n",
+    "    def execute_sell_order(self, symbol):\n",
+    "        print(f\"Executing sell order for {symbol}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Running the Analysis\n",
+    "\n",
+    "Now let's create an instance of our `FinancialAnalysisSystem` and run an analysis."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "DEBUG:agentneo.tracing.agent_tracer:Successfully updated and committed AgentCallModel with id 39\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Analysis for AAPL:\n",
+      "Stock Data: {'symbol': 'AAPL', 'price': 215.96, 'change': -1.66}\n",
+      "News Sentiment: 0.04545454545454545\n",
+      "Economic Indicators: {'gdp_growth': 4.75, 'unemployment_rate': 4.82, 'inflation_rate': 0.87}\n",
+      "\n",
+      "Investment Recommendation:\n",
+      "Given the information provided on AAPL and the broader economic indicators, and considering an investor with moderate risk tolerance, a specific investment recommendation would be as follows:\n",
+      "\n",
+      "### Investment Recommendation: Diversified Approach with Focus on AAPL\n",
+      "\n",
+      "### 1. Partial Investment in AAPL:\n",
+      "Given AAPL's current trading price and the slight dip it has experienced, it could be an opportune moment to buy the stock for those with a moderate risk tolerance. The decrease in AAPL’s share price might not necessarily be due to company-specific negatives as it's within normal market fluctuations. Additionally, the marginally positive news sentiment suggests there isn't a prevailing negative view on the company, which could be seen as a positive indicator. It would be prudent to allocate a portion of the investment portfolio to AAPL shares, capitalizing on the current lower price with the expectation of long-term growth. AAPL's historical performance, its strong product ecosystem, and continuous innovation could be viewed as catalysts for future growth.\n",
+      "Executing buy order for AAPL\n",
+      "Tracing Completed.\n",
+      "Data saved to the database and JSON file.\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Create an instance of FinancialAnalysisSystem\n",
+    "analysis_system = FinancialAnalysisSystem()\n",
+    "\n",
+    "# Run an analysis for Apple stock with moderate risk tolerance\n",
+    "analysis_system.run_analysis(\"AAPL\", \"moderate\")\n",
+    "\n",
+    "# Stop the tracer when analysis is complete\n",
+    "tracer.stop()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Evaluation using Metrics"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "\u001b[92m17:21:46 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:21:48 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:21:48 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:21:51 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:21:51 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:21:55 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:21:55 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:21:57 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:21:57 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:22:01 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:22:01 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:22:02 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:22:02 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-4o-mini; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:22:07 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:22:07 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:22:08 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:22:08 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:22:09 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:22:09 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:22:10 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:22:10 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:22:11 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:22:11 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:22:13 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:22:13 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:22:14 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:22:14 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:22:15 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n",
+      "\u001b[92m17:22:15 - LiteLLM:INFO\u001b[0m: utils.py:2740 - \n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:LiteLLM:\n",
+      "LiteLLM completion() model= gpt-3.5-turbo; provider = openai\n",
+      "INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
+      "\u001b[92m17:22:17 - LiteLLM:INFO\u001b[0m: utils.py:938 - Wrapper: Completed Call, calling success_handler\n",
+      "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n"
+     ]
+    }
+   ],
+   "source": [
+    "exe = Evaluation(session=neo_session, trace_id=tracer.trace_id)\n",
+    "\n",
+    "# run a single metric\n",
+    "exe.evaluate(metric_list=['goal_decomposition_efficiency', \n",
+    "                         'goal_fulfillment_rate', \n",
+    "                         'tool_call_correctness_rate', \n",
+    "                         'tool_call_success_rate'])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[{'metric_name': 'goal_decomposition_efficiency',\n",
+       "  'score': 0.85,\n",
+       "  'reason': 'The decomposition of the original goal into sub-tasks is largely effective, with each sub-task logically contributing to the overall objective of making an informed investment decision. The tools are appropriately assigned to each sub-task, and the sequence of tasks follows a logical progression from data gathering to analysis and finally to execution. However, while the sub-tasks cover most aspects of the original goal, the sentiment analysis could be more granular by providing a more detailed breakdown of sentiment scores for each article. Overall, the decomposition is efficient and would likely scale well for similar investment analysis tasks.',\n",
+       "  'result_detail': {'metric_name': 'goal_fulfillment_rate',\n",
+       "   'config': {},\n",
+       "   'result': {'originalGoal': 'Make informed investment decisions regarding Apple Inc. (AAPL) stock by understanding current market conditions, including stock performance, news sentiment, and economic indicators.',\n",
+       "    'subtasks': ['Fetch stock data for AAPL.',\n",
+       "     'Fetch news articles related to AAPL.',\n",
+       "     'Analyze sentiment for each news article.',\n",
+       "     'Fetch economic indicators.',\n",
+       "     'Analyze market conditions based on stock performance, news sentiment, and economic indicators.',\n",
+       "     'Generate an investment recommendation based on the market outlook and investor risk tolerance.',\n",
+       "     'Execute a buy order for AAPL shares.'],\n",
+       "    'score': 0.85,\n",
+       "    'reason': 'The decomposition of the original goal into sub-tasks is largely effective, with each sub-task logically contributing to the overall objective of making an informed investment decision. The tools are appropriately assigned to each sub-task, and the sequence of tasks follows a logical progression from data gathering to analysis and finally to execution. However, while the sub-tasks cover most aspects of the original goal, the sentiment analysis could be more granular by providing a more detailed breakdown of sentiment scores for each article. Overall, the decomposition is efficient and would likely scale well for similar investment analysis tasks.'}},\n",
+       "  'config': {},\n",
+       "  'start_time': '2024-10-22T17:21:46.507620',\n",
+       "  'end_time': '2024-10-22T17:21:55.573536',\n",
+       "  'duration': 9.065916},\n",
+       " {'metric_name': 'goal_fulfillment_rate',\n",
+       "  'score': 0.8,\n",
+       "  'reason': \"The system responses effectively address the user's intent to make informed investment decisions regarding AAPL stock. The responses provide relevant stock data, news articles, sentiment analysis, and economic indicators, which are crucial for evaluating the investment potential. The stock performance data indicates a slight dip, which aligns with the user's interest in understanding current market conditions. The news sentiment score, although low, suggests a marginally positive outlook, which is a relevant factor for investment decisions. The analysis of market conditions offers a balanced view, indicating that the price drop may be part of normal fluctuations rather than a significant negative event. Furthermore, the investment recommendation is tailored to the user's moderate risk tolerance, suggesting a partial investment in AAPL, which is a proactive strategy. However, the execution of the buy order is missing, which is a critical step in fulfilling the user's intent to invest. This omission prevents a perfect score, as the user expressed a clear intent to execute a buy order based on the insights provided. Overall, the responses are comprehensive and align well with the user's goals, warranting a score of 0.8.\",\n",
+       "  'result_detail': {'metric_name': 'goal_fulfillment_rate',\n",
+       "   'config': {},\n",
+       "   'result': {'inputGoal': \"The user aims to make informed investment decisions regarding Apple Inc. (AAPL) stock. They seek to understand the current market conditions, including stock performance, news sentiment, and economic indicators, to evaluate the potential for investment. After analyzing the data, the user expresses a moderate risk tolerance and ultimately decides to execute a buy order for AAPL shares, indicating a proactive approach to capitalizing on the stock's current price dip. This reflects a clear intent to invest strategically based on the gathered insights.\",\n",
+       "    'relevantResponses': \"fetch_stock_data: {'symbol': 'AAPL', 'price': 215.96, 'change': -1.66}\\n\\nfetch_news_articles: ['AAPL announces new product line', 'AAPL reports quarterly earnings', 'AAPL faces regulatory scrutiny']\\n\\nanalyze_sentiment: 0.13636363636363635\\n\\nanalyze_sentiment: 0.0\\n\\nanalyze_sentiment: 0.0\\n\\nfetch_economic_indicators: {'gdp_growth': 4.75, 'unemployment_rate': 4.82, 'inflation_rate': 0.87}\\n\\nanalyze_market_conditions: Given the information you've provided about AAPL and the broader economic indicators, here's a brief market outlook:\\n\\n### **Stock Performance: AAPL**\\nAAPL is currently trading at $215.96, which represents a 1.66% decrease. Such a dip could be the result of various factors including market sentiment, company-specific news, or sector-wide shifts. Without more context, it’s difficult to pinpoint the cause, but the drop isn't drastic, suggesting it could be part of normal market fluctuations rather than a response to negative company-specific news.\\n\\n### **News Sentiment**\\nThe news sentiment score is 0.04545454545454545. This score, hovering slightly above zero, indicates a marginally positive sentiment in the news\\n\\ngenerate_investment_recommendation: Given the information provided on AAPL and the broader economic indicators, and considering an investor with moderate risk tolerance, a specific investment recommendation would be as follows:\\n\\n### Investment Recommendation: Diversified Approach with Focus on AAPL\\n\\n### 1. Partial Investment in AAPL:\\nGiven AAPL's current trading price and the slight dip it has experienced, it could be an opportune moment to buy the stock for those with a moderate risk tolerance. The decrease in AAPL’s share price might not necessarily be due to company-specific negatives as it's within normal market fluctuations. Additionally, the marginally positive news sentiment suggests there isn't a prevailing negative view on the company, which could be seen as a positive indicator. It would be prudent to allocate a portion of the investment portfolio to AAPL shares, capitalizing on the current lower price with the expectation of long-term growth. AAPL's historical performance, its strong product ecosystem, and continuous innovation could be viewed as catalysts for future growth. \\n\\n\\n\\nexecute_buy_order: None\",\n",
+       "    'score': 0.8,\n",
+       "    'reason': \"The system responses effectively address the user's intent to make informed investment decisions regarding AAPL stock. The responses provide relevant stock data, news articles, sentiment analysis, and economic indicators, which are crucial for evaluating the investment potential. The stock performance data indicates a slight dip, which aligns with the user's interest in understanding current market conditions. The news sentiment score, although low, suggests a marginally positive outlook, which is a relevant factor for investment decisions. The analysis of market conditions offers a balanced view, indicating that the price drop may be part of normal fluctuations rather than a significant negative event. Furthermore, the investment recommendation is tailored to the user's moderate risk tolerance, suggesting a partial investment in AAPL, which is a proactive strategy. However, the execution of the buy order is missing, which is a critical step in fulfilling the user's intent to invest. This omission prevents a perfect score, as the user expressed a clear intent to execute a buy order based on the insights provided. Overall, the responses are comprehensive and align well with the user's goals, warranting a score of 0.8.\"}},\n",
+       "  'config': {},\n",
+       "  'start_time': '2024-10-22T17:21:55.573671',\n",
+       "  'end_time': '2024-10-22T17:22:01.774679',\n",
+       "  'duration': 6.201008},\n",
+       " {'metric_name': 'tool_call_correctness_rate',\n",
+       "  'score': 0.7142857142857143,\n",
+       "  'reason': 'The correctness rate of 0.71 (or 71%) indicates that out of the total 7 tool calls made, 5 were appropriate and aligned with the intended tools for the query. \\n\\nIn this interaction, the user requested an analysis of market conditions for AAPL stock, which required fetching stock data, analyzing sentiment, and reviewing economic indicators. The intended tools were correctly identified as `fetch_stock_data`, `analyze_sentiment`, and `fetch_economic_indicators`. \\n\\nHowever, the total calls made included 2 additional calls that were not necessary for fulfilling the query. These could have been calls to `fetch_news_articles` or `execute_buy_order`, which do not directly contribute to the analysis requested. \\n\\nThe discrepancy between the intended tools and the actual tool usage led to the 2 incorrect calls, resulting in a correctness rate of 5 correct calls out of 7 total calls. This highlights the importance of using only the relevant tools for a given query to improve efficiency and accuracy in tool usage.',\n",
+       "  'result_detail': {'metric_name': 'tool_correctness',\n",
+       "   'config': {},\n",
+       "   'result': {'score': 0.7142857142857143,\n",
+       "    'reason': 'The correctness rate of 0.71 (or 71%) indicates that out of the total 7 tool calls made, 5 were appropriate and aligned with the intended tools for the query. \\n\\nIn this interaction, the user requested an analysis of market conditions for AAPL stock, which required fetching stock data, analyzing sentiment, and reviewing economic indicators. The intended tools were correctly identified as `fetch_stock_data`, `analyze_sentiment`, and `fetch_economic_indicators`. \\n\\nHowever, the total calls made included 2 additional calls that were not necessary for fulfilling the query. These could have been calls to `fetch_news_articles` or `execute_buy_order`, which do not directly contribute to the analysis requested. \\n\\nThe discrepancy between the intended tools and the actual tool usage led to the 2 incorrect calls, resulting in a correctness rate of 5 correct calls out of 7 total calls. This highlights the importance of using only the relevant tools for a given query to improve efficiency and accuracy in tool usage.',\n",
+       "    'details': {'correct_calls': 5,\n",
+       "     'total_calls': 7,\n",
+       "     'intended_tools': ['fetch_stock_data',\n",
+       "      'analyze_sentiment',\n",
+       "      'fetch_economic_indicators'],\n",
+       "     'available_tools': ['fetch_stock_data',\n",
+       "      'analyze_sentiment',\n",
+       "      'fetch_news_articles',\n",
+       "      'execute_buy_order',\n",
+       "      'fetch_economic_indicators']}}},\n",
+       "  'config': {},\n",
+       "  'start_time': '2024-10-22T17:22:01.774778',\n",
+       "  'end_time': '2024-10-22T17:22:07.358702',\n",
+       "  'duration': 5.583924},\n",
+       " {'metric_name': 'tool_call_success_rate',\n",
+       "  'score': 0.8571428571428571,\n",
+       "  'reason': \"The tool call had an overall success rate of 0.86, with the majority of tool calls (6 out of 7) being successful. The successful tool calls returned the expected output without any visible errors, indicating that the tool was able to retrieve and process the requested information effectively. However, there was one failed tool call where the output was 'None', suggesting that there may have been an issue with generating the output. In general, the tool's performance was good, but there may be room for improvement in handling certain edge cases or errors.\",\n",
+       "  'result_detail': {'metric_name': 'tool_call_success_rate',\n",
+       "   'config': {},\n",
+       "   'result': {'score': 0.8571428571428571,\n",
+       "    'reason': \"The tool call had an overall success rate of 0.86, with the majority of tool calls (6 out of 7) being successful. The successful tool calls returned the expected output without any visible errors, indicating that the tool was able to retrieve and process the requested information effectively. However, there was one failed tool call where the output was 'None', suggesting that there may have been an issue with generating the output. In general, the tool's performance was good, but there may be room for improvement in handling certain edge cases or errors.\"}},\n",
+       "  'config': {},\n",
+       "  'start_time': '2024-10-22T17:22:07.358790',\n",
+       "  'end_time': '2024-10-22T17:22:17.297285',\n",
+       "  'duration': 9.938495}]"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "results = exe.get_results()\n",
+    "results"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Analyzing the Results\n",
+    "\n",
+    "After running the analysis, you can examine the output to see the stock data, news sentiment, economic indicators, and the investment recommendation. The AgentNeo tracer will have logged all the steps of the process, which you can later analyze using the AgentNeo dashboard.\n",
+    "\n",
+    "To launch the AgentNeo dashboard and analyze the traces, you can use:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "INFO:root:Port 3000 is busy. Finding an available port...\n",
+      "INFO:root:Using port 3002\n",
+      "INFO:root:Dashboard launched successfully. Access it at: http://localhost:3002\n"
+     ]
+    }
+   ],
+   "source": [
+    "neo_session.launch_dashboard(port=3000)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This will allow you to visualize the execution flow, identify any bottlenecks, and gain insights into the decision-making process of your financial analysis system."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "base",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.4"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}