neurostats-API 0.0.14__tar.gz → 0.0.16__tar.gz

{neurostats_api-0.0.14/neurostats_API.egg-info → neurostats_api-0.0.16}/PKG-INFO

@@ -1,12 +1,18 @@
 Metadata-Version: 2.1
 Name: neurostats_API
-Version: 0.0.14
+Version: 0.0.16
 Summary: The service of NeuroStats website
 Home-page: https://github.com/NeurowattStats/NeuroStats_API.git
 Author: JasonWang@Neurowatt
 Author-email: jason@neurowatt.ai
 Requires-Python: >=3.6
 Description-Content-Type: text/markdown
+Requires-Dist: numpy>=2.1.0
+Requires-Dist: pandas>=2.2.0
+Requires-Dist: pymongo
+Requires-Dist: pytz
+Requires-Dist: python-dotenv
+Requires-Dist: yfinance
 
 # neurostats_API
 
@@ -83,7 +89,7 @@ pip install neurostats-API
 ```Python 
 >>> import neurostats_API
 >>> print(neurostats_API.__version__)
-0.0.14
+0.0.16
 ```
 
 ### 得到最新一期的評價資料與歷年評價
@@ -667,7 +673,121 @@ fetcher.query()
 
 請注意`range`, `last_range`, `52week_range`這三個項目型態為字串，其餘為float
 
+
+## TEJ 相關
+### 會計師簽證財務資料
+```Python
+from neurostats_API import FinanceReportFetcher
+
+mongo_uri = <MongoDB 的 URI>
+db_name = 'company' # 連接的DB名稱
+collection_name = "TWN/AINVFQ1" # 連接的collection對象
+
+fetcher = FinanceReportFetcher(
+    mongo_uri = mongo_uri,
+    db_name = db_name,
+    collection_name = collection_name
+)
+
+data = fetcher.get(
+    ticker = "2330" # 任意的股票代碼
+    fetch_mode = fetcher.FetchMode.QOQ_NOCAL # 取得模式
+    start_date = "2005-01-01",
+    end_date = "2024-12-31",
+    report_type = "Q",
+    indexes = []
+) # -> pd.DataFrame or Dict[pd.DataFrame] 
+```
+- `ticker`: 股票代碼
+
+- `fetch_mode` : 取得模式，為`fetcher.YOY_NOCAL` 或 `fetcher.QOQ_NOCAL`
+    - `YOY_NOCAL`: 以end_date為準，取得與end_date同季的歷年資料，時間範圍以start_date為起始
+        > 例如`start_date = "2020-07-01"`, `end_date = "2024-01-01"`，會回傳2020~2024的第一季資料
+
+    - `QOQ_NOCAL`: 時間範圍內的每季資料
+
+    - `QOQ`: 時間範圍內每季的每個index的數值以及QoQ
+
+    - `YoY`: 以end_date為準，取得與end_date同季的歷年資料以及成長率，時間範圍以start_date為起始
+
+- `start_date`: 開始日期，不設定時預設為`2005-01-01`
+
+- `end_date`: 結束日期，不設定時預設為資料庫最新資料的日期
+
+- `report_type`: 選擇哪種報告，預設為`Q`
+    - `A`: 當年累計
+    - `Q`: 當季數值
+    - `TTM`: 移動四季 (包括當季在內，往前累計四個季度)
+
+- `indexes`: 選擇的column，需要以TEJ提供的欄位名稱為準，不提供時或提供`[]`會回傳全部column
+   - 範例輸入: `['bp41', 'bp51']`
+
+[TEJ資料集連結](https://tquant.tejwin.com/%E8%B3%87%E6%96%99%E9%9B%86/)
+請看 `會計師簽證財務資料`
+
+#### 回傳資料
+##### `YOY_NOCAL` 與 `QOQ_NOCAL` 
+為回傳`pd.DataFrame`，column名稱為<年份>Q<季>， row名稱為指定財報項目
+```Python
+# fetch_mode = fetcher.FetchMode.QOQ_NOCAL
+        2024Q3        2024Q2        2024Q1
+bp41  7.082005e+07  6.394707e+07  5.761001e+07
+bp51  3.111298e+09  3.145373e+09  3.091985e+09
+
+# fetch_mode = fetcher.FetchMode.YOY_NOCAL
+        2024Q3        2023Q3        2022Q3
+bp41  7.082005e+07  5.377231e+07  6.201822e+07
+bp51  3.111298e+09  3.173919e+09  2.453840e+09
+```
+
+##### `YOY` 與 `QOQ`
+回傳為`Dict[pd.DataFrame]`, key 為指定的index, DataFrame中則是該index歷年的數值與成長率
+```Python
+# fetch_mode = fetcher.FetchMode.QOQ
+{
+        'bp41':               
+        2024Q3        2024Q2        2024Q1
+value   7.082005e+07  6.394707e+07  5.761001e+07
+growth  1.074791e-01  1.099994e-01  5.532101e-03, 
+        'bp51':               
+        2024Q3        2024Q2        2024Q1
+value   3.111298e+09  3.145373e+09  3.091985e+09
+growth -1.083335e-02  1.726663e-02 -4.159542e-03
+}
+
+# fetch_mode = fetcher.FetchMode.YOY
+{
+'bp41':               
+        2024Q3        2023Q3        2022Q3
+value   7.082005e+07  5.377231e+07  6.201822e+07
+YoY_1            NaN           NaN  4.130744e-01
+YoY_3   1.729171e-01  9.556684e-02  1.883274e-01
+YoY_5   1.389090e-01  1.215242e-01  1.642914e-01
+YoY_10  1.255138e-01  1.356297e-01  1.559702e-01, 
+'bp51':               
+        2024Q3        2023Q3        2022Q3
+value   3.111298e+09  3.173919e+09  2.453840e+09
+YoY_1            NaN           NaN  3.179539e-01
+YoY_3   1.866752e-01  2.766851e-01  2.638677e-01
+YoY_5   2.068132e-01  2.479698e-01  1.815106e-01
+YoY_10  1.420500e-01  1.586797e-01  1.551364e-01
+}
+```
+
+
 ## 版本紀錄
+## 0.0.16
+- 處理ValueFetcher的error #issue76
+
+- tej_fetcher新增 QOQ, YOY功能
+
+## 0.0.15
+- TechFetcher中新增指數條件
+
+- 新增tej_fetcher索取TEJ相關的資料
+
+- package新增depensnecy，可以安裝需要的相關package
+
 ## 0.0.14
 - 修改部分財報資料錯誤的乘以1000的問題
 

neurostats_api-0.0.14/PKG-INFO → neurostats_api-0.0.16/README.md

@@ -1,13 +1,3 @@
-Metadata-Version: 2.1
-Name: neurostats_API
-Version: 0.0.14
-Summary: The service of NeuroStats website
-Home-page: https://github.com/NeurowattStats/NeuroStats_API.git
-Author: JasonWang@Neurowatt
-Author-email: jason@neurowatt.ai
-Requires-Python: >=3.6
-Description-Content-Type: text/markdown
-
 # neurostats_API
 
 - [檔案架構](#檔案架構)
@@ -83,7 +73,7 @@ pip install neurostats-API
 ```Python 
 >>> import neurostats_API
 >>> print(neurostats_API.__version__)
-0.0.14
+0.0.16
 ```
 
 ### 得到最新一期的評價資料與歷年評價
@@ -667,7 +657,121 @@ fetcher.query()
 
 請注意`range`, `last_range`, `52week_range`這三個項目型態為字串，其餘為float
 
+
+## TEJ 相關
+### 會計師簽證財務資料
+```Python
+from neurostats_API import FinanceReportFetcher
+
+mongo_uri = <MongoDB 的 URI>
+db_name = 'company' # 連接的DB名稱
+collection_name = "TWN/AINVFQ1" # 連接的collection對象
+
+fetcher = FinanceReportFetcher(
+    mongo_uri = mongo_uri,
+    db_name = db_name,
+    collection_name = collection_name
+)
+
+data = fetcher.get(
+    ticker = "2330" # 任意的股票代碼
+    fetch_mode = fetcher.FetchMode.QOQ_NOCAL # 取得模式
+    start_date = "2005-01-01",
+    end_date = "2024-12-31",
+    report_type = "Q",
+    indexes = []
+) # -> pd.DataFrame or Dict[pd.DataFrame] 
+```
+- `ticker`: 股票代碼
+
+- `fetch_mode` : 取得模式，為`fetcher.YOY_NOCAL` 或 `fetcher.QOQ_NOCAL`
+    - `YOY_NOCAL`: 以end_date為準，取得與end_date同季的歷年資料，時間範圍以start_date為起始
+        > 例如`start_date = "2020-07-01"`, `end_date = "2024-01-01"`，會回傳2020~2024的第一季資料
+
+    - `QOQ_NOCAL`: 時間範圍內的每季資料
+
+    - `QOQ`: 時間範圍內每季的每個index的數值以及QoQ
+
+    - `YoY`: 以end_date為準，取得與end_date同季的歷年資料以及成長率，時間範圍以start_date為起始
+
+- `start_date`: 開始日期，不設定時預設為`2005-01-01`
+
+- `end_date`: 結束日期，不設定時預設為資料庫最新資料的日期
+
+- `report_type`: 選擇哪種報告，預設為`Q`
+    - `A`: 當年累計
+    - `Q`: 當季數值
+    - `TTM`: 移動四季 (包括當季在內，往前累計四個季度)
+
+- `indexes`: 選擇的column，需要以TEJ提供的欄位名稱為準，不提供時或提供`[]`會回傳全部column
+   - 範例輸入: `['bp41', 'bp51']`
+
+[TEJ資料集連結](https://tquant.tejwin.com/%E8%B3%87%E6%96%99%E9%9B%86/)
+請看 `會計師簽證財務資料`
+
+#### 回傳資料
+##### `YOY_NOCAL` 與 `QOQ_NOCAL` 
+為回傳`pd.DataFrame`，column名稱為<年份>Q<季>， row名稱為指定財報項目
+```Python
+# fetch_mode = fetcher.FetchMode.QOQ_NOCAL
+        2024Q3        2024Q2        2024Q1
+bp41  7.082005e+07  6.394707e+07  5.761001e+07
+bp51  3.111298e+09  3.145373e+09  3.091985e+09
+
+# fetch_mode = fetcher.FetchMode.YOY_NOCAL
+        2024Q3        2023Q3        2022Q3
+bp41  7.082005e+07  5.377231e+07  6.201822e+07
+bp51  3.111298e+09  3.173919e+09  2.453840e+09
+```
+
+##### `YOY` 與 `QOQ`
+回傳為`Dict[pd.DataFrame]`, key 為指定的index, DataFrame中則是該index歷年的數值與成長率
+```Python
+# fetch_mode = fetcher.FetchMode.QOQ
+{
+        'bp41':               
+        2024Q3        2024Q2        2024Q1
+value   7.082005e+07  6.394707e+07  5.761001e+07
+growth  1.074791e-01  1.099994e-01  5.532101e-03, 
+        'bp51':               
+        2024Q3        2024Q2        2024Q1
+value   3.111298e+09  3.145373e+09  3.091985e+09
+growth -1.083335e-02  1.726663e-02 -4.159542e-03
+}
+
+# fetch_mode = fetcher.FetchMode.YOY
+{
+'bp41':               
+        2024Q3        2023Q3        2022Q3
+value   7.082005e+07  5.377231e+07  6.201822e+07
+YoY_1            NaN           NaN  4.130744e-01
+YoY_3   1.729171e-01  9.556684e-02  1.883274e-01
+YoY_5   1.389090e-01  1.215242e-01  1.642914e-01
+YoY_10  1.255138e-01  1.356297e-01  1.559702e-01, 
+'bp51':               
+        2024Q3        2023Q3        2022Q3
+value   3.111298e+09  3.173919e+09  2.453840e+09
+YoY_1            NaN           NaN  3.179539e-01
+YoY_3   1.866752e-01  2.766851e-01  2.638677e-01
+YoY_5   2.068132e-01  2.479698e-01  1.815106e-01
+YoY_10  1.420500e-01  1.586797e-01  1.551364e-01
+}
+```
+
+
 ## 版本紀錄
+## 0.0.16
+- 處理ValueFetcher的error #issue76
+
+- tej_fetcher新增 QOQ, YOY功能
+
+## 0.0.15
+- TechFetcher中新增指數條件
+
+- 新增tej_fetcher索取TEJ相關的資料
+
+- package新增depensnecy，可以安裝需要的相關package
+
 ## 0.0.14
 - 修改部分財報資料錯誤的乘以1000的問題
 
@@ -693,4 +797,4 @@ fetcher.query()
 - 處理銀行公司在finanace_overview會報錯誤的問題(未完全解決，因銀行公司財報有許多名稱不同，目前都會顯示為None)
 
 ### 0.0.9
-- 更新指標的資料型態: 單位為日, %, 倍轉為字串
+- 更新指標的資料型態: 單位為日, %, 倍轉為字串

neurostats_api-0.0.16/neurostats_API/__init__.py

@@ -0,0 +1,13 @@
+__version__='0.0.16'
+
+from .fetchers import (
+    BalanceSheetFetcher,
+    CashFlowFetcher,
+    FinanceOverviewFetcher,
+    FinanceReportFetcher,
+    InstitutionFetcher,
+    MarginTradingFetcher,
+    MonthRevenueFetcher,
+    TechFetcher,
+    ProfitLoseFetcher
+)

{neurostats_api-0.0.14 → neurostats_api-0.0.16}/neurostats_API/fetchers/__init__.py

@@ -2,6 +2,8 @@ from .base import StatsDateTime, StatsFetcher
 from .balance_sheet import BalanceSheetFetcher
 from .cash_flow import CashFlowFetcher
 from .finance_overview import FinanceOverviewFetcher
+from .tej_finance_report import FinanceReportFetcher
+from .tech import TechFetcher
 from .institution import InstitutionFetcher
 from .margin_trading import MarginTradingFetcher
 from .month_revenue import MonthRevenueFetcher

neurostats_api-0.0.16/neurostats_API/fetchers/base.py

@@ -0,0 +1,154 @@
+import abc
+from pymongo import MongoClient
+import pandas as pd
+import json
+import pytz
+from datetime import datetime, timedelta, date
+from ..utils import StatsDateTime, StatsProcessor, YoY_Calculator
+import yaml
+
+
+class StatsFetcher:
+
+    def __init__(self, ticker, db_client):
+        self.ticker = ticker
+        self.db = db_client["company"]    # Replace with your database name
+        self.collection = self.db["twse_stats"]
+
+        self.timezone = pytz.timezone("Asia/Taipei")
+
+        self.target_metric_dict = {
+            'value': ['value'],
+            'value_and_percentage': ['value', 'percentage'],
+            'percentage': ['percentage'],
+            'grand_total': ['grand_total'],
+            'grand_total_values': ['grand_total', 'grand_total_percentage'],
+            'grand_total_percentage': ['grand_total_percentage'],
+            'growth': [f'YoY_{i}' for i in [1, 3, 5, 10]],
+            'grand_total_growth': [f"YoY_{i}" for i in [1, 3, 5, 10]]
+        }
+
+    def prepare_query(self):
+        return [
+            {
+                "$match": {
+                    "ticker": self.ticker,
+                }
+            },
+        ]
+
+    def collect_data(self, start_date, end_date):
+        pipeline = self.prepare_query()
+
+        fetched_data = list(self.collection.aggregate(pipeline))
+
+        return fetched_data[0]
+
+    def str_to_datetime(self, date_str):
+        year, month, day = [int(num) for num in date_str.split("-")]
+
+        date = datetime.strptime(date_str, "%Y-%m-%d")
+        date = self.timezone.localize(date)
+
+        season = (month - 1) // 3 + 1
+
+        return StatsDateTime(date, year, month, day, season)
+
+
+class BaseTEJFetcher(abc.ABC):
+
+    def __init__(self):
+        self.client = None
+        self.db = None
+        self.collection = None
+
+    @abc.abstractmethod
+    def get(self):
+        pass
+
+    def get_latest_data_time(self, ticker):
+        latest_data = self.collection.find_one({"ticker": ticker}, {"last_update": 1, "_id": 0})
+
+        try:
+            latest_date = latest_data['last_update']["latest_data_date"]
+        except Exception as e:
+            latest_date = None
+
+        return latest_date
+
+    def cal_YoY(self, data_dict: dict, start_year: int, end_year: int, season: int):
+        year_shifts = [1, 3, 5, 10]
+        return_dict = {}
+        for year in range(start_year, end_year + 1):
+            year_data = data_dict[f"{year}Q{season}"]
+            year_keys = list(year_data.keys())
+            for key in year_keys:
+                if (key in 'season'):
+                    continue
+
+                if (isinstance(year_data[key], (int, float))):
+                    temp_dict = {"value": year_data[key]}
+
+                    for shift in year_shifts:
+                        this_value = year_data[key]
+                        try:
+                            past_year = str(year - shift)
+                            last_value = data_dict[f"{past_year}Q{season}"][key]
+                            temp_dict[f"YoY_{shift}"] = YoY_Calculator.cal_growth(this_value, last_value, delta=shift)
+                        except Exception as e:
+                            temp_dict[f"YoY_{shift}"] = None
+
+                    year_data[key] = temp_dict
+
+                else:
+                    year_data.pop(key)
+
+            return_dict[f"{year}Q{season}"] = year_data
+
+        return return_dict
+
+    def cal_QoQ(self, data_dict):
+        return_dict = {}
+        for i, time_index in enumerate(data_dict.keys()):
+            year, season = time_index.split("Q")
+            year = int(year)
+            season = int(season)
+            if (season == 1):
+                last_year = year - 1
+                last_season = 4
+            else:
+                last_year = year
+                last_season = season - 1
+
+            this_data = data_dict[time_index]
+            this_keys = list(this_data.keys())
+            for key in this_keys:
+                if (key in 'season'):
+                    continue
+
+                this_value = this_data[key]
+
+                if (isinstance(this_value, (int, float))):
+                    temp_dict = {"value": this_value}
+
+                    try:
+                        last_value = data_dict[f"{last_year}Q{last_season}"][key]['value']
+
+                        temp_dict['growth'] = YoY_Calculator.cal_growth(this_value, last_value, delta=1)
+                    except Exception as e:
+                        temp_dict['growth'] = None
+
+                    this_data[key] = temp_dict
+
+                else:
+                    this_data.pop(key)
+            return_dict[time_index] = this_data
+        return return_dict
+
+    def get_dict_of_df(self, data_dict):
+        """
+        dict[dict] -> dict[df]
+        """
+        for key in data_dict.keys():
+            data_dict[key] = pd.DataFrame.from_dict(data_dict[key])
+        return data_dict

{neurostats_api-0.0.14 → neurostats_api-0.0.16}/neurostats_API/fetchers/tech.py

@@ -1,9 +1,16 @@
 from .base import StatsFetcher
 import pandas as pd
+import yfinance as yf
 
 class TechFetcher(StatsFetcher):
     
     def __init__(self, ticker:str, db_client):
+
+        """
+        The Capitalization-Weighted Index includes the following tickers:
+        ['GSPC', 'IXIC', 'DJI', 'TWII']   
+        """
+        
         super().__init__(ticker, db_client)
         self.full_ohlcv = self._get_ohlcv()
         self.basic_indexes = ['SMA5', 'SMA20', 'SMA60', 'EMA5', 'EMA20', 
@@ -40,16 +47,36 @@ class TechFetcher(StatsFetcher):
         )
 
     def _get_ohlcv(self):
-        query = {'ticker': self.ticker}
-        ticker_full = list(self.collection.find(query))
+        
+        if self.ticker in ['GSPC', 'IXIC', 'DJI', 'TWII']:
+            
+            full_tick = f'^{self.ticker}'
+            yf_ticker = yf.Ticker(full_tick)
+            origin_df = yf_ticker.history(period="10y")
+            origin_df = origin_df.reset_index()
+            origin_df["Date"] = pd.to_datetime(origin_df["Date"]).dt.date
+            df = origin_df.rename(
+                columns={
+                    "Date": "date",
+                    "Open": "open",
+                    "High": "high",
+                    "Low": "low",
+                    "Close": "close",
+                    "Volume": "volume"
+                }
+            )
+        else:
+
+            query = {'ticker': self.ticker}
+            ticker_full = list(self.collection.find(query))
 
-        if not ticker_full:
-            raise ValueError(f"No data found for ticker: {self.ticker}")
+            if not ticker_full:
+                raise ValueError(f"No data found for ticker: {self.ticker}")
 
-        if 'daily_data' not in ticker_full[0] or ticker_full[0]['daily_data'] is None:
-            raise KeyError("Missing 'daily_data' in the retrieved data")
+            if 'daily_data' not in ticker_full[0] or ticker_full[0]['daily_data'] is None:
+                raise KeyError("Missing 'daily_data' in the retrieved data")
 
-        df = pd.DataFrame(ticker_full[0]['daily_data'])
+            df = pd.DataFrame(ticker_full[0]['daily_data'])
 
         selected_cols = ['date','open','high','low','close','volume']