logdetective 0.2.0__tar.gz → 0.2.2__tar.gz

logdetective-0.2.2/PKG-INFO

@@ -0,0 +1,188 @@
+Metadata-Version: 2.1
+Name: logdetective
+Version: 0.2.2
+Summary: Log using LLM AI to search for build/test failures and provide ideas for fixing these.
+License: Apache-2.0
+Author: Jiri Podivin
+Author-email: jpodivin@gmail.com
+Requires-Python: >=3.11,<4.0
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Natural Language :: English
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Internet :: Log Analysis
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Debuggers
+Provides-Extra: server
+Requires-Dist: drain3 (>=0.9.11,<0.10.0)
+Requires-Dist: huggingface-hub (>=0.23.2,<0.24.0)
+Requires-Dist: llama-cpp-python (>=0.2.56,<0.3.0)
+Requires-Dist: requests (>=2.31.0,<3.0.0)
+Project-URL: homepage, https://github.com/fedora-copr/logdetective
+Project-URL: issues, https://github.com/fedora-copr/logdetective/issues
+Description-Content-Type: text/markdown
+
+Log Detective
+=============
+
+A Python tool to analyze logs using a Language Model (LLM) and Drain template miner.
+
+Installation
+------------
+
+The logdetective project is published on the the the the the [Pypi repository](https://pypi.org/project/logdetective/). The `pip` tool can be used for installation.
+
+First, ensure that the necessary dependencies for the `llama-cpp-python` project are installed. For Fedora, install `gcc-c++`:
+
+    # for Fedora it will be:
+    dnf install gcc-c++
+
+**From Pypi repository**
+
+Then, install the `logdetective` project using pip:
+
+    # then install logdetective project
+    pip install logdetective
+
+**Local repository install**
+
+    pip install .
+
+Usage
+-----
+
+To analyze a log file, run the script with the following command line arguments:
+- `url` (required): The URL of the log file to be analyzed.
+- `--model` (optional, default: "Mistral-7B-Instruct-v0.2-GGUF"): The path or URL of the language model for analysis. As we are using LLama.cpp we want this to be in the `gguf` format. You can include the download link to the model here. If the model is already on your machine it will skip the download.
+- `--summarizer` (optional, default: "drain"): Choose between LLM and Drain template miner as the log summarizer. You can also provide the path to an existing language model file instead of using a URL.
+- `--n_lines` (optional, default: 8): The number of lines per chunk for LLM analysis. This only makes sense when you are summarizing with LLM.
+- `--n_clusters` (optional, default 8): Number of clusters for Drain to organize log chunks into. This only makes sense when you are summarizing with Drain
+
+Example usage:
+
+    logdetective https://example.com/logs.txt
+
+Or if the log file is stored locally:
+
+    logdetective ./data/logs.txt
+
+Example you want to use a different model:
+
+    logdetective https://example.com/logs.txt --model https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct.Q5_K_S.gguf?download=true
+
+
+Real Example
+------------
+Let's have a look at a real world example. Log Detective can work with any logs though we optimize it for build logs.
+
+We're going to analyze a failed build of a python-based library that happened in Fedora Koji buildsystem:
+```
+$ logdetective https://kojipkgs.fedoraproject.org//work/tasks/8157/117788157/build.log
+Explanation:
+[Child return code was: 0] : The rpm build process executed successfully without any errors until the 'check' phase.
+
+[wamp/test/test_wamp_component_aio.py::test_asyncio_component] : Pytest found
+two tests marked with '@pytest.mark.asyncio' but they are not async functions.
+This warning can be ignored unless the tests are intended to be run
+asynchronously.
+
+[wamp/test/test_wamp_component_aio.py::test_asyncio_component_404] : Another
+Pytest warning for the same issue as test_asyncio_component.
+
+[-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html] :
+This line is not related to the error, but it is a reminder to refer to Pytest
+documentation for handling warnings.
+
+[=========================== short test summary info
+============================] : This section shows the summary of tests that
+were executed. It shows the number of tests passed, failed, skipped,
+deselected, and warnings.
+
+[FAILED wamp/test/test_wamp_cryptosign.py::TestSigVectors::test_vectors] : A
+failed test is reported with the name of the test file, the name of the test
+method, and the name of the test case that failed. In this case,
+TestSigVectors::test_vectors failed.
+
+[FAILED
+websocket/test/test_websocket_protocol.py::WebSocketClientProtocolTests::test_auto_ping]
+: Another failed test is reported with the same format as the previous test. In
+this case, it is WebSocketClientProtocolTests::test_auto_ping that failed.
+
+[FAILED websocket/test/test_websocket_protocol.py::WebSocketServerProtocolTests::test_interpolate_server_status_template]
+: A third failed test is reported with the same format as the previous tests.
+In this case, it is
+WebSocketServerProtocolTests::test_interpolate_server_status_template that
+failed.
+
+[FAILED websocket/test/test_websocket_protocol.py::WebSocketServerProtocolTests::test_sendClose_reason_with_no_code]
+: Another failed test is reported. This time it is
+WebSocketServerProtocolTests::test_sendClose_reason_with_no_code.
+
+[FAILED websocket/test/test_websocket_protocol.py::WebSocketServerProtocolTests::test_sendClose_str_reason]
+: Another failed test is reported with the same test file and test method name,
+but a different test case name: test_sendClose_str_reason.
+
+[==== 13 failed, 195 passed, 64 skipped, 13 deselected, 2 warnings in 6.55s
+=====] : This is the summary of all tests that were executed, including the
+number of tests that passed, failed, were skipped, deselected, or produced
+warnings. In this case, there were 13 failed tests among a total of 211 tests.
+
+[error: Bad exit status from /var/tmp/rpm-tmp.8C0L25 (%check)] : An error
+message is reported indicating that the 'check' phase of the rpm build process
+failed with a bad exit status.
+```
+
+It looks like a wall of text. Similar to any log. The main difference is that here we have the most significant lines of a logfile wrapped in `[ ] : ` and followed by textual explanation of the log text done by mistral 7b.
+
+
+Contributing
+------------
+
+Contributions are welcome! Please submit a pull request if you have any improvements or new features to add. Make sure your changes pass all existing tests before submitting.
+
+To develop logdetective, you should fork this repository, clone your fork, and install dependencies using pip:
+
+    git clone https://github.com/yourusername/logdetective.git
+    cd logdetective
+    pip install .
+
+Make changes to the code as needed and run pre-commit.
+
+Tests
+-----
+
+The [tox](https://github.com/tox-dev/tox) is used to manage tests. Please install `tox` package into your distribution and run:
+
+    tox
+
+This will create a virtual environment with dependencies and run all the tests. For more information follow the tox help.
+
+To run only a specific test execute this:
+
+    tox run -e style # to run flake8
+
+or
+
+    tox run -e lint # to run pylint
+
+Server
+------
+
+FastApi based server is implemented in `logdetective/server.py`. In order to run in a development mode,
+simply start llama-cpp-python server with your chosen model as described in llama-cpp-python [docs](https://llama-cpp-python.readthedocs.io/en/latest/server/#running-the-server).
+
+Afterwards, start the logdetective server with `fastapi dev logdetective/server.py --port 8080`.
+Requests can then be made with post requests, for example:
+
+    curl --header "Content-Type: application/json" --request POST --data '{"url":"<YOUR_URL_HERE>"}' http://localhost:8080/analyze
+
+
+License
+-------
+
+This project is licensed under the Apache-2.0 License - see the LICENSE file for details.
+

logdetective-0.2.2/README.md

@@ -0,0 +1,159 @@
+Log Detective
+=============
+
+A Python tool to analyze logs using a Language Model (LLM) and Drain template miner.
+
+Installation
+------------
+
+The logdetective project is published on the the the the the [Pypi repository](https://pypi.org/project/logdetective/). The `pip` tool can be used for installation.
+
+First, ensure that the necessary dependencies for the `llama-cpp-python` project are installed. For Fedora, install `gcc-c++`:
+
+    # for Fedora it will be:
+    dnf install gcc-c++
+
+**From Pypi repository**
+
+Then, install the `logdetective` project using pip:
+
+    # then install logdetective project
+    pip install logdetective
+
+**Local repository install**
+
+    pip install .
+
+Usage
+-----
+
+To analyze a log file, run the script with the following command line arguments:
+- `url` (required): The URL of the log file to be analyzed.
+- `--model` (optional, default: "Mistral-7B-Instruct-v0.2-GGUF"): The path or URL of the language model for analysis. As we are using LLama.cpp we want this to be in the `gguf` format. You can include the download link to the model here. If the model is already on your machine it will skip the download.
+- `--summarizer` (optional, default: "drain"): Choose between LLM and Drain template miner as the log summarizer. You can also provide the path to an existing language model file instead of using a URL.
+- `--n_lines` (optional, default: 8): The number of lines per chunk for LLM analysis. This only makes sense when you are summarizing with LLM.
+- `--n_clusters` (optional, default 8): Number of clusters for Drain to organize log chunks into. This only makes sense when you are summarizing with Drain
+
+Example usage:
+
+    logdetective https://example.com/logs.txt
+
+Or if the log file is stored locally:
+
+    logdetective ./data/logs.txt
+
+Example you want to use a different model:
+
+    logdetective https://example.com/logs.txt --model https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct.Q5_K_S.gguf?download=true
+
+
+Real Example
+------------
+Let's have a look at a real world example. Log Detective can work with any logs though we optimize it for build logs.
+
+We're going to analyze a failed build of a python-based library that happened in Fedora Koji buildsystem:
+```
+$ logdetective https://kojipkgs.fedoraproject.org//work/tasks/8157/117788157/build.log
+Explanation:
+[Child return code was: 0] : The rpm build process executed successfully without any errors until the 'check' phase.
+
+[wamp/test/test_wamp_component_aio.py::test_asyncio_component] : Pytest found
+two tests marked with '@pytest.mark.asyncio' but they are not async functions.
+This warning can be ignored unless the tests are intended to be run
+asynchronously.
+
+[wamp/test/test_wamp_component_aio.py::test_asyncio_component_404] : Another
+Pytest warning for the same issue as test_asyncio_component.
+
+[-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html] :
+This line is not related to the error, but it is a reminder to refer to Pytest
+documentation for handling warnings.
+
+[=========================== short test summary info
+============================] : This section shows the summary of tests that
+were executed. It shows the number of tests passed, failed, skipped,
+deselected, and warnings.
+
+[FAILED wamp/test/test_wamp_cryptosign.py::TestSigVectors::test_vectors] : A
+failed test is reported with the name of the test file, the name of the test
+method, and the name of the test case that failed. In this case,
+TestSigVectors::test_vectors failed.
+
+[FAILED
+websocket/test/test_websocket_protocol.py::WebSocketClientProtocolTests::test_auto_ping]
+: Another failed test is reported with the same format as the previous test. In
+this case, it is WebSocketClientProtocolTests::test_auto_ping that failed.
+
+[FAILED websocket/test/test_websocket_protocol.py::WebSocketServerProtocolTests::test_interpolate_server_status_template]
+: A third failed test is reported with the same format as the previous tests.
+In this case, it is
+WebSocketServerProtocolTests::test_interpolate_server_status_template that
+failed.
+
+[FAILED websocket/test/test_websocket_protocol.py::WebSocketServerProtocolTests::test_sendClose_reason_with_no_code]
+: Another failed test is reported. This time it is
+WebSocketServerProtocolTests::test_sendClose_reason_with_no_code.
+
+[FAILED websocket/test/test_websocket_protocol.py::WebSocketServerProtocolTests::test_sendClose_str_reason]
+: Another failed test is reported with the same test file and test method name,
+but a different test case name: test_sendClose_str_reason.
+
+[==== 13 failed, 195 passed, 64 skipped, 13 deselected, 2 warnings in 6.55s
+=====] : This is the summary of all tests that were executed, including the
+number of tests that passed, failed, were skipped, deselected, or produced
+warnings. In this case, there were 13 failed tests among a total of 211 tests.
+
+[error: Bad exit status from /var/tmp/rpm-tmp.8C0L25 (%check)] : An error
+message is reported indicating that the 'check' phase of the rpm build process
+failed with a bad exit status.
+```
+
+It looks like a wall of text. Similar to any log. The main difference is that here we have the most significant lines of a logfile wrapped in `[ ] : ` and followed by textual explanation of the log text done by mistral 7b.
+
+
+Contributing
+------------
+
+Contributions are welcome! Please submit a pull request if you have any improvements or new features to add. Make sure your changes pass all existing tests before submitting.
+
+To develop logdetective, you should fork this repository, clone your fork, and install dependencies using pip:
+
+    git clone https://github.com/yourusername/logdetective.git
+    cd logdetective
+    pip install .
+
+Make changes to the code as needed and run pre-commit.
+
+Tests
+-----
+
+The [tox](https://github.com/tox-dev/tox) is used to manage tests. Please install `tox` package into your distribution and run:
+
+    tox
+
+This will create a virtual environment with dependencies and run all the tests. For more information follow the tox help.
+
+To run only a specific test execute this:
+
+    tox run -e style # to run flake8
+
+or
+
+    tox run -e lint # to run pylint
+
+Server
+------
+
+FastApi based server is implemented in `logdetective/server.py`. In order to run in a development mode,
+simply start llama-cpp-python server with your chosen model as described in llama-cpp-python [docs](https://llama-cpp-python.readthedocs.io/en/latest/server/#running-the-server).
+
+Afterwards, start the logdetective server with `fastapi dev logdetective/server.py --port 8080`.
+Requests can then be made with post requests, for example:
+
+    curl --header "Content-Type: application/json" --request POST --data '{"url":"<YOUR_URL_HERE>"}' http://localhost:8080/analyze
+
+
+License
+-------
+
+This project is licensed under the Apache-2.0 License - see the LICENSE file for details.

logdetective-0.2.2/logdetective/constants.py

@@ -0,0 +1,27 @@
+
+# pylint: disable=line-too-long
+DEFAULT_ADVISOR = "TheBloke/Mistral-7B-Instruct-v0.2-GGUF"
+
+PROMPT_TEMPLATE = """
+Given following log snippets, and nothing else, explain what failure, if any, occured during build of this package.
+
+{}
+
+Analysis of the failure must be in a format of [X] : [Y], where [X] is a log snippet, and [Y] is the explanation.
+
+Finally, drawing on information from all snippets, provide complete explanation of the issue.
+
+Analysis:
+
+"""
+
+SUMMARIZE_PROMPT_TEMPLATE = """
+Does following log contain error or issue?
+
+Log:
+
+{}
+
+Answer:
+
+"""

logdetective-0.2.2/logdetective/extractors.py

@@ -0,0 +1,89 @@
+import os
+import logging
+
+import drain3
+from drain3.template_miner_config import TemplateMinerConfig
+from llama_cpp import Llama, LlamaGrammar
+
+from logdetective.constants import SUMMARIZE_PROMPT_TEMPLATE
+from logdetective.utils import get_chunks
+
+LOG = logging.getLogger("logdetective")
+
+
+class LLMExtractor:
+    """
+    A class that extracts relevant information from logs using a language model.
+    """
+    def __init__(self, model: Llama, n_lines: int = 2):
+        self.model =  model
+        self.n_lines = n_lines
+        self.grammar = LlamaGrammar.from_string(
+            "root ::= (\"Yes\" | \"No\")", verbose=False)
+
+    def __call__(self, log: str, n_lines: int = 2, neighbors: bool = False) -> str:
+        chunks = self.rate_chunks(log)
+        out = self.create_extract(chunks, neighbors)
+        return out
+
+    def rate_chunks(self, log: str) -> list[tuple]:
+        """Scan log by the model and store results.
+
+        :param log: log file content
+        """
+        results = []
+        log_lines = log.split("\n")
+
+        for i in range(0, len(log_lines), self.n_lines):
+            block = '\n'.join(log_lines[i:i + self.n_lines])
+            prompt = SUMMARIZE_PROMPT_TEMPLATE.format(log)
+            out = self.model(prompt, max_tokens=7, grammar=self.grammar)
+            out = f"{out['choices'][0]['text']}\n"
+            results.append((block, out))
+
+        return results
+
+    def create_extract(self, chunks: list[tuple], neighbors: bool = False) -> str:
+        """Extract interesting chunks from the model processing.
+        """
+        interesting = []
+        summary = ""
+        # pylint: disable=consider-using-enumerate
+        for i in range(len(chunks)):
+            if chunks[i][1].startswith("Yes"):
+                interesting.append(i)
+                if neighbors:
+                    interesting.extend([max(i - 1, 0), min(i + 1, len(chunks) - 1)])
+
+        interesting = set(interesting)
+
+        for i in interesting:
+            summary += chunks[i][0] + "\n"
+
+        return summary
+
+
+class DrainExtractor:
+    """A class that extracts information from logs using a template miner algorithm.
+    """
+    def __init__(self, verbose: bool = False, context: bool = False, max_clusters=8):
+        config = TemplateMinerConfig()
+        config.load(f"{os.path.dirname(__file__)}/drain3.ini")
+        config.profiling_enabled = verbose
+        config.drain_max_clusters = max_clusters
+        self.miner = drain3.TemplateMiner(config=config)
+        self.verbose = verbose
+        self.context = context
+
+    def __call__(self, log: str) -> str:
+        out = ""
+        for chunk in get_chunks(log):
+            processed_line = self.miner.add_log_message(chunk)
+            LOG.debug(processed_line)
+        sorted_clusters = sorted(self.miner.drain.clusters, key=lambda it: it.size, reverse=True)
+        for chunk in get_chunks(log):
+            cluster = self.miner.match(chunk, "always")
+            if cluster in sorted_clusters:
+                out += f"{chunk}\n"
+                sorted_clusters.remove(cluster)
+        return out

logdetective-0.2.2/logdetective/logdetective.py

@@ -0,0 +1,76 @@
+import argparse
+import logging
+import sys
+
+from logdetective.constants import DEFAULT_ADVISOR
+from logdetective.utils import process_log, initialize_model, retrieve_log_content
+from logdetective.extractors import LLMExtractor, DrainExtractor
+
+LOG = logging.getLogger("logdetective")
+
+
+def main():
+    """Main execution function."""
+    parser = argparse.ArgumentParser("logdetective")
+    parser.add_argument("file", type=str,
+                        default="", help="The URL or path to the log file to be analyzed.")
+    parser.add_argument("-M", "--model",
+                        help="The path or Hugging Face name of the language model for analysis.",
+                        type=str, default=DEFAULT_ADVISOR)
+    parser.add_argument("-F", "--filename_suffix",
+                        help="Suffix of the model file name to be retrieved from Hugging Face.\
+                            Makes sense only if the model is specified with Hugging Face name.",
+                        default="Q4_K_S.gguf")
+    parser.add_argument("-S", "--summarizer", type=str, default="drain",
+                        help="Choose between LLM and Drain template miner as the log summarizer.\
+                                LLM must be specified as path to a model, URL or local file.")
+    parser.add_argument("-N", "--n_lines", type=int,
+                        default=8, help="The number of lines per chunk for LLM analysis.\
+                            This only makes sense when you are summarizing with LLM.")
+    parser.add_argument("-C", "--n_clusters", type=int, default=8,
+                        help="Number of clusters for Drain to organize log chunks into.\
+                            This only makes sense when you are summarizing with Drain")
+    parser.add_argument("-v", "--verbose", action='count', default=0)
+    parser.add_argument("-q", "--quiet", action='store_true')
+    args = parser.parse_args()
+
+    if args.verbose and args.quiet:
+        sys.stderr.write("Error: --quiet and --verbose is mutually exclusive.\n")
+        sys.exit(2)
+
+    # Logging facility setup
+    log_level = logging.INFO
+    if args.verbose >= 1:
+        log_level = logging.DEBUG
+    if args.quiet:
+        log_level = 0
+
+    logging.basicConfig(stream=sys.stdout)
+    LOG.setLevel(log_level)
+
+    # Primary model initialization
+    model = initialize_model(args.model, filename_suffix=args.filename_suffix,
+                             verbose=args.verbose > 2)
+
+    # Log file summarizer selection and initialization
+    if args.summarizer == "drain":
+        extractor = DrainExtractor(args.verbose > 1, context=True, max_clusters=args.n_clusters)
+    else:
+        summarizer_model = initialize_model(args.summarizer, verbose=args.verbose > 2)
+        extractor = LLMExtractor(summarizer_model, args.verbose > 1)
+
+    LOG.info("Getting summary")
+
+    log = retrieve_log_content(args.file)
+    log_summary = extractor(log)
+
+    ratio = len(log_summary.split('\n')) / len(log.split('\n'))
+    LOG.debug("Log summary: \n %s", log_summary)
+    LOG.info("Compression ratio: %s", ratio)
+
+    LOG.info("Analyzing the text")
+    print(f"Explanation: \n{process_log(log_summary, model)}")
+
+
+if __name__ == "__main__":
+    main()

logdetective-0.2.2/logdetective/server.py

@@ -0,0 +1,56 @@
+import logging
+import os
+import json
+
+from fastapi import FastAPI
+from pydantic import BaseModel
+
+import requests
+
+from logdetective.constants import PROMPT_TEMPLATE
+from logdetective.extractors import DrainExtractor
+
+
+class BuildLog(BaseModel):
+    """Model of data submitted to API.
+    """
+    url: str
+
+LOG = logging.getLogger("logdetective")
+
+app = FastAPI()
+
+LLM_CPP_SERVER_ADDRESS = os.environ.get("LLAMA_CPP_SERVER", " http://localhost")
+LLM_CPP_SERVER_PORT = os.environ.get("LLAMA_CPP_SERVER_PORT", 8000)
+LLM_CPP_SERVER_TIMEOUT = os.environ.get("LLAMA_CPP_SERVER_TIMEOUT", 600)
+LOG_SOURCE_REQUEST_TIMEOUT = os.environ.get("LOG_SOURCE_REQUEST_TIMEOUT", 60)
+
+@app.post("/analyze", )
+async def analyze_log(build_log: BuildLog):
+    """Provide endpoint for log file submission and analysis.
+    Request must be in form {"url":"<YOUR_URL_HERE>"}.
+    """
+    extractor = DrainExtractor(verbose=True, context=True, max_clusters=8)
+
+    LOG.info("Getting summary")
+
+    log = requests.get(build_log.url, timeout=int(LOG_SOURCE_REQUEST_TIMEOUT)).text
+    log_summary = extractor(log)
+
+    ratio = len(log_summary.split('\n')) / len(log.split('\n'))
+    LOG.debug("Log summary: \n %s", log_summary)
+    LOG.info("Compression ratio: %s", ratio)
+
+    LOG.info("Analyzing the text")
+    data = {
+            "prompt": PROMPT_TEMPLATE.format(log_summary),
+            "max_tokens": "0"}
+
+    # Expects llama-cpp server to run on LLM_CPP_SERVER_ADDRESS:LLM_CPP_SERVER_PORT
+    response = requests.post(
+        f"{LLM_CPP_SERVER_ADDRESS}:{LLM_CPP_SERVER_PORT}/v1/completions",
+        headers={"Content-Type":"application/json"},
+        data=json.dumps(data),
+        timeout=int(LLM_CPP_SERVER_TIMEOUT))
+
+    return response.text

logdetective-0.2.2/logdetective/utils.py

@@ -0,0 +1,99 @@
+import logging
+import os
+from urllib.parse import urlparse
+
+import requests
+
+from llama_cpp import Llama
+from logdetective.constants import PROMPT_TEMPLATE
+
+
+LOG = logging.getLogger("logdetective")
+
+
+def chunk_continues(text: str, index: int) -> bool:
+    """Set of heuristics for determining whether or not
+    does the current chunk of log text continue on next line.
+    """
+    conditionals = [
+        lambda i, string: string[i + 1].isspace(),
+        lambda i, string: string[i - 1] == "\\"
+    ]
+
+    for c in conditionals:
+        y = c(index, text)
+        if y:
+            return True
+
+    return False
+
+
+def get_chunks(text: str):
+    """Split log into chunks according to heuristic
+    based on whitespace and backslash presence.
+    """
+    text_len = len(text)
+    i = 0
+    chunk = ""
+    while i < text_len:
+        chunk += text[i]
+        if text[i] == '\n':
+            if i + 1 < text_len and chunk_continues(text, i):
+                i += 1
+                continue
+            yield chunk
+            chunk = ""
+        i += 1
+
+
+def initialize_model(model_pth: str, filename_suffix: str = ".gguf", verbose: bool = False) -> Llama:
+    """Initialize Llama class for inference.
+    Args:
+        model_pth (str): path to gguf model file or Hugging Face name
+        filename_suffix (str): suffix of the model file name to be pulled from Hugging Face
+        verbose (bool): level of verbosity for llamacpp
+    """
+    if os.path.isfile(model_pth):
+        model = Llama(
+            model_path=model_pth,
+            n_ctx=0,  # Maximum context for the model
+            verbose=verbose)
+    else:
+        model = Llama.from_pretrained(
+            model_pth,
+            f"*{filename_suffix}",
+            n_ctx=0,  # Maximum context for the model
+            verbose=verbose)
+
+    return model
+
+
+def process_log(log: str, model: Llama) -> str:
+    """
+    Processes a given log using the provided language model and returns its summary.
+
+    Args:
+        log (str): The input log to be processed.
+        model (Llama): The language model used for processing the log.
+
+    Returns:
+        str: The summary of the given log generated by the language model.
+    """
+    return model(PROMPT_TEMPLATE.format(log), max_tokens=0)["choices"][0]["text"]
+
+def retrieve_log_content(log_path: str) -> str:
+    """Get content of the file on the log_path path."""
+    parsed_url = urlparse(log_path)
+    log = ""
+
+    if not parsed_url.scheme:
+        if not os.path.exists(log_path):
+            raise ValueError(f"Local log {log_path} doesn't exist!")
+
+        with open(log_path, "rt") as f:
+            log = f.read()
+
+    else:
+        log = requests.get(log_path, timeout=60).text
+
+    return log

{logdetective-0.2.0 → logdetective-0.2.2}/pyproject.toml

@@ -1,11 +1,14 @@
 [tool.poetry]
 name = "logdetective"
-version = "0.2.0"
+version = "0.2.2"
 description = "Log using LLM AI to search for build/test failures and provide ideas for fixing these."
 authors = ["Jiri Podivin <jpodivin@gmail.com>"]
 license = "Apache-2.0"
 readme = "README.md"
 include = ["logdetective/drain3.ini"]
+packages = [
+    { include = "logdetective" }
+]
 classifiers = [
     "Development Status :: 4 - Beta",
     "Environment :: Console",
@@ -26,9 +29,8 @@ issues = "https://github.com/fedora-copr/logdetective/issues"
 python = "^3.11"
 requests = "^2.31.0"
 llama-cpp-python = "^0.2.56"
-tiktoken = "^0.6.0"
 drain3 = "^0.9.11"
-progressbar2 = "^4.0.0"
+huggingface-hub = "^0.23.2"
 
 [build-system]
 requires = ["poetry-core"]
@@ -37,6 +39,9 @@ build-backend = "poetry.core.masonry.api"
 [tool.poetry.scripts]
 logdetective = 'logdetective.logdetective:main'
 
+[tool.poetry.extras]
+server = ["fastapi", "pydantic"]
+
 [tool.pylint]
 disable = [
     "inconsistent-return-statements",

logdetective-0.2.0/PKG-INFO

@@ -1,90 +0,0 @@
-Metadata-Version: 2.1
-Name: logdetective
-Version: 0.2.0
-Summary: Log using LLM AI to search for build/test failures and provide ideas for fixing these.
-License: Apache-2.0
-Author: Jiri Podivin
-Author-email: jpodivin@gmail.com
-Requires-Python: >=3.11,<4.0
-Classifier: Development Status :: 4 - Beta
-Classifier: Environment :: Console
-Classifier: Intended Audience :: Developers
-Classifier: License :: OSI Approved :: Apache Software License
-Classifier: Natural Language :: English
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Topic :: Internet :: Log Analysis
-Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
-Classifier: Topic :: Software Development :: Debuggers
-Requires-Dist: drain3 (>=0.9.11,<0.10.0)
-Requires-Dist: llama-cpp-python (>=0.2.56,<0.3.0)
-Requires-Dist: progressbar2 (>=4.0.0,<5.0.0)
-Requires-Dist: requests (>=2.31.0,<3.0.0)
-Requires-Dist: tiktoken (>=0.6.0,<0.7.0)
-Project-URL: homepage, https://github.com/fedora-copr/logdetective
-Project-URL: issues, https://github.com/fedora-copr/logdetective/issues
-Description-Content-Type: text/markdown
-
-Log Detective
-=============
-
-A Python tool to analyze logs using a Language Model (LLM) and Drain template miner.
-
-Installation
-------------
-
-    # optionaly when you prefer system packages
-    dnf install python3-jsonpickle python3-tiktoken
-    # install all remaining packages
-    pip install .
-
-Usage
------
-
-To analyze a log file, run the script with the following command line arguments:
-- `url` (required): The URL of the log file to be analyzed.
-- `--model` (optional, default: "Mistral-7B-Instruct-v0.2-GGUF"): The path or URL of the language model for analysis.
-- `--summarizer` (optional, default: "drain"): Choose between LLM and Drain template miner as the log summarizer. You can also provide the path to an existing language model file instead of using a URL.
-- `--n_lines` (optional, default: 5): The number of lines per chunk for LLM analysis. This only makes sense when you are summarizing with LLM.
-
-Example usage:
-
-    ~/.local/bin/logdetective https://example.com/logs.txt
-
-
-Contributing
-------------
-
-Contributions are welcome! Please submit a pull request if you have any improvements or new features to add. Make sure your changes pass all existing tests before submitting.
-
-To develop logdetective, you should fork this repository, clone your fork, and install dependencies using pip:
-
-    git clone https://github.com/yourusername/logdetective.git
-    cd logdetective
-    pip install .
-
-Make changes to the code as needed and run pre-commit.
-
-Tests
------
-
-The [tox](https://github.com/tox-dev/tox) is used to manage tests. Please install `tox` package into your distribution and run:
-
-    tox
-
-This will create a virtual environment with dependencies and run all the tests. For more information follow the tox help.
-
-To run only a specific test execute this:
-
-    tox run -e style # to run flake8
-
-or
-
-    tox run -e lint # to run pylint
-
-License
--------
-
-This project is licensed under the Apache-2.0 License - see the LICENSE file for details.
-

logdetective-0.2.0/README.md

@@ -1,61 +0,0 @@
-Log Detective
-=============
-
-A Python tool to analyze logs using a Language Model (LLM) and Drain template miner.
-
-Installation
-------------
-
-    # optionaly when you prefer system packages
-    dnf install python3-jsonpickle python3-tiktoken
-    # install all remaining packages
-    pip install .
-
-Usage
------
-
-To analyze a log file, run the script with the following command line arguments:
-- `url` (required): The URL of the log file to be analyzed.
-- `--model` (optional, default: "Mistral-7B-Instruct-v0.2-GGUF"): The path or URL of the language model for analysis.
-- `--summarizer` (optional, default: "drain"): Choose between LLM and Drain template miner as the log summarizer. You can also provide the path to an existing language model file instead of using a URL.
-- `--n_lines` (optional, default: 5): The number of lines per chunk for LLM analysis. This only makes sense when you are summarizing with LLM.
-
-Example usage:
-
-    ~/.local/bin/logdetective https://example.com/logs.txt
-
-
-Contributing
-------------
-
-Contributions are welcome! Please submit a pull request if you have any improvements or new features to add. Make sure your changes pass all existing tests before submitting.
-
-To develop logdetective, you should fork this repository, clone your fork, and install dependencies using pip:
-
-    git clone https://github.com/yourusername/logdetective.git
-    cd logdetective
-    pip install .
-
-Make changes to the code as needed and run pre-commit.
-
-Tests
------
-
-The [tox](https://github.com/tox-dev/tox) is used to manage tests. Please install `tox` package into your distribution and run:
-
-    tox
-
-This will create a virtual environment with dependencies and run all the tests. For more information follow the tox help.
-
-To run only a specific test execute this:
-
-    tox run -e style # to run flake8
-
-or
-
-    tox run -e lint # to run pylint
-
-License
--------
-
-This project is licensed under the Apache-2.0 License - see the LICENSE file for details.

logdetective-0.2.0/logdetective/logdetective.py

@@ -1,275 +0,0 @@
-import argparse
-import logging
-import os
-import sys
-from urllib.request import urlretrieve
-
-import drain3
-import numpy as np
-import progressbar
-import requests
-from drain3.template_miner_config import TemplateMinerConfig
-from llama_cpp import Llama, LlamaGrammar
-
-# pylint: disable=line-too-long
-DEFAULT_ADVISOR = "https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF/resolve/main/mistral-7b-instruct-v0.2.Q4_K_S.gguf?download=true"
-
-# pylint: disable=line-too-long
-DEFAULT_LLM_RATER = "https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/tinyllama-1.1b-chat-v1.0.Q4_K_S.gguf?download=true"
-
-PROMPT_TEMPLATE = """
-Given following log snippets, and nothing else, explain what failure, if any occured during build of this package.
-Ignore strings wrapped in <: :>, such as <:*:>.
-
-{}
-
-Analysis of the failure must be in a format of [X] : [Y], where [X] is a log snippet, and [Y] is the explanation.
-
-Finally, drawing on information from all snippets, provide complete explanation of the issue.
-
-Analysis:
-
-"""
-
-SUMMARIZE_PROPT_TEMPLATE = """
-Does following log contain error or issue?
-
-Log:
-
-{}
-
-Answer:
-
-"""
-
-CACHE_LOC = "~/.cache/logdetective/"
-
-LOG = logging.getLogger("logdetective")
-
-
-class MyProgressBar():
-    """Show progress when downloading model."""
-    def __init__(self):
-        self.pbar = None
-
-    def __call__(self, block_num, block_size, total_size):
-        if not self.pbar:
-            self.pbar = progressbar.ProgressBar(maxval=total_size)
-            self.pbar.start()
-
-        downloaded = block_num * block_size
-        if downloaded < total_size:
-            self.pbar.update(downloaded)
-        else:
-            self.pbar.finish()
-
-
-def chunk_continues(text: str, index: int) -> bool:
-    """Set of heuristics for determining whether or not
-    does the current chunk of log text continue on next line.
-    """
-    conditionals = [
-        lambda i, string: string[i + 1].isspace(),
-        lambda i, string: string[i - 1] == "\\"
-    ]
-
-    for c in conditionals:
-        y = c(index, text)
-        if y:
-            return True
-
-    return False
-
-
-def get_chunks(text: str):
-    """Split log into chunks according to heuristic
-    based on whitespace and backslash presence.
-    """
-    text_len = len(text)
-    i = 0
-    chunk = ""
-    while i < text_len:
-        chunk += text[i]
-        if text[i] == '\n':
-            if i + 1 < text_len and chunk_continues(text, i):
-                i += 1
-                continue
-            yield chunk
-            chunk = ""
-        i += 1
-
-
-class LLMExtractor:
-    """
-    A class that extracts relevant information from logs using a language model.
-    """
-    def __init__(self, model_path: str, verbose: bool):
-        self.model = Llama(
-            model_path=model_path,
-            n_ctx=0,
-            verbose=verbose)
-        self.grammar = LlamaGrammar.from_string(
-            "root ::= (\"Yes\" | \"No\")", verbose=False)
-
-    def __call__(self, log: str, n_lines: int = 2, neighbors: bool = False) -> str:
-        chunks = self.rate_chunks(log, n_lines)
-        out = self.create_extract(chunks, neighbors)
-        return out
-
-    def rate_chunks(self, log: str, n_lines: int = 2) -> list[tuple]:
-        """Scan log by the model and store results.
-
-        :param log: log file content
-        :param n_lines: How many lines should the model take into consideration
-        """
-        results = []
-        log_lines = log.split("\n")
-
-        for i in range(0, len(log_lines), n_lines):
-            block = '\n'.join(log_lines[i:i + n_lines])
-            prompt = SUMMARIZE_PROPT_TEMPLATE.format(log)
-            out = self.model(prompt, max_tokens=7, grammar=self.grammar)
-            out = f"{out['choices'][0]['text']}\n"
-            results.append((block, out))
-
-        return results
-
-    def create_extract(self, chunks: list[tuple], neighbors: bool = False) -> str:
-        """Extract interesting chunks from the model processing.
-        """
-        interesting = []
-        summary = ""
-        # pylint: disable=consider-using-enumerate
-        for i in range(len(chunks)):
-            if chunks[i][1].startswith("Yes"):
-                interesting.append(i)
-                if neighbors:
-                    interesting.extend([max(i - 1, 0), min(i + 1, len(chunks) - 1)])
-
-        interesting = np.unique(interesting)
-
-        for i in interesting:
-            summary += chunks[i][0] + "\n"
-
-        return summary
-
-
-class DrainExtractor:
-    """A class that extracts information from logs using a template miner algorithm.
-    """
-    def __init__(self, verbose: bool = False, context: bool = False):
-        config = TemplateMinerConfig()
-        config.load(f"{os.path.dirname(__file__)}/drain3.ini")
-        config.profiling_enabled = verbose
-        self.miner = drain3.TemplateMiner(config=config)
-        self.verbose = verbose
-        self.context = context
-
-    def __call__(self, log: str) -> str:
-        out = ""
-        for chunk in get_chunks(log):
-            procesed_line = self.miner.add_log_message(chunk)
-            LOG.debug(procesed_line)
-        sorted_clusters = sorted(self.miner.drain.clusters, key=lambda it: it.size, reverse=True)
-        for chunk in get_chunks(log):
-            cluster = self.miner.match(chunk, "always")
-            if cluster in sorted_clusters:
-                out += f"{chunk}\n"
-                sorted_clusters.remove(cluster)
-        return out
-
-
-def download_model(url: str, verbose: bool = False) -> str:
-    """ Downloads a language model from a given URL and saves it to the cache directory.
-
-    Args:
-        url (str): The URL of the language model to be downloaded.
-
-    Returns:
-        str: The local file path of the downloaded language model.
-    """
-    path = os.path.join(
-        os.path.expanduser(CACHE_LOC), url.split('/')[-1])
-
-    LOG.info("Downloading model from %s to %s", url, path)
-    if not os.path.exists(path):
-        if verbose:
-            path, _status = urlretrieve(url, path, MyProgressBar())
-        else:
-            path, _status = urlretrieve(url, path)
-
-    return path
-
-
-def process_log(log: str, model: Llama) -> str:
-    """
-    Processes a given log using the provided language model and returns its summary.
-
-    Args:
-        log (str): The input log to be processed.
-        model (Llama): The language model used for processing the log.
-
-    Returns:
-        str: The summary of the given log generated by the language model.
-    """
-    return model(PROMPT_TEMPLATE.format(log), max_tokens=0)["choices"][0]["text"]
-
-
-def main():
-    """Main execution function."""
-    parser = argparse.ArgumentParser("logdetective")
-    parser.add_argument("url", type=str, default="")
-    parser.add_argument("-M", "--model", type=str, default=DEFAULT_ADVISOR)
-    parser.add_argument("-S", "--summarizer", type=str, default="drain")
-    parser.add_argument("-N", "--n_lines", type=int, default=5)
-    parser.add_argument("-v", "--verbose", action='count', default=0)
-    parser.add_argument("-q", "--quiet", action='store_true')
-
-    args = parser.parse_args()
-
-    if args.verbose and args.quiet:
-        sys.stderr.write("Error: --quiet and --verbose is mutually exclusive.\n")
-        sys.exit(2)
-    log_level = logging.INFO
-    if args.verbose >= 1:
-        log_level = logging.DEBUG
-    if args.quiet:
-        log_level = 0
-    logging.basicConfig(stream=sys.stdout)
-    LOG.setLevel(log_level)
-
-    if not os.path.exists(CACHE_LOC):
-        os.makedirs(os.path.expanduser(CACHE_LOC), exist_ok=True)
-
-    if not os.path.isfile(args.model):
-        model_pth = download_model(args.model, not args.quiet)
-    else:
-        model_pth = args.model
-
-    if args.summarizer == "drain":
-        extractor = DrainExtractor(args.verbose > 1, context=True)
-    elif os.path.isfile(args.summarizer):
-        extractor = LLMExtractor(args.summarizer, args.verbose > 1)
-    else:
-        summarizer_pth = download_model(args.summarizer, not args.quiet)
-        extractor = LLMExtractor(summarizer_pth, args.verbose > 1)
-
-    LOG.info("Getting summary")
-    model = Llama(
-        model_path=model_pth,
-        n_ctx=0,
-        verbose=args.verbose > 2)
-
-    log = requests.get(args.url, timeout=60).text
-    log_summary = extractor(log)
-
-    ratio = len(log_summary.split('\n')) / len(log.split('\n'))
-    LOG.debug("Log summary: \n %s", log_summary)
-    LOG.info("Compression ratio: %s", ratio)
-
-    LOG.info("Analyzing the text")
-    print(f"Explanation: \n{process_log(log_summary, model)}")
-
-
-if __name__ == "__main__":
-    main()