logdetective 2.2.1__tar.gz → 2.3.0__tar.gz

{logdetective-2.2.1 → logdetective-2.3.0}/PKG-INFO

@@ -1,8 +1,9 @@
-Metadata-Version: 2.3
+Metadata-Version: 2.4
 Name: logdetective
-Version: 2.2.1
+Version: 2.3.0
 Summary: Log using LLM AI to search for build/test failures and provide ideas for fixing these.
 License: Apache-2.0
+License-File: LICENSE
 Author: Jiri Podivin
 Author-email: jpodivin@gmail.com
 Requires-Python: >=3.11,<4.0
@@ -15,6 +16,7 @@ Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
 Classifier: Topic :: Internet :: Log Analysis
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 Classifier: Topic :: Software Development :: Debuggers
@@ -93,12 +95,13 @@ Usage
 -----
 
 To analyze a log file, run the script with the following command line arguments:
-- `url` (required): The URL of the log file to be analyzed.
-- `--model` (optional, default: "Mistral-7B-Instruct-v0.2-GGUF"): The path or URL of the language model for analysis. As we are using LLama.cpp we want this to be in the `gguf` format. You can include the download link to the model here. If the model is already on your machine it will skip the download.
+- `file` (required): The path or URL of the log file to be analyzed.
+- `--model` (optional, default: "Mistral-7B-Instruct-v0.3-GGUF"): The path or Hugging space name of the language model for analysis. For models from Hugging Face, write them as `namespace/repo_name`. As we are using LLama.cpp we want this to be in the `gguf` format. If the model is already on your machine it will skip the download.
+- `--filename_suffix` (optional, default "Q4_K.gguf"): You can specify which suffix of the file to use. This option is applied when specifying model using the Hugging Face repository.
 - `--summarizer` DISABLED: LLM summarization option was removed. Argument is kept for backward compatibility only.(optional, default: "drain"): Choose between LLM and Drain template miner as the log summarizer. You can also provide the path to an existing language model file instead of using a URL.
 - `--n_lines` DISABLED: LLM summarization option was removed. Argument is kept for backward compatibility only. (optional, default: 8): The number of lines per chunk for LLM analysis. This only makes sense when you are summarizing with LLM.
-- `--n_clusters` (optional, default 8): Number of clusters for Drain to organize log chunks into. This only makes sense when you are summarizing with Drain
-- `--skip_snippets` Path to patterns for skipping snippets.
+- `--n_clusters` (optional, default 8): Number of clusters for Drain to organize log chunks into. This only makes sense when you are summarizing with Drain.
+- `--skip_snippets` Path to patterns for skipping snippets (in YAML).
 
 Example usage:
 
@@ -108,14 +111,10 @@ Or if the log file is stored locally:
 
     logdetective ./data/logs.txt
 
-Example you want to use a different model:
+Examples of using different models. Note the use of `--filename_suffix` (or `-F`) option, useful for models that were quantized:
 
-    logdetective https://example.com/logs.txt --model https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct.Q5_K_S.gguf?download=true
-    logdetective https://example.com/logs.txt --model QuantFactory/Meta-Llama-3-8B-Instruct-GGUF
-
-Example of different suffix (useful for models that were quantized)
-
-    logdetective https://kojipkgs.fedoraproject.org//work/tasks/3367/131313367/build.log --model 'fedora-copr/granite-3.2-8b-instruct-GGUF' -F Q4_K.gguf
+    logdetective https://example.com/logs.txt --model QuantFactory/Meta-Llama-3-8B-Instruct-GGUF --filename_suffix Q5_K_S.gguf
+    logdetective https://kojipkgs.fedoraproject.org//work/tasks/3367/131313367/build.log --model 'fedora-copr/granite-3.2-8b-instruct-GGUF' -F Q4_K_M.gguf
 
 Example of altered prompts:
 
@@ -124,9 +123,9 @@ Example of altered prompts:
      logdetective https://kojipkgs.fedoraproject.org//work/tasks/3367/131313367/build.log --prompts ~/my-prompts.yml
 
 
-Note that streaming with some models (notably Meta-Llama-3 is broken) is broken and can be worked around by `no-stream` option:
+Note that streaming with some models (notably Meta-Llama-3) is broken and can be worked around by `no-stream` option:
 
-    logdetective https://example.com/logs.txt --model QuantFactory/Meta-Llama-3-8B-Instruct-GGUF --no-stream
+    logdetective https://example.com/logs.txt --model QuantFactory/Meta-Llama-3-8B-Instruct-GGUF --filename_suffix Q5_K_M.gguf --no-stream
 
 
 Real Example

{logdetective-2.2.1 → logdetective-2.3.0}/README.md

@@ -41,12 +41,13 @@ Usage
 -----
 
 To analyze a log file, run the script with the following command line arguments:
-- `url` (required): The URL of the log file to be analyzed.
-- `--model` (optional, default: "Mistral-7B-Instruct-v0.2-GGUF"): The path or URL of the language model for analysis. As we are using LLama.cpp we want this to be in the `gguf` format. You can include the download link to the model here. If the model is already on your machine it will skip the download.
+- `file` (required): The path or URL of the log file to be analyzed.
+- `--model` (optional, default: "Mistral-7B-Instruct-v0.3-GGUF"): The path or Hugging space name of the language model for analysis. For models from Hugging Face, write them as `namespace/repo_name`. As we are using LLama.cpp we want this to be in the `gguf` format. If the model is already on your machine it will skip the download.
+- `--filename_suffix` (optional, default "Q4_K.gguf"): You can specify which suffix of the file to use. This option is applied when specifying model using the Hugging Face repository.
 - `--summarizer` DISABLED: LLM summarization option was removed. Argument is kept for backward compatibility only.(optional, default: "drain"): Choose between LLM and Drain template miner as the log summarizer. You can also provide the path to an existing language model file instead of using a URL.
 - `--n_lines` DISABLED: LLM summarization option was removed. Argument is kept for backward compatibility only. (optional, default: 8): The number of lines per chunk for LLM analysis. This only makes sense when you are summarizing with LLM.
-- `--n_clusters` (optional, default 8): Number of clusters for Drain to organize log chunks into. This only makes sense when you are summarizing with Drain
-- `--skip_snippets` Path to patterns for skipping snippets.
+- `--n_clusters` (optional, default 8): Number of clusters for Drain to organize log chunks into. This only makes sense when you are summarizing with Drain.
+- `--skip_snippets` Path to patterns for skipping snippets (in YAML).
 
 Example usage:
 
@@ -56,14 +57,10 @@ Or if the log file is stored locally:
 
     logdetective ./data/logs.txt
 
-Example you want to use a different model:
+Examples of using different models. Note the use of `--filename_suffix` (or `-F`) option, useful for models that were quantized:
 
-    logdetective https://example.com/logs.txt --model https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct.Q5_K_S.gguf?download=true
-    logdetective https://example.com/logs.txt --model QuantFactory/Meta-Llama-3-8B-Instruct-GGUF
-
-Example of different suffix (useful for models that were quantized)
-
-    logdetective https://kojipkgs.fedoraproject.org//work/tasks/3367/131313367/build.log --model 'fedora-copr/granite-3.2-8b-instruct-GGUF' -F Q4_K.gguf
+    logdetective https://example.com/logs.txt --model QuantFactory/Meta-Llama-3-8B-Instruct-GGUF --filename_suffix Q5_K_S.gguf
+    logdetective https://kojipkgs.fedoraproject.org//work/tasks/3367/131313367/build.log --model 'fedora-copr/granite-3.2-8b-instruct-GGUF' -F Q4_K_M.gguf
 
 Example of altered prompts:
 
@@ -72,9 +69,9 @@ Example of altered prompts:
      logdetective https://kojipkgs.fedoraproject.org//work/tasks/3367/131313367/build.log --prompts ~/my-prompts.yml
 
 
-Note that streaming with some models (notably Meta-Llama-3 is broken) is broken and can be worked around by `no-stream` option:
+Note that streaming with some models (notably Meta-Llama-3) is broken and can be worked around by `no-stream` option:
 
-    logdetective https://example.com/logs.txt --model QuantFactory/Meta-Llama-3-8B-Instruct-GGUF --no-stream
+    logdetective https://example.com/logs.txt --model QuantFactory/Meta-Llama-3-8B-Instruct-GGUF --filename_suffix Q5_K_M.gguf --no-stream
 
 
 Real Example

{logdetective-2.2.1 → logdetective-2.3.0}/logdetective/extractors.py

@@ -26,6 +26,9 @@ class Extractor:
         self.skip_snippets = skip_snippets
         self.max_snippet_len = max_snippet_len
 
+        if self.verbose:
+            LOG.setLevel(logging.DEBUG)
+
     def __call__(self, log: str) -> list[Tuple[int, str]]:
         raise NotImplementedError
 

{logdetective-2.2.1 → logdetective-2.3.0}/logdetective/server/gitlab.py

@@ -11,9 +11,13 @@ import gitlab.v4
 import gitlab.v4.objects
 import jinja2
 import aiohttp
+import backoff
 
 from logdetective.server.config import SERVER_CONFIG, LOG
-from logdetective.server.exceptions import LogsTooLargeError
+from logdetective.server.exceptions import (
+    LogsTooLargeError,
+    LogDetectiveConnectionError,
+)
 from logdetective.server.llm import perform_staged_analysis
 from logdetective.server.metric import add_new_metrics, update_metrics
 from logdetective.server.models import (
@@ -29,6 +33,7 @@ from logdetective.server.database.models import (
     GitlabMergeRequestJobs,
 )
 from logdetective.server.compressors import RemoteLogCompressor
+from logdetective.server.utils import connection_error_giveup
 
 MR_REGEX = re.compile(r"refs/merge-requests/(\d+)/.*$")
 FAILURE_LOG_REGEX = re.compile(r"(\w*\.log)")
@@ -91,8 +96,8 @@ async def process_gitlab_job_event(
         log_url, preprocessed_log = await retrieve_and_preprocess_koji_logs(
             gitlab_cfg, job
         )
-    except LogsTooLargeError:
-        LOG.error("Could not retrieve logs. Too large.")
+    except (LogsTooLargeError, LogDetectiveConnectionError) as ex:
+        LOG.error("Could not retrieve logs due to %s", ex)
         raise
 
     # Submit log to Log Detective and await the results.
@@ -151,6 +156,9 @@ def is_eligible_package(project_name: str):
     return True
 
 
+@backoff.on_exception(
+    backoff.expo, ConnectionResetError, max_time=60, on_giveup=connection_error_giveup
+)
 async def retrieve_and_preprocess_koji_logs(
     gitlab_cfg: GitLabInstanceConfig,
     job: gitlab.v4.objects.ProjectJob,
@@ -256,8 +264,7 @@ async def retrieve_and_preprocess_koji_logs(
     LOG.debug("Failed architecture: %s", failed_arch)
 
     log_path = failed_arches[failed_arch].as_posix()
-
-    log_url = f"{gitlab_cfg.api_path}/projects/{job.project_id}/jobs/{job.id}/artifacts/{log_path}"  # pylint: disable=line-too-long
+    log_url = f"{gitlab_cfg.url}/{gitlab_cfg.api_path}/projects/{job.project_id}/jobs/{job.id}/artifacts/{log_path}"  # pylint: disable=line-too-long
     LOG.debug("Returning contents of %s%s", gitlab_cfg.url, log_url)
 
     # Return the log as a file-like object with .read() function

{logdetective-2.2.1 → logdetective-2.3.0}/logdetective/server/models.py

@@ -307,6 +307,7 @@ class GitLabInstanceConfig(BaseModel):  # pylint: disable=too-many-instance-attr
 
     name: str = None
     url: str = None
+    # Path to API of the gitlab instance, assuming `url` as prefix.
     api_path: str = None
     api_token: str = None
 

logdetective-2.3.0/logdetective.1.asciidoc

@@ -0,0 +1,96 @@
+= logdetective(1)
+:doctype: manpage
+:man source: logdetective 1.0
+:man manual: User Commands
+
+== NAME
+
+logdetective - Analyze and summarize log files using LLM and Drain templates
+
+== SYNOPSIS
+
+*logdetective* [_OPTIONS_] *file*
+
+== DESCRIPTION
+
+*logdetective* is a tool that analyzes log files with a LLM using the Drain log template miner. It can consume logs from a local path or a URL, summarize them, and cluster them for easier inspection.
+
+== POSITIONAL ARGUMENTS
+
+*file*::
+  The URL or path to the log file to be analyzed.
+
+== OPTIONS
+
+*-h*, *--help*::
+  Show usage description and exit.
+
+*-M* *MODEL*, *--model* *MODEL*::
+  The path to the language model for analysis (if stored locally). You can also specify the model by name based on the repo on Hugging face (see Examples). Repo id must be in the form `'namespace/repo_name'`. As we are using `LLama.cpp` we want this to be in the gguf format. If the model is already on your machine it will skip the download. (optional, default: "fedora-copr/Mistral-7B-Instruct-v0.3-GGUF")
+
+*-F* *FILENAME_SUFFIX*, *--filename_suffix* *FILENAME_SUFFIX*::
+  Define the suffix of the model file name to retrieve from Hugging Face. This option only applies when the model is specified by its Hugging face repo name, and not its path. (default `Q4_K.gguf`)
+
+*-n*, *--no-stream*::
+  Disable streaming output of analysis results.
+
+*-C* *N_CLUSTERS*, *--n_clusters* *N_CLUSTERS*::
+  Number of clusters to use with the Drain summarizer. Ignored if `LLM` summarizer is selected. (optional, default 8)
+
+*-v*, *--verbose*::
+  Enable verbose output during processing (use `-vv` or `-vvv` for higher levels of verbosity).
+
+*-q*, *--quiet*::
+  Suppress non-essential output.
+
+*--prompts* *PROMPTS_FILE*::
+  Path to prompt configuration file where you can customize (override default) prompts sent to the LLM. See https://github.com/fedora-copr/logdetective/blob/main/logdetective/prompts.yml for reference.
+  +
+  Prompts need to have a form compatible with Python format string syntax (see https://docs.python.org/3/library/string.html#format-string-syntax) with spaces, or replacement fields marked with curly braces, `{}` left for insertion of snippets. Number of replacement fields in new prompts must be the same as in original, although their position may be different.
+
+*--temperature* *TEMPERATURE*::
+  Temperature for inference. Higher temperature leads to more creative and variable outputs. (default 0.8)
+
+*--csgrep*::
+  Use `csgrep` to process the log before analysis. When working with logs containing messages from GCC, it can be beneficial to employ additional extractor based on `csgrep` tool, to ensure that the messages are kept intact. Since `csgrep` is not available as a Python package, it must be installed separately, with a package manager or from https://github.com/csutils/csdiff.
+
+*--skip_snippets* *SNIPPETS_FILE*::
+  Path to patterns for skipping snippets. User can specify regular expressions matching log chunks (which may not contribute to the analysis of the problem), along with simple description. Patterns to be skipped must be defined in a `yaml` file as a dictionary, where key is a description and value is a regex.
+  +
+  Examples:
+
+    contains_capital_a: "^.*A.*"
+
+    starts_with_numeric: "^[0-9].*"
+
+    child_exit_code_zero: "Child return code was: 0"
+
+== EXAMPLES
+
+Example usage:
+
+  $ logdetective https://example.com/logs.txt
+
+Or if the log file is stored locally:
+
+  $ logdetective ./data/logs.txt
+
+Analyze a local log file using a LLM found locally:
+
+  $ logdetective -M /path/to/llm /var/log/syslog
+
+With specific model from HuggingFace (`namespace/repo_name`, note that `--filename_suffix` is also needed):
+
+  $ logdetective https://example.com/logs.txt --model QuantFactory/Meta-Llama-3-8B-Instruct-GGUF --filename_suffix Q5_K_S.gguf
+
+Cluster logs from a URL (using Drain):
+
+  $ logdetective -C 10 https://example.com/logs.txt
+
+== SEE ALSO
+
+https://logdetective.com
+
+== ADDITIONAL NOTES
+
+Note that *logdetective* works as intended with instruction tuned text generation models in GGUF format.

{logdetective-2.2.1 → logdetective-2.3.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "logdetective"
-version = "2.2.1"
+version = "2.3.0"
 description = "Log using LLM AI to search for build/test failures and provide ideas for fixing these."
 authors = ["Jiri Podivin <jpodivin@gmail.com>"]
 license = "Apache-2.0"

logdetective-2.2.1/logdetective.1.asciidoc

@@ -1,85 +0,0 @@
-= logdetective(1)
-:doctype: manpage
-:man source: logdetective 1.0
-:man manual: User Commands
-
-== NAME
-
-logdetective - Analyze and summarize log files using LLM or Drain templates
-
-== SYNOPSIS
-
-*logdetective* [_OPTIONS_] *file*
-
-== DESCRIPTION
-
-*logdetective* is a tool that analyzes log files using either a large language
-model (LLM) or the Drain log template miner. It can consume logs from a local
-path or a URL, summarize them, and cluster them for easier inspection.
-
-== POSITIONAL ARGUMENTS
-
-*file*::
-  The URL or path to the log file to be analyzed.
-
-== OPTIONS
-
-*-h*, *--help*::
-  Show this help message and exit.
-
-*-M* *MODEL*, *--model* *MODEL*::
-  The path or URL of the language model for analysis. As we are using LLama.cpp we want this to be in the gguf format. You can include the download link to the model here. If the model is already on your machine it will skip the download. (optional, default: "Mistral-7B-Instruct-v0.2-GGUF")
-
-*-F* *FILENAME_SUFFIX*, *--filename_suffix* *FILENAME_SUFFIX*::
-  Define the suffix of the model file name to retrieve from Hugging Face. This option only applies when the model is specified by name (not a path).
-
-*-n*, *--no-stream*::
-  Disable streaming output of analysis results.
-
-*-S* *SUMMARIZER*, *--summarizer* *SUMMARIZER*::
-  Choose between LLM and Drain template miner as the log summarizer. You can also provide the path to an existing language model file instead of using a URL. (optional, default: "drain")
-
-*-N* *N_LINES*, *--n_lines* *N_LINES*::
-  Number of lines per chunk for LLM analysis. Only applicable when `LLM` is used as the summarizer. (optional, default: 8)
-
-*-C* *N_CLUSTERS*, *--n_clusters* *N_CLUSTERS*::
-  Number of clusters to use with the Drain summarizer. Ignored if `LLM` summarizer is selected. (optional, default 8)
-
-*-v*, *--verbose*::
-  Enable verbose output during processing.
-
-*-q*, *--quiet*::
-  Suppress non-essential output.
-
-*--prompts* *PROMPTS*::
-  Path to prompt configuration file where you can customize prompts sent to `LLM`.
-
-*--temperature* *TEMPERATURE*::
-  Temperature for inference.
-
-== EXAMPLES
-
-Example usage:
-
-  $ logdetective https://example.com/logs.txt
-
-Or if the log file is stored locally:
-
-  $ logdetective ./data/logs.txt
-
-Analyze a local log file using an LLM model:
-
-  $ logdetective -M /path/to/llm-model -S LLM -N 100 /var/log/syslog
-
-With specific models:
-
-  $ logdetective https://example.com/logs.txt --model https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct.Q5_K_S.gguf?download=true
-  $ logdetective https://example.com/logs.txt --model QuantFactory/Meta-Llama-3-8B-Instruct-GGUF
-
-Cluster logs from a URL using Drain:
-
-  $ logdetective -S Drain -C 10 https://example.com/logs.txt
-
-== SEE ALSO
-
-https://logdetective.com