javacore-analyser 2.3.0.dev1__tar.gz → 2.3.0.dev2__tar.gz

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/CONTRIBUTING.md

@@ -94,7 +94,7 @@ To run web application:
 
 ## Build pip package 
 Follow the steps from [Packaging projects](https://packaging.python.org/en/latest/tutorials/packaging-projects/).
-Currently Chris has an API keys for test and production pypi
+Currently Chris and Tad have an API keys for test and production pypi
 
 ## Build container localy  
 To build a container:  
@@ -104,6 +104,11 @@ or
 
 `docker build -t javacore-analyser .`
 
+If you want to build a particular version, you need to add `--build-arg version=` argument, for example:
+`podman build --build-arg version='==2.1' -t javacore-analyser .` 
+or 
+`podman build --build-arg version='<2.1' -t javacore-analyser .`  
+
 To start the container:  
 `podman run -it --rm --name javacore-analyser --mount type=bind,src="local-dir-on-fs",target=/reports -p 5001:5000 javacore-analyser`  
 
@@ -115,6 +120,43 @@ or
 `-p` specifies port mapping. The application in container is running on port 5000. You can map it to another port on 
 your machine (5001 in this example).
 
+
+## Publish the container to ghcr.io/ibm/javacore-analyser
+Once you built the container locally, you might need to publish it on ghcr.io/ibm/javacore-analyser.
+Here are the instructions: 
+https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry.  
+Steps:
+1. Generate the token by navigating to https://github.com/settings/tokens/new?scopes=write:packages. 
+   * Select the read:packages scope to download container images and read their metadata.
+   * Select the write:packages scope to download and upload container images and read and write their metadata.
+   * Select the delete:packages scope to delete container images.  
+   NOTE: you can use the same token for multiple pushes until it does not expire.
+2. Tag image:
+   ```commandline
+   podman image tag localhost/javacore-analyser ghcr.io/ibm/javacore-analyser:latest
+   podman image tag localhost/javacore-analyser ghcr.io/ibm/javacore-analyser:2.1
+   ```
+3. Push the image:
+   ```commandline
+   export CR_PAT=<token-id generated in step 1>
+   echo $CR_PAT | podman login ghcr.io -u USERNAME --password-stdin  
+   podman push ghcr.io/ibm/javacore-analyser:2.1
+   podman push ghcr.io/ibm/javacore-analyser:latest
+   ```
+  
+## Build and Publish multiplatform image 
+To publish the image for multiple platforms, follow these instructions:
+https://developers.redhat.com/articles/2023/11/03/how-build-multi-architecture-container-images#benefits_of_multi_architecture_containers
+```commandline
+<Generate token if you do not have it yet>
+export CR_PAT=<token-id generated in step 1>
+echo $CR_PAT | podman login ghcr.io -u USERNAME --password-stdin  
+podman manifest create javacore-analyser:2.1
+podman build --platform linux/amd64,linux/arm64,linux/i386  --manifest javacore-analyser:2.1  .
+podman manifest push javacore-analyser:2.1 docker://ghcr.io/ibm/javacore-analyser:2.1
+podman manifest push javacore-analyser:2.1 docker://ghcr.io/ibm/javacore-analyser:latest
+```
+
 ## Releasing a new version
 Release a new version is partially automated by Travis. Here are the steps:  
 1. Make sure you are on `main` branch in your repository.
@@ -162,6 +204,7 @@ https://github.com/settings/tokens/new
 6. Publish release.
 7. Copy release notes to [CHANGELOG.md](CHANGELOG.md) file.
 
+
 ## Testing
 As default the tests in Pycharm are ran in the current selected directory. However we want to run them in main 
 directory of the tool (**javacore-analyser** directory, not **test** directory). 

javacore_analyser-2.3.0.dev2/Dockerfile

@@ -0,0 +1,20 @@
+#
+# Copyright IBM Corp. 2024 - 2025
+# SPDX-License-Identifier: Apache-2.0
+#
+
+FROM python:3
+
+LABEL org.opencontainers.image.source="https://github.com/IBM/javacore-analyser"
+LABEL org.opencontainers.image.description="This is a tool to analyse IBM Javacore files and provide the report used to analyse hang/outage and performance issues."
+LABEL org.opencontainers.image.licenses="Apache-2.0"
+
+EXPOSE 5000/tcp
+RUN mkdir /reports
+VOLUME ["/reports"]
+
+# As default we do not set the version to have the latest one for build.
+ARG version=""
+#RUN ["pip", "install", "--no-cache-dir", "--root-user-action", "ignore", "javacore-analyser${version}"]
+RUN pip install --no-cache-dir --root-user-action ignore javacore-analyser${version}
+CMD ["javacore_analyser_web", "--port=5000", "--reports-dir=/reports"]

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: javacore_analyser
-Version: 2.3.0.dev1
+Version: 2.3.0.dev2
 Summary: The tool to review IBM Javacore files
 Project-URL: Homepage, https://github.com/IBM/javacore-analyser
 Project-URL: Issues, https://github.com/IBM/javacore-analyser/issues
@@ -312,19 +312,20 @@ You can type the following command to obtain the help:
      or  
   `python -m javacore_analyser web --port=5000 --reports-dir=/data/reports_dir`  
 
+
    The first parameter set the port to use by application. If not specified, 5000 will be used.  
    The second parameter sets where the reports need to be stored. If not set, then the `reports` dir will be created in current location.  
 
 Now you can type (http://localhost:5000/).  
 
 ### Running container image
-There is an unofficial Docker/Podman container managed by one of projects developers. Use the following command 
+There is a Docker/Podman container managed by one of projects developers. Use the following command 
 to start it:
 
-`podman run -it --rm --name javacore-analyser --mount type=bind,src="/local-reports-dir",target=/reports -p 5001:5000 ghcr.io/kkazmierczyk/javacore-analyser:latest`
+`podman run -it --rm --name javacore-analyser --mount type=bind,src="/local-reports-dir",target=/reports -p 5001:5000 ghcr.io/ibm/javacore-analyser:latest`
 
 or  
-`docker run -it --rm --name javacore-analyser --mount type=bind,src="/local-reports-dir",target=/reports -p 5001:5000 ghcr.io/kkazmierczyk/javacore-analyser:latest`  
+`docker run -it --rm --name javacore-analyser --mount type=bind,src="/local-reports-dir",target=/reports -p 5001:5000 ghcr.io/ibm/javacore-analyser:latest`  
 
 The `mount` option specifies where you want locally to store the reports. The reports in the container are stored in 
 `/reports` directory. If you remove mount option, the application will work but the reports will not persist after 

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/README.md

@@ -73,19 +73,20 @@ You can type the following command to obtain the help:
      or  
   `python -m javacore_analyser web --port=5000 --reports-dir=/data/reports_dir`  
 
+
    The first parameter set the port to use by application. If not specified, 5000 will be used.  
    The second parameter sets where the reports need to be stored. If not set, then the `reports` dir will be created in current location.  
 
 Now you can type (http://localhost:5000/).  
 
 ### Running container image
-There is an unofficial Docker/Podman container managed by one of projects developers. Use the following command 
+There is a Docker/Podman container managed by one of projects developers. Use the following command 
 to start it:
 
-`podman run -it --rm --name javacore-analyser --mount type=bind,src="/local-reports-dir",target=/reports -p 5001:5000 ghcr.io/kkazmierczyk/javacore-analyser:latest`
+`podman run -it --rm --name javacore-analyser --mount type=bind,src="/local-reports-dir",target=/reports -p 5001:5000 ghcr.io/ibm/javacore-analyser:latest`
 
 or  
-`docker run -it --rm --name javacore-analyser --mount type=bind,src="/local-reports-dir",target=/reports -p 5001:5000 ghcr.io/kkazmierczyk/javacore-analyser:latest`  
+`docker run -it --rm --name javacore-analyser --mount type=bind,src="/local-reports-dir",target=/reports -p 5001:5000 ghcr.io/ibm/javacore-analyser:latest`  
 
 The `mount` option specifies where you want locally to store the reports. The reports in the container are stored in 
 `/reports` directory. If you remove mount option, the application will work but the reports will not persist after 

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/requirements.txt

@@ -6,4 +6,3 @@ waitress # Production WSGI server
 flask # WSGI server for development the code
 tqdm
 haralyzer
-

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/src/javacore_analyser/constants.py

@@ -1,5 +1,5 @@
 #
-# Copyright IBM Corp. 2024 - 2024
+# Copyright IBM Corp. 2024 - 2025
 # SPDX-License-Identifier: Apache-2.0
 #
 
@@ -24,8 +24,10 @@ UNKNOWN = 'Unknown'
 STACK_TRACE = '4XESTACKTRACE'
 NATIVE_STACK_TRACE = '4XENATIVESTACK'
 DATETIME = '1TIDATETIME'
+STARTTIME = '1CISTARTTIME'
 SIGINFO = '1TISIGINFO'
 ENCODING = '1TICHARSET'
+CMD_LINE = '1CICMDLINE'
 
 DATA_OUTPUT_SUBDIR = '/data/'
 DEFAULT_FILE_DELIMITER = ';'

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/src/javacore_analyser/data/jquery/wait2scripts.js

@@ -260,8 +260,13 @@ const loadChart = function () {
 
   for(let i=1; i<snapshotsNumber; i++){
     let rowEl = document.getElementById('all_threads_table_thread_xsl').rows[i];
-    inputData.push(Number(rowEl.cells[3].innerHTML));
-    labels.push(String(rowEl.cells[0].innerHTML));
+    let value = Number(rowEl.cells[3].innerText);
+
+    // verify the input data
+    if(!isNaN(value)){
+        inputData.push(Number(rowEl.cells[3].innerText));
+        labels.push(String(rowEl.cells[0].innerText));
+    }
   }
 
   new Chart(ctx, {

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/src/javacore_analyser/data/xml/report.xsl

@@ -159,7 +159,7 @@
                     <button data-search="clear">✖</button>
                 </div>
                 <div class="content">
-                <h1>Wait Report</h1>
+                <h1>Javacore Analyser Report</h1>
                 <div class="margined">
                     from data between
                     <b><xsl:value-of select="doc/report_info/javacores_generation_time/starting_time"/></b> and
@@ -383,6 +383,14 @@
                                     <td>Os level</td>
                                     <td class="left"><xsl:value-of select="doc/system_info/os_level"/></td>
                                 </tr>
+                                <tr>
+                                    <td>JVM startup time</td>
+                                    <td class="left"><xsl:value-of select="doc/system_info/jvm_start_time"/></td>
+                                </tr>
+                                 <tr>
+                                    <td>Command line</td>
+                                    <td class="left"><xsl:value-of select="doc/system_info/cmd_line"/></td>
+                                </tr>
                             </tbody>
                         </table>
                         <h4>Java Arguments</h4>
@@ -533,6 +541,7 @@
                         that may be reused for unrelated tasks. Two tasks with different thread names are therefore treated
                         as separate threads for the purpose of this report, even if they are executed in the scope of the same
                         Thread java object.
+                        The address of the java Thread object is included for each thread. This corresponds to the address reported in Java heapdumps.
                         The table can be sorted by clicking on any column header.
                         The following information is displayed for each thread:
                         <ul>
@@ -586,7 +595,9 @@
                                         <a>
                                             <xsl:attribute name="id"><xsl:value-of select="concat('toggle_thread_name',$i)"/></xsl:attribute>
                                             <xsl:attribute name="href"><xsl:value-of select="concat('javascript:expand_stack(stack',$i,',toggle_thread_name',$i,')')"/></xsl:attribute>
-                                            <xsl:attribute name="class">expandit</xsl:attribute><xsl:value-of select="thread_name"/></a>
+                                            <xsl:attribute name="class">expandit</xsl:attribute>
+                                            <xsl:value-of select="thread_name"/>
+                                        </a>
                                         <a class="right" target="_blank">
                                             <xsl:attribute name="href">
                                                 <xsl:value-of select="concat('threads/thread_', thread_hash, '.html')"/>
@@ -596,6 +607,7 @@
                                         <br/>
                                         <div  style="display:none;" >
                                             <xsl:attribute name="id"><xsl:value-of select="concat('stack',$i)"/></xsl:attribute>
+                                            java/lang/Thread:<xsl:value-of select="thread_address"/>
                                             <xsl:for-each select="*[starts-with(name(), 'stack')]">
                                                 <div>
                                                     <xsl:choose>

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/src/javacore_analyser/data/xml/threads/thread.xsl

@@ -33,7 +33,11 @@
                 </div>
                 <div class="content">
                     <p class="right"><a href="../index.html"> Back to Main page </a></p>
-                    <h2>Wait Report for thread: <b><xsl:value-of select="thread_name"/></b></h2>
+                    <h2>
+                        Wait Report for thread: <b><xsl:value-of select="thread_name"/></b>
+                        <br/>
+                        java/lang/Thread:<xsl:value-of select="thread_address"/>
+                    </h2>
                     <xsl:choose>
                         <xsl:when test="//javacore_count = 1">
                             System resource utilization data cannot be calculated with only a single javacore.

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/src/javacore_analyser/java_thread.py

@@ -8,6 +8,14 @@ from javacore_analyser.abstract_snapshot_collection import AbstractSnapshotColle
 
 class Thread(AbstractSnapshotCollection):
 
+    def __init__(self):
+        super().__init__()
+        self.thread_address = ""
+
+    def create(self, thread_snapshot):
+        super().create(thread_snapshot)
+        self.thread_address = thread_snapshot.thread_address
+
     def get_hash(self):
         id_str = self.name + str(self.id)
         hashcode = abs(hash(id_str))
@@ -26,6 +34,11 @@ class Thread(AbstractSnapshotCollection):
         name_node.appendChild(doc.createTextNode(self.name + " (" + str(self.id) + ")"))
         thread_node.appendChild(name_node)
 
+        # thread address
+        thread_address_node = doc.createElement("thread_address")
+        thread_address_node.appendChild(doc.createTextNode(self.thread_address))
+        thread_node.appendChild(thread_address_node)
+
         # hash
         hash_node = doc.createElement("thread_hash")
         hash_node.appendChild(doc.createTextNode(self.get_hash()))

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/src/javacore_analyser/javacore_set.py

@@ -68,6 +68,8 @@ class JavacoreSet:
         self.os_level = ""
         self.architecture = ""
         self.java_version = ""
+        self.jvm_start_time = ""
+        self.cmd_line = ""
         self.user_args = []
         # end of static information
         self.files = []
@@ -191,6 +193,8 @@ class JavacoreSet:
         logging.debug("Architecture: {}".format(self.architecture))
         logging.debug("Java version: {}".format(self.java_version))
         logging.debug("OS Level: {}".format(self.os_level))
+        logging.debug("JVM Startup time: {}".format(self.jvm_start_time))
+        logging.debug("Command line: {}".format(self.cmd_line))
 
     @staticmethod
     def create(path):
@@ -267,6 +271,12 @@ class JavacoreSet:
                 elif line.startswith(JAVA_VERSION):
                     self.java_version = line[len(JAVA_VERSION) + 1:].strip()
                     continue
+                elif line.startswith(STARTTIME):
+                    self.jvm_start_time = line[line.find(":") + 1:].strip()
+                    continue
+                elif line.startswith(CMD_LINE):
+                    self.cmd_line = line[len(CMD_LINE) + 1:].strip()
+                    continue
         except Exception as ex:
             logging.exception(ex)
             logging.error(f'Error during processing file: {file.name} \n'
@@ -472,6 +482,14 @@ class JavacoreSet:
         system_info_node.appendChild(os_level_node)
         os_level_node.appendChild(self.doc.createTextNode(self.os_level))
 
+        jvm_startup_time = self.doc.createElement("jvm_start_time")
+        system_info_node.appendChild(jvm_startup_time)
+        jvm_startup_time.appendChild(self.doc.createTextNode(self.jvm_start_time))
+
+        cmd_line = self.doc.createElement("cmd_line")
+        system_info_node.appendChild(cmd_line)
+        cmd_line.appendChild(self.doc.createTextNode(self.cmd_line))
+
         doc_node.appendChild(self.get_blockers_xml())
         doc_node.appendChild(self.threads.get_xml(self.doc))
         doc_node.appendChild(self.stacks.get_xml(self.doc))

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/src/javacore_analyser/thread_snapshot.py

@@ -21,6 +21,7 @@ class ThreadSnapshot:
         self.allocated_mem = 0
         self.name = None
         self.thread_id = None
+        self.thread_address = None
         self.thread = None
         self.javacore = None
         self.blocker = None
@@ -38,6 +39,7 @@ class ThreadSnapshot:
         snapshot.file = file
         snapshot.javacore = javacore
         snapshot.name = snapshot.get_thread_name(line)
+        snapshot.thread_address = snapshot.get_thread_address(line)
         snapshot.parse_state(line)
         snapshot.parse_snapshot_data()
         return snapshot
@@ -76,6 +78,18 @@ class ThreadSnapshot:
         name = name.translate(str.maketrans({"\01": "[SOH]"}))
         return name
 
+    def get_thread_address(self, line):
+        """ assuming line format:
+        3XMTHREADINFO      "Default Executor-thread-27781" J9VMThread:0x0000000009443300,
+        omrthread_t:0x000000A8B62C3758, java/lang/Thread:0x00000008432D4140, state:B, prio=5 """
+        match = re.search("(java/lang/Thread:)(0x[0-9a-fA-F]+)", line)
+        address = ""
+        if match:   # native threads don't have an address
+            address = match.group(2)
+        if self.javacore:
+            address = self.javacore.encode(address)
+        return address
+
     def get_thread_hash(self):
         if self.thread:
             return self.thread.get_hash()
@@ -198,18 +212,6 @@ class ThreadSnapshot:
         file_name = ""
         if self.file:
             file_name = self.javacore.filename.split(os.sep)[-1].strip()
-        # thread name
-        thread_name_node = doc.createElement("thread_name")
-        thread_name_node.appendChild(doc.createTextNode(self.name))
-        thread_snapshot_node.appendChild(thread_name_node)
-        # thread id
-        thread_name_node = doc.createElement("thread_id")
-        thread_name_node.appendChild(doc.createTextNode(str(self.get_thread_id())))
-        thread_snapshot_node.appendChild(thread_name_node)
-        # thread hash
-        thread_hash_node = doc.createElement("thread_hash")
-        thread_hash_node.appendChild(doc.createTextNode(str(self.get_thread_hash())))
-        thread_snapshot_node.appendChild(thread_hash_node)
         # CPU usage
         cpu_usage_node = doc.createElement("cpu_usage")
         cpu_usage_node.appendChild(doc.createTextNode(str(self.get_cpu_usage_inc())))

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/src/javacore_analyser/tips.py

@@ -73,8 +73,8 @@ class DifferentIssuesTip:
     MAX_INTERVAL_FOR_JAVACORES = 330  # 330 seconds (5 minutes and a little more time)
     DIFFERENT_ISSUES_MESSAGE = """[WARNING] The time interval between javacore {0} and {1} is {2:.0f} seconds, while 
     the recommended maximum interval between two javacores is 300 seconds (5 minutes). It is likely that these
-    two javacores do not correspond to one occurrence of a single issue. Please review the list of javacores received 
-    from the customer and run WAIT2 only against the ones that are applicable for the issue you are investigating. """
+    two javacores do not correspond to one occurrence of a single issue. Please review the list of javacores 
+    and the tool only against the ones that are applicable for the issue you are investigating. """
 
     @staticmethod
     def generate(javacore_set):
@@ -103,7 +103,7 @@ class TooFewJavacoresTip:
 
     MIN_NUMBER_OF_JAVACORES = 10
 
-    ONE_JAVACORE_WARNING = '''[WARNING] You generated this WAIT2 report with only one javacore. 
+    ONE_JAVACORE_WARNING = '''[WARNING] You generated this the report with only one javacore. 
                                 CPU usage calculation is not possible.'''
 
     NOT_ENOUGH_JAVACORES_MESSAGE = """[WARNING] You ran the tool against {0} Javacores. The analysis 

{javacore_analyser-2.3.0.dev1 → javacore_analyser-2.3.0.dev2}/src/javacore_analyser/verbose_gc.py

@@ -1,11 +1,12 @@
 #
-# Copyright IBM Corp. 2024 - 2024
+# Copyright IBM Corp. 2024 - 2025
 # SPDX-License-Identifier: Apache-2.0
 #
 
 import logging
 import ntpath
 from datetime import datetime
+from pathlib import Path
 from xml.dom.minidom import Element, parseString
 
 from tqdm import tqdm
@@ -110,22 +111,11 @@ class VerboseGcFile:
         self.__parse()
 
     def __parse(self):
-        # read in the file as collection of lines
-        file = None
         try:
-            xml_text = ""
-            root_closing_tag_available = False
-            file = open(self.__path, 'r+')
-            lines = file.readlines()
-            # find the last non-empty line
-            for line in lines:
-                line = line.strip()
-                # workaround for https://github.com/eclipse-openj9/openj9/issues/17978
-                line = line.replace("", "?")
-                xml_text = xml_text + line
-                if line == ROOT_CLOSING_TAG: root_closing_tag_available = True
-
-            if not root_closing_tag_available:
+            file = Path(self.__path)
+            xml_text = file.read_text()
+            xml_text = xml_text.replace("", "?")
+            if ROOT_CLOSING_TAG not in xml_text:
                 xml_text = xml_text + ROOT_CLOSING_TAG
                 logging.debug("adding closing tag")
 
@@ -133,8 +123,6 @@ class VerboseGcFile:
             self.__root = self.__doc.documentElement
         except Exception as ex:
             raise GcVerboseProcessingException() from ex
-        finally:
-            if not file.closed: file.close()
 
     def get_file_name(self):
         head, tail = ntpath.split(self.__path)

javacore_analyser-2.3.0.dev1/Dockerfile

@@ -1,15 +0,0 @@
-#
-# Copyright IBM Corp. 2024 - 2024
-# SPDX-License-Identifier: Apache-2.0
-#
-
-FROM python:3
-
-EXPOSE 5000/tcp
-ENV REPORTS_DIR=/reports
-RUN mkdir /reports
-VOLUME ["/reports"]
-
-RUN pip install --no-cache-dir --root-user-action ignore javacore-analyser
-
-CMD [ "javacore_analyser_web" ]