webchanges 3.28.0rc0__tar.gz → 3.28.2__tar.gz

{webchanges-3.28.0rc0/webchanges.egg-info → webchanges-3.28.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: webchanges
-Version: 3.28.0rc0
+Version: 3.28.2
 Summary: Web Changes Delivered. AI-Summarized. Totally Anonymous.
 Author-email: Mike Borsetti <mike+webchanges@borsetti.com>
 Maintainer-email: Mike Borsetti <mike+webchanges@borsetti.com>
@@ -195,7 +195,9 @@ For Generative AI summaries (BETA), you need a free `API Key from Google Cloud A
 
 Installation
 ============
-Install **webchanges** |pypi_version| |format| |status| |security| with:
+|pypi_version| |format| |status| |security|
+
+Install **webchanges**  with:
 
 .. code-block:: bash
 
@@ -203,7 +205,7 @@ Install **webchanges** |pypi_version| |format| |status| |security| with:
 
 Running in Docker
 -----------------
-**webchanges** can easily run in a container; you can find a `Docker <https://www.docker.com/>`__ implementation
+**webchanges** can easily run in a container and you will find a `Docker <https://www.docker.com/>`__ implementation
 `here <https://github.com/yubiuser/webchanges-docker>`__.
 
 
@@ -247,20 +249,19 @@ Schedule
 --------
 **webchanges** leverages the power of a system scheduler:
 
-- On Linux you can use cron, and a tool like `crontab.guru <https://crontab.guru>`__ can build a
-  schedule expression for you (note: see `here <https://www.computerhope.com/unix/ucrontab.htm>`__ if you have never
-  used cron before);
+- On Linux you can use cron, with the help of a tool like `crontab.guru <https://crontab.guru>`__ (help `here
+  <https://www.computerhope.com/unix/ucrontab.htm>`__);
 - On Windows you can use `Windows Task Scheduler <https://en.wikipedia.org/wiki/Windows_Task_Scheduler>`__;
 - On macOS you can use `launchd <https://developer.apple
-  .com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html>`__ (note: see `here
-  <https://launchd.info/>`__ if you have never used launchd before).
+  .com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html>`__ (help `here
+  <https://launchd.info/>`__).
 
 
 Code
 ====
-|coveralls| |issues|
+|coveralls| |issues| |code_style|
 
-The code and issues tracker are hosted on `GitHub <https://github.com/mborsetti/webchanges>`__.
+The code, issues tracker, and discussions are hosted on `GitHub <https://github.com/mborsetti/webchanges>`__.
 
 
 Contributing
@@ -344,7 +345,7 @@ Example enhancements to HTML reporting:
 .. |format| image:: https://img.shields.io/pypi/format/webchanges.svg
     :target: https://pypi.org/project/webchanges/
     :alt: Kit format
-.. |downloads| image:: https://static.pepy.tech/badge/webchanges
+.. |downloads| image:: https://img.shields.io/pypi/dm/webchanges.svg
     :target: https://www.pepy.tech/project/webchanges
     :alt: PyPI downloads
 .. |license| image:: https://img.shields.io/pypi/l/webchanges.svg
@@ -356,15 +357,24 @@ Example enhancements to HTML reporting:
 .. |readthedocs| image:: https://img.shields.io/readthedocs/webchanges/stable.svg?label=
     :target: https://webchanges.readthedocs.io/
     :alt: Documentation status
-.. |CI| image:: https://github.com/mborsetti/webchanges/actions/workflows/ci-cd.yaml/badge.svg?event=push
+.. |old_CI| image:: https://github.com/mborsetti/webchanges/actions/workflows/ci-cd.yaml/badge.svg?event=push
+    :target: https://github.com/mborsetti/webchanges/actions
+    :alt: CI testing status
+.. |CI| image:: https://img.shields.io/github/check-runs/mborsetti/webchanges/main
     :target: https://github.com/mborsetti/webchanges/actions
     :alt: CI testing status
-.. |coveralls| image:: https://coveralls.io/repos/github/mborsetti/webchanges/badge.svg?branch=main
+.. |old_coveralls| image:: https://coveralls.io/repos/github/mborsetti/webchanges/badge.svg?branch=main
+    :target: https://coveralls.io/github/mborsetti/webchanges?branch=main
+    :alt: Code coverage by Coveralls
+.. |coveralls| image:: https://img.shields.io/coverallsCoverage/github/mborsetti/webchanges.svg
     :target: https://coveralls.io/github/mborsetti/webchanges?branch=main
     :alt: Code coverage by Coveralls
+.. |code_style| image:: https://img.shields.io/badge/code%20style-black-000000.svg
+    :target: https://github.com/psf/black
+    :alt: Code style black
 .. |status| image:: https://img.shields.io/pypi/status/webchanges.svg
     :target: https://pypi.org/project/webchanges/
     :alt: Package stability
-.. |security| image:: https://img.shields.io/badge/security-bandit-yellow.svg
+.. |security| image:: https://img.shields.io/badge/security-bandit-green.svg
     :target: https://github.com/PyCQA/bandit
     :alt: Security Status

{webchanges-3.28.0rc0 → webchanges-3.28.2}/README.rst

@@ -29,7 +29,9 @@ For Generative AI summaries (BETA), you need a free `API Key from Google Cloud A
 
 Installation
 ============
-Install **webchanges** |pypi_version| |format| |status| |security| with:
+|pypi_version| |format| |status| |security|
+
+Install **webchanges**  with:
 
 .. code-block:: bash
 
@@ -37,7 +39,7 @@ Install **webchanges** |pypi_version| |format| |status| |security| with:
 
 Running in Docker
 -----------------
-**webchanges** can easily run in a container; you can find a `Docker <https://www.docker.com/>`__ implementation
+**webchanges** can easily run in a container and you will find a `Docker <https://www.docker.com/>`__ implementation
 `here <https://github.com/yubiuser/webchanges-docker>`__.
 
 
@@ -81,20 +83,19 @@ Schedule
 --------
 **webchanges** leverages the power of a system scheduler:
 
-- On Linux you can use cron, and a tool like `crontab.guru <https://crontab.guru>`__ can build a
-  schedule expression for you (note: see `here <https://www.computerhope.com/unix/ucrontab.htm>`__ if you have never
-  used cron before);
+- On Linux you can use cron, with the help of a tool like `crontab.guru <https://crontab.guru>`__ (help `here
+  <https://www.computerhope.com/unix/ucrontab.htm>`__);
 - On Windows you can use `Windows Task Scheduler <https://en.wikipedia.org/wiki/Windows_Task_Scheduler>`__;
 - On macOS you can use `launchd <https://developer.apple
-  .com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html>`__ (note: see `here
-  <https://launchd.info/>`__ if you have never used launchd before).
+  .com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html>`__ (help `here
+  <https://launchd.info/>`__).
 
 
 Code
 ====
-|coveralls| |issues|
+|coveralls| |issues| |code_style|
 
-The code and issues tracker are hosted on `GitHub <https://github.com/mborsetti/webchanges>`__.
+The code, issues tracker, and discussions are hosted on `GitHub <https://github.com/mborsetti/webchanges>`__.
 
 
 Contributing
@@ -178,7 +179,7 @@ Example enhancements to HTML reporting:
 .. |format| image:: https://img.shields.io/pypi/format/webchanges.svg
     :target: https://pypi.org/project/webchanges/
     :alt: Kit format
-.. |downloads| image:: https://static.pepy.tech/badge/webchanges
+.. |downloads| image:: https://img.shields.io/pypi/dm/webchanges.svg
     :target: https://www.pepy.tech/project/webchanges
     :alt: PyPI downloads
 .. |license| image:: https://img.shields.io/pypi/l/webchanges.svg
@@ -190,15 +191,24 @@ Example enhancements to HTML reporting:
 .. |readthedocs| image:: https://img.shields.io/readthedocs/webchanges/stable.svg?label=
     :target: https://webchanges.readthedocs.io/
     :alt: Documentation status
-.. |CI| image:: https://github.com/mborsetti/webchanges/actions/workflows/ci-cd.yaml/badge.svg?event=push
+.. |old_CI| image:: https://github.com/mborsetti/webchanges/actions/workflows/ci-cd.yaml/badge.svg?event=push
+    :target: https://github.com/mborsetti/webchanges/actions
+    :alt: CI testing status
+.. |CI| image:: https://img.shields.io/github/check-runs/mborsetti/webchanges/main
     :target: https://github.com/mborsetti/webchanges/actions
     :alt: CI testing status
-.. |coveralls| image:: https://coveralls.io/repos/github/mborsetti/webchanges/badge.svg?branch=main
+.. |old_coveralls| image:: https://coveralls.io/repos/github/mborsetti/webchanges/badge.svg?branch=main
+    :target: https://coveralls.io/github/mborsetti/webchanges?branch=main
+    :alt: Code coverage by Coveralls
+.. |coveralls| image:: https://img.shields.io/coverallsCoverage/github/mborsetti/webchanges.svg
     :target: https://coveralls.io/github/mborsetti/webchanges?branch=main
     :alt: Code coverage by Coveralls
+.. |code_style| image:: https://img.shields.io/badge/code%20style-black-000000.svg
+    :target: https://github.com/psf/black
+    :alt: Code style black
 .. |status| image:: https://img.shields.io/pypi/status/webchanges.svg
     :target: https://pypi.org/project/webchanges/
     :alt: Package stability
-.. |security| image:: https://img.shields.io/badge/security-bandit-yellow.svg
+.. |security| image:: https://img.shields.io/badge/security-bandit-green.svg
     :target: https://github.com/PyCQA/bandit
     :alt: Security Status

{webchanges-3.28.0rc0 → webchanges-3.28.2}/webchanges/__init__.py

@@ -22,7 +22,7 @@ __project_name__ = __package__
 # * MINOR version when you add functionality in a backwards compatible manner, and
 # * MICRO or PATCH version when you make backwards compatible bug fixes. We no longer use '0'
 # If unsure on increments, use pkg_resources.parse_version to parse
-__version__ = '3.28.0rc0'
+__version__ = '3.28.2'
 __description__ = (
     'Check web (or command output) for changes since last run and notify.\n'
     '\n'

webchanges-3.28.2/webchanges/__main__.py

@@ -0,0 +1,10 @@
+import sys
+from pathlib import Path
+
+parent_dir = Path(__file__).parent.parent
+sys.path.insert(1, str(parent_dir))
+
+if __name__ == '__main__':
+    from cli import main
+
+    main()

{webchanges-3.28.0rc0 → webchanges-3.28.2}/webchanges/cli.py

@@ -244,19 +244,18 @@ def first_run(command_config: CommandConfig) -> None:
 def load_hooks(hooks_file: Path) -> None:
     """Load hooks file."""
     if not hooks_file.is_file():
-        warnings.warn(
-            f'Hooks file not imported because {hooks_file} is not a file',
-            ImportWarning,
-        )
+        # do not use ImportWarning as it could be suppressed
+        warnings.warn(f'Hooks file not imported because {hooks_file} is not a file', RuntimeWarning)
         return
 
     hooks_file_errors = file_ownership_checks(hooks_file)
     if hooks_file_errors:
+        logger.debug('Here should come the warning')
+        # do not use ImportWarning as it could be suppressed
         warnings.warn(
-            f'Hooks file {hooks_file} not imported because '
-            f" {' and '.join(hooks_file_errors)}.\n"
+            f"Hooks file {hooks_file} not not imported because{' and '.join(hooks_file_errors)}.\n"
             f'(see {__docs_url__}en/stable/hooks.html#important-note-for-hooks-file)',
-            ImportWarning,
+            RuntimeWarning,
         )
     else:
         logger.info(f'Importing hooks module from {hooks_file}')
@@ -372,6 +371,10 @@ def main() -> None:  # pragma: no cover
     # Set up the logger to verbose if needed
     setup_logger(command_config.verbose, command_config.log_file)
 
+    # log defaults
+    logger.debug(f'Default config path is {config_path}')
+    logger.debug(f'Default data path is {data_path}')
+
     # For speed, run these here
     handle_unitialized_actions(command_config)
 
@@ -401,6 +404,7 @@ def main() -> None:  # pragma: no cover
 
     # load config (which for syntax checking requires hooks to be loaded too)
     if command_config.hooks_files:
+        logger.debug(f'Hooks files to be loaded: {command_config.hooks_files}')
         for hooks_file in command_config.hooks_files:
             load_hooks(hooks_file)
     config_storage.load()

{webchanges-3.28.0rc0 → webchanges-3.28.2}/webchanges/command.py

@@ -498,8 +498,8 @@ class UrlwatchCommand:
 
     def test_differ(self, arg_test_differ: list[str]) -> int:
         """
-        Runs diffs for a job on all the saved snapshots and outputs the result to stdout or whatever reporter is
-        selected with --test-reporter.
+        Runs diffs for a job on all the saved snapshots and outputs the result to stdout or the reporter selected
+        with --test-reporter.
 
         :param arg_test_differ: Either the job_id or a list containing [job_id, max_diffs]
         :return: 1 if error, 0 if successful.

{webchanges-3.28.0rc0 → webchanges-3.28.2}/webchanges/differs.py

@@ -991,6 +991,7 @@ class ImageDiffer(DifferBase):
         def ai_google(
             old_image: Image.Image,
             new_image: Image.Image,
+            diff_image: Image.Image,
             directives: AiGoogleDirectives,
         ) -> str:
             """Summarize changes in image using Generative AI (ALPHA)."""
@@ -1073,10 +1074,6 @@ class ImageDiffer(DifferBase):
                     }
                 }
 
-            # Create a diff image
-            # diff_image = ImageChops.difference(old_image, new_image)
-            # diff_image.format = new_image.format
-
             # upload to Google
             additional_parts: list[dict[str, dict[str, str]]] = []
             executor = ThreadPoolExecutor()
@@ -1085,7 +1082,7 @@ class ImageDiffer(DifferBase):
                 (
                     ('old image', old_image),
                     ('new image', new_image),
-                    # ('differences image', diff_image),
+                    ('differences image', diff_image),
                 ),
             ):
                 if 'error' not in additional_part:
@@ -1107,22 +1104,33 @@ class ImageDiffer(DifferBase):
                 'focus on the most significant changes.'
             )
             model_prompt = (
+                'You are a skilled visual analyst tasked with analyzing two versions of an image and summarizing the '
+                'key differences between them. The audience for your summary is already familiar with the '
+                "image's content, so you should focus only on the most significant differences.\n\n"
                 '**Instructions:**\n\n'
-                f"1. Compare the new image {additional_parts[1]['file_data']['file_uri']} to the old image "
-                f"{additional_parts[0]['file_data']['file_uri']}, focusing only on major changes.\n"
-                '2. Summarize the meaning of the changes in a clear and concise manner. Be specific.\n'
-                '3. Use Markdown formatting to structure your summary effectively. Use headings, bullet points, and '
-                'other Markdown elements as needed to enhance readability.\n'
-                '4. Restrict your analysis and summary to these two specific images. Do not introduce external '
-                'information or assumptions. Do not introduce knowledge from images you may have already seen.\n'
+                '1. Carefully examine the yellow areas in the image '
+                f"{additional_parts[2]['file_data']['file_uri']}, identify the differences, and describe them.\n"
+                f"2. Refer to the old version of the image {additional_parts[0]['file_data']['file_uri']} and the new "
+                f" version {additional_parts[1]['file_data']['file_uri']}.\n"
+                '3. You are only interested in those differences, such as additions, removals, or alterations, that '
+                'modify the intended message or interpretation.\n'
+                '4. Summarize the identified differences, except those ignored, in a clear and concise manner, '
+                'explaining how the meaning has shifted or evolved in the new version compared to the old version only '
+                'when necessary. Be specific and provide examples to illustrate your points when needed.\n'
+                '5. If there are only additions to the image, then summarize the additions.\n'
+                '6. Use Markdown formatting to structure your summary effectively. Use headings, bullet points, '
+                'and other Markdown elements as needed to enhance readability.\n'
+                '7. Restrict your analysis and summary to the information provided within these images. Do '
+                'not introduce external information or assumptions.\n'
             )
-            summary = AIGoogleDiffer._send_to_model(
+            summary, _ = AIGoogleDiffer._send_to_model(
                 self.job,
                 system_instructions,
                 model_prompt,
                 additional_parts=additional_parts,  # type: ignore[arg-type]
                 directives=directives,
             )
+
             return summary
 
         data_type = directives.get('data_type', 'url')
@@ -1134,8 +1142,8 @@ class ImageDiffer(DifferBase):
         if data_type == 'url':
             old_image = load_image_from_web(self.state.old_data)
             new_image = load_image_from_web(self.state.new_data)
-            old_data = f' (<a href="{self.state.old_data}">Old image</a>)'
-            new_data = f' (<a href="{self.state.new_data}">New image</a>)'
+            old_data = f' (<a href="{self.state.old_data}" target="_blank">Old image</a>)'
+            new_data = f' (<a href="{self.state.new_data}" target="_blank">New image</a>)'
         elif data_type == 'ascii85':
             old_image = load_image_from_ascii85(self.state.old_data)
             new_image = load_image_from_ascii85(self.state.new_data)
@@ -1149,8 +1157,8 @@ class ImageDiffer(DifferBase):
         else:  # 'filename'
             old_image = load_image_from_file(self.state.old_data)
             new_image = load_image_from_file(self.state.new_data)
-            old_data = f' (<a href="file://{self.state.old_data}">Old image</a>)'
-            new_data = f' (<a href="file://{self.state.new_data}">New image</a>)'
+            old_data = f' (<a href="file://{self.state.old_data}" target="_blank">Old image</a>)'
+            new_data = f' (<a href="file://{self.state.new_data}" target="_blank">New image</a>)'
 
         # Check formats  TODO: is it needed? under which circumstances?
         # if new_image.format != old_image.format:
@@ -1202,7 +1210,7 @@ class ImageDiffer(DifferBase):
         # prepare AI summary
         summary = ''
         if 'ai_google' in directives:
-            summary = ai_google(old_image, new_image, directives.get('ai_google', {}))
+            summary = ai_google(old_image, new_image, diff_image, directives.get('ai_google', {}))
 
         # Prepare HTML output
         htm = [
@@ -1246,8 +1254,14 @@ class ImageDiffer(DifferBase):
         )
         footer = f'Summary generated by Google Generative AI (ai_google directive(s): {directives_text})'
         return {
-            'text': f'{summary}\n\n\nA visualization is available in HTML reports.\n------------\n{footer}',
-            'markdown': f'{summary}\n\n\nA visualization is available in HTML reports.\n* * *\n{footer}',
+            'text': (
+                f'{summary}\n\n\nA visualization of differences is available in {__package__} HTML reports.'
+                f'\n------------\n{footer}'
+            ),
+            'markdown': (
+                f'{summary}\n\n\nA visualization of differences is available in {__package__} HTML reports.'
+                f'\n* * *\n{footer}'
+            ),
             'html': '<br>\n'.join(
                 [
                     mark_to_html(summary, extras={'tables'}).replace('<h2>', '<h3>').replace('</h2>', '</h3>'),
@@ -1273,9 +1287,7 @@ class AIGoogleDiffer(DifferBase):
     __kind__ = 'ai_google'
 
     __supported_directives__ = {
-        'model': (
-            'model name from https://ai.google.dev/gemini-api/docs/models/gemini (default: gemini-1.5-flash-latest)'
-        ),
+        'model': ('model name from https://ai.google.dev/gemini-api/docs/models/gemini (default: gemini-2.0-flash)'),
         'system_instructions': (
             'Optional tone and style instructions for the model (default: see documentation at'
             'https://webchanges.readthedocs.io/en/stable/differs.html#ai-google-diff)'
@@ -1300,20 +1312,17 @@ class AIGoogleDiffer(DifferBase):
         model_prompt: str,
         additional_parts: list[dict[str, str | dict[str, str]]] | None = None,
         directives: AiGoogleDirectives | None = None,
-    ) -> str:
-        """Creates the summary request to the model"""
+    ) -> tuple[str, str]:
+        """Creates the summary request to the model; returns the summary and the version of the actual model used."""
         api_version = '1beta'
         if directives is None:
             directives = {}
-        if 'model' not in directives:
-            directives['model'] = 'gemini-1.5-pro'  # also for footer
-        model = directives.get('model')
+        model = directives.get('model', 'gemini-2.0-flash')
         timeout = directives.get('timeout', 300)
         max_output_tokens = directives.get('max_output_tokens')
         temperature = directives.get('temperature', 0.0)
-        top_p = directives.get('top_p')
+        top_p = directives.get('top_p', 1.0 if temperature == 0.0 else None)
         top_k = directives.get('top_k')
-
         GOOGLE_AI_API_KEY = os.environ.get('GOOGLE_AI_API_KEY', '').rstrip()
         if len(GOOGLE_AI_API_KEY) != 39:
             logger.error(
@@ -1323,7 +1332,8 @@ class AIGoogleDiffer(DifferBase):
             return (
                 f'## ERROR in summarizing changes using Google AI:\n'
                 f'Environment variable GOOGLE_AI_API_KEY not found or is of the incorrect length '
-                f'{len(GOOGLE_AI_API_KEY)}.\n'
+                f'{len(GOOGLE_AI_API_KEY)}.\n',
+                '',
             )
 
         data: dict[str, Any] = {
@@ -1341,6 +1351,7 @@ class AIGoogleDiffer(DifferBase):
         if directives.get('tools'):
             data['tools'] = directives['tools']
         logger.info(f'Job {job.index_number}: Making the content generation request to Google AI model {model}')
+        model_version = model  # default
         try:
             r = httpx.Client(http2=True).post(  # noqa: S113 Call to httpx without timeout
                 f'https://generativelanguage.googleapis.com/v{api_version}/models/{model}:generateContent?'
@@ -1360,6 +1371,8 @@ class AIGoogleDiffer(DifferBase):
                         f'AI summary unavailable: Model did not return any candidate output:\n'
                         f'{jsonlib.dumps(result, ensure_ascii=True, indent=2)}'
                     )
+                model_version = result['modelVersion']
+
             elif r.status_code == 400:
                 summary = (
                     f'AI summary unavailable: Received error from {r.url.host}: '
@@ -1377,7 +1390,7 @@ class AIGoogleDiffer(DifferBase):
                 f'AI summary unavailable: HTTP client error: {e} when requesting data from ' f'{e.request.url.host}'
             )
 
-        return summary
+        return summary, model_version
 
     def differ(
         self,
@@ -1394,8 +1407,8 @@ class AIGoogleDiffer(DifferBase):
             RuntimeWarning,
         )
 
-        def get_ai_summary(prompt: str, system_instructions: str) -> str:
-            """Generate AI summary from unified diff, or an error message"""
+        def get_ai_summary(prompt: str, system_instructions: str) -> tuple[str, str]:
+            """Generate AI summary from unified diff, or an error message, plus the model version."""
             GOOGLE_AI_API_KEY = os.environ.get('GOOGLE_AI_API_KEY', '').rstrip()
             if len(GOOGLE_AI_API_KEY) != 39:
                 logger.error(
@@ -1405,12 +1418,10 @@ class AIGoogleDiffer(DifferBase):
                 return (
                     f'## ERROR in summarizing changes using {self.__kind__}:\n'
                     f'Environment variable GOOGLE_AI_API_KEY not found or is of the incorrect length '
-                    f'{len(GOOGLE_AI_API_KEY)}.\n'
+                    f'{len(GOOGLE_AI_API_KEY)}.\n',
+                    '',
                 )
 
-            if 'model' not in directives:
-                directives['model'] = 'gemini-1.5-flash-latest'  # also for footer
-
             if '{unified_diff' in prompt:  # matches unified_diff or unified_diff_new
                 default_context_lines = 9999 if '{unified_diff}' in prompt else 0  # none if only unified_diff_new
                 context_lines = directives.get('prompt_ud_context_lines', default_context_lines)
@@ -1427,7 +1438,7 @@ class AIGoogleDiffer(DifferBase):
                 )
                 if not unified_diff:
                     # no changes
-                    return ''
+                    return '', ''
             else:
                 unified_diff = ''
 
@@ -1442,7 +1453,7 @@ class AIGoogleDiffer(DifferBase):
 
             # check if data is different (same data is sent during testing)
             if '{old_text}' in prompt and '{new_text}' in prompt and self.state.old_data == self.state.new_data:
-                return ''
+                return '', ''
 
             model_prompt = prompt.format(
                 unified_diff=unified_diff,
@@ -1451,14 +1462,14 @@ class AIGoogleDiffer(DifferBase):
                 new_text=self.state.new_data,
             )
 
-            summary = self._send_to_model(
+            summary, model_version = self._send_to_model(
                 self.job,
                 system_instructions,
                 model_prompt,
                 directives=directives,
             )
 
-            return summary
+            return summary, model_version
 
         if directives.get('additions_only') or self.job.additions_only:
             default_system_instructions = (
@@ -1474,7 +1485,7 @@ class AIGoogleDiffer(DifferBase):
                 'You are a skilled journalist tasked with analyzing two versions of a text and summarizing the key '
                 'differences in meaning between them. The audience for your summary is already familiar with the '
                 "text's content, so you can focus on the most significant changes.\n\n"
-                "**Instructions:**\n\n'"
+                '**Instructions:**\n\n'
                 '1. Carefully examine the old version of the text, provided within the `<old_version>` and '
                 '`</old_version>` tags.\n'
                 '2. Carefully examine the new version of the text, provided within the `<new_version>` and '
@@ -1485,25 +1496,38 @@ class AIGoogleDiffer(DifferBase):
                 '5. Summarize the identified differences, except those ignored, in a clear and concise manner, '
                 'explaining how the meaning has shifted or evolved in the new version compared to the old version only '
                 'when necessary. Be specific and provide examples to illustrate your points when needed.\n'
-                '6. If ther are only additions to the text, then summarize the additions.\n'
+                '6. If there are only additions to the text, then summarize the additions.\n'
                 '7. Use Markdown formatting to structure your summary effectively. Use headings, bullet points, '
                 'and other Markdown elements as needed to enhance readability.\n'
                 '8. Restrict your analysis and summary to the information provided within the `<old_version>` and '
-                '`<new_version> tags. Do not introduce external information or assumptions.\n'
+                '`<new_version>` tags. Do not introduce external information or assumptions.\n'
             )
             default_prompt = '<old_version>\n{old_text}\n</old_version>\n\n<new_version>\n{new_text}\n</new_version>'
         system_instructions = directives.get('system_instructions', default_system_instructions)
         prompt = directives.get('prompt', default_prompt).replace('\\n', '\n')
-        summary = get_ai_summary(prompt, system_instructions)
+        summary, model_version = get_ai_summary(prompt, system_instructions)
         if not summary:
             self.state.verb = 'changed,no_report'
             return {'text': '', 'markdown': '', 'html': ''}
         newline = '\n'  # For Python < 3.12 f-string {} compatibility
         back_n = '\\n'  # For Python < 3.12 f-string {} compatibility
-        directives_text = (
-            ', '.join(f'{key}={str(value).replace(newline, back_n)}' for key, value in directives.items()) or 'None'
+        directives.pop('model', None)
+        if directives:
+            directives_text = (
+                ' (differ directive(s): '
+                + (
+                    ', '.join(f'{key}={str(value).replace(newline, back_n)}' for key, value in directives.items())
+                    or 'None'
+                )
+                + ')'
+            )
+        else:
+            directives_text = ''
+        footer = (
+            f"Summary by Google Generative AI's model {model_version}{directives_text}"
+            if model_version and directives_text
+            else ''
         )
-        footer = f'Summary generated by Google Generative AI (differ directive(s): {directives_text})'
         temp_unfiltered_diff: dict[Literal['text', 'markdown', 'html'], str] = {}
         for rep_kind in ['text', 'html']:  # markdown is same as text
             unified_report = DifferBase.process(
@@ -1515,17 +1539,16 @@ class AIGoogleDiffer(DifferBase):
                 temp_unfiltered_diff,
             )
         return {
-            'text': f"{summary}\n\n{unified_report['text']}\n------------\n{footer}",
-            'markdown': f"{summary}\n\n{unified_report['markdown']}\n* * *\n{footer}",
+            'text': f"{summary}\n\n{unified_report['text']}" + (f'\n------------\n{footer}' if footer else ''),
+            'markdown': f"{summary}\n\n{unified_report['markdown']}" + (f'\n* * *\n{footer}' if footer else ''),
             'html': '\n'.join(
                 [
                     mark_to_html(summary, extras={'tables'}).replace('<h2>', '<h3>').replace('</h2>', '</h3>'),
                     '<br>',
                     '<br>',
                     unified_report['html'],
-                    '-----<br>',
-                    f'<i><small>{footer}</small></i>',
                 ]
+                + (['-----<br>', f'<i><small>{footer}</small></i>'] if footer else [])
             ),
         }
 

{webchanges-3.28.0rc0 → webchanges-3.28.2}/webchanges/filters.py

@@ -194,7 +194,7 @@ class FilterBase(metaclass=TrackSubClasses):
                 if unknown_keys and '<any>' not in allowed_keys:
                     raise ValueError(
                         f'Job {job_index_number}: Filter {filter_kind} does not support subfilter or filter '
-                        f"directive(s) {', '.join(unknown_keys)} (supported: {', '.join(allowed_keys)})."
+                        f"directive(s) {', '.join(unknown_keys)}. Only {', '.join(allowed_keys)} are supported."
                     )
 
             yield filter_kind, subfilter
@@ -794,7 +794,10 @@ class FormatJsonFilter(FilterBase):
         self.job.set_to_monospace()
         sort_keys = subfilter.get('sort_keys', False)
         indentation = int(subfilter.get('indentation', 4))
-        parsed_json = jsonlib.loads(data)
+        try:
+            parsed_json = jsonlib.loads(data)
+        except jsonlib.JSONDecodeError as e:
+            return f"Filter '{self.__kind__}' returned JSONDecodeError: {e}\n\n{data!s}", mime_type
         if not mime_type.endswith('json'):
             mime_type = 'application/json'
         return jsonlib.dumps(parsed_json, ensure_ascii=False, sort_keys=sort_keys, indent=indentation), mime_type

{webchanges-3.28.0rc0 → webchanges-3.28.2}/webchanges/handler.py

@@ -67,7 +67,7 @@ class JobState(ContextManager):
     exception: Exception | None = None
     generated_diff: dict[Literal['text', 'markdown', 'html'], str]
     history_dic_snapshots: dict[str | bytes, Snapshot]
-    new_data: str | bytes
+    new_data: str | bytes = ''
     new_error_data: ErrorData = {}
     new_etag: str
     new_mime_type: str = ''
@@ -323,7 +323,7 @@ class JobState(ContextManager):
                 _generated_diff, _mime_type = FilterBase.process(  # type: ignore[assignment]
                     filter_kind, subfilter, self, _generated_diff, _mime_type
                 )
-        self.generated_diff[report_kind] = _generated_diff
+        self.generated_diff[report_kind] = str(_generated_diff)
 
         return self.generated_diff[report_kind]
 
@@ -370,7 +370,7 @@ class Report:
 
         :param job_state: The JobState object with the information of the job run.
         """
-        if job_state.exception is not None and job_state.exception is not NotModifiedError:
+        if job_state.exception is not None and not isinstance(job_state.exception, NotModifiedError):
             logger.info(
                 f'Job {job_state.job.index_number}: Got exception while processing job {job_state.job}',
                 exc_info=job_state.exception,

{webchanges-3.28.0rc0 → webchanges-3.28.2}/webchanges/jobs.py

@@ -786,7 +786,6 @@ class UrlJob(UrlJobBase):
                 http_error_msg = f'{response.status_code} Client Error: {reason} for url: {response.url}'
             else:
                 http_error_msg = f'{response.status_code} Server Error: {reason} for url: {response.url}'
-            logger.error(f'httpx received {http_error_msg}')
 
             if response.status_code != 404:
                 try:
@@ -858,8 +857,7 @@ class UrlJob(UrlJobBase):
                 urllib3.util.ssl_.DEFAULT_CIPHERS += 'HIGH:!DH:!aNULL'  # type: ignore[attr-defined]
             except AttributeError:
                 logger.error(
-                    'Unable to ignore_dh_key_too_small due to bug in '
-                    'requests.packages.urrlib3.util.ssl.DEFAULT_CIPHERS'
+                    'Unable to ignore_dh_key_too_small due to bug in requests.packages.urrlib3.util.ssl.DEFAULT_CIPHERS'
                 )
                 logger.error('See https://github.com/psf/requests/issues/6443')
 
@@ -1105,7 +1103,6 @@ class UrlJob(UrlJobBase):
         ):
             # Instead of a full traceback, just show the error
             exception_str = str(exception).strip()
-            print(f'{exception_str=} {exception.args=} {type(exception)=}')
             if self.proxy and (
                 (httpx and isinstance(exception, httpx.TransportError))
                 or any(
@@ -1373,7 +1370,7 @@ class BrowserJob(UrlJobBase):
                     ignore_default_args=ignore_default_args,
                     timeout=timeout,
                     headless=headless,
-                    proxy=proxy,
+                    proxy=proxy,  # type: ignore[arg-type]
                 )
                 browser_version = browser.version
                 user_agent = headers.pop(
@@ -1402,7 +1399,7 @@ class BrowserJob(UrlJobBase):
                     args=args,
                     ignore_default_args=ignore_default_args,
                     headless=headless,
-                    proxy=proxy,
+                    proxy=proxy,  # type: ignore[arg-type]
                     no_viewport=no_viewport,
                     ignore_https_errors=self.ignore_https_errors,
                     extra_http_headers=dict(headers),
@@ -1811,7 +1808,7 @@ class BrowserJob(UrlJobBase):
         :param tb: The traceback.format_exc() string.
         :returns: A string to display and/or use in reports.
         """
-        exception_str = f'Browser error in {str(exception).strip()}'
+        exception_str = str(exception).strip()
         print(f'{exception_str=}, {tb=}')
         if self.proxy and 'net::ERR' in exception_str:
             exception_str += f'\n\n(Job has proxy {self.proxy})'

{webchanges-3.28.0rc0 → webchanges-3.28.2}/webchanges/reporters.py

@@ -117,7 +117,7 @@ if sys.platform == 'win32':
     try:
         from colorama import AnsiToWin32
     except ImportError as e:  # pragma: no cover
-        AnsiToWin32 = str(e)  # type: ignore[assignment]
+        AnsiToWin32 = str(e)  # type: ignore[assignment,misc]
 
 logger = logging.getLogger(__name__)
 

{webchanges-3.28.0rc0 → webchanges-3.28.2}/webchanges/storage.py

@@ -750,8 +750,12 @@ class YamlConfigStorage(BaseYamlFileStorage):
         if 'job_defaults' in config_for_extras:
             # Create missing 'job_defaults' keys from DEFAULT_CONFIG
             for key in DEFAULT_CONFIG['job_defaults']:
+                if 'job_defaults' not in config_for_extras:
+                    config_for_extras['job_defaults'] = {}
                 config_for_extras['job_defaults'][key] = None  # type: ignore[literal-required]
             for key in DEFAULT_CONFIG['differ_defaults']:
+                if 'differ_defaults' not in config_for_extras:
+                    config_for_extras['differ_defaults'] = {}
                 config_for_extras['differ_defaults'][key] = None  # type: ignore[literal-required]
         if 'hooks' in sys.modules:
             # Remove extra keys in config used in hooks (they are not in DEFAULT_CONFIG)
@@ -794,8 +798,8 @@ class YamlConfigStorage(BaseYamlFileStorage):
                     )
                 else:
                     config['job_defaults']['command'] = config['job_defaults'].pop(
-                        'shell'
-                    )  # type: ignore[typeddict-item]
+                        'shell'  # type: ignore[typeddict-item]
+                    )
 
             for key in {'all', 'url', 'browser', 'command'}:
                 if key not in config['job_defaults']:

{webchanges-3.28.0rc0 → webchanges-3.28.2/webchanges.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: webchanges
-Version: 3.28.0rc0
+Version: 3.28.2
 Summary: Web Changes Delivered. AI-Summarized. Totally Anonymous.
 Author-email: Mike Borsetti <mike+webchanges@borsetti.com>
 Maintainer-email: Mike Borsetti <mike+webchanges@borsetti.com>
@@ -195,7 +195,9 @@ For Generative AI summaries (BETA), you need a free `API Key from Google Cloud A
 
 Installation
 ============
-Install **webchanges** |pypi_version| |format| |status| |security| with:
+|pypi_version| |format| |status| |security|
+
+Install **webchanges**  with:
 
 .. code-block:: bash
 
@@ -203,7 +205,7 @@ Install **webchanges** |pypi_version| |format| |status| |security| with:
 
 Running in Docker
 -----------------
-**webchanges** can easily run in a container; you can find a `Docker <https://www.docker.com/>`__ implementation
+**webchanges** can easily run in a container and you will find a `Docker <https://www.docker.com/>`__ implementation
 `here <https://github.com/yubiuser/webchanges-docker>`__.
 
 
@@ -247,20 +249,19 @@ Schedule
 --------
 **webchanges** leverages the power of a system scheduler:
 
-- On Linux you can use cron, and a tool like `crontab.guru <https://crontab.guru>`__ can build a
-  schedule expression for you (note: see `here <https://www.computerhope.com/unix/ucrontab.htm>`__ if you have never
-  used cron before);
+- On Linux you can use cron, with the help of a tool like `crontab.guru <https://crontab.guru>`__ (help `here
+  <https://www.computerhope.com/unix/ucrontab.htm>`__);
 - On Windows you can use `Windows Task Scheduler <https://en.wikipedia.org/wiki/Windows_Task_Scheduler>`__;
 - On macOS you can use `launchd <https://developer.apple
-  .com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html>`__ (note: see `here
-  <https://launchd.info/>`__ if you have never used launchd before).
+  .com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html>`__ (help `here
+  <https://launchd.info/>`__).
 
 
 Code
 ====
-|coveralls| |issues|
+|coveralls| |issues| |code_style|
 
-The code and issues tracker are hosted on `GitHub <https://github.com/mborsetti/webchanges>`__.
+The code, issues tracker, and discussions are hosted on `GitHub <https://github.com/mborsetti/webchanges>`__.
 
 
 Contributing
@@ -344,7 +345,7 @@ Example enhancements to HTML reporting:
 .. |format| image:: https://img.shields.io/pypi/format/webchanges.svg
     :target: https://pypi.org/project/webchanges/
     :alt: Kit format
-.. |downloads| image:: https://static.pepy.tech/badge/webchanges
+.. |downloads| image:: https://img.shields.io/pypi/dm/webchanges.svg
     :target: https://www.pepy.tech/project/webchanges
     :alt: PyPI downloads
 .. |license| image:: https://img.shields.io/pypi/l/webchanges.svg
@@ -356,15 +357,24 @@ Example enhancements to HTML reporting:
 .. |readthedocs| image:: https://img.shields.io/readthedocs/webchanges/stable.svg?label=
     :target: https://webchanges.readthedocs.io/
     :alt: Documentation status
-.. |CI| image:: https://github.com/mborsetti/webchanges/actions/workflows/ci-cd.yaml/badge.svg?event=push
+.. |old_CI| image:: https://github.com/mborsetti/webchanges/actions/workflows/ci-cd.yaml/badge.svg?event=push
+    :target: https://github.com/mborsetti/webchanges/actions
+    :alt: CI testing status
+.. |CI| image:: https://img.shields.io/github/check-runs/mborsetti/webchanges/main
     :target: https://github.com/mborsetti/webchanges/actions
     :alt: CI testing status
-.. |coveralls| image:: https://coveralls.io/repos/github/mborsetti/webchanges/badge.svg?branch=main
+.. |old_coveralls| image:: https://coveralls.io/repos/github/mborsetti/webchanges/badge.svg?branch=main
+    :target: https://coveralls.io/github/mborsetti/webchanges?branch=main
+    :alt: Code coverage by Coveralls
+.. |coveralls| image:: https://img.shields.io/coverallsCoverage/github/mborsetti/webchanges.svg
     :target: https://coveralls.io/github/mborsetti/webchanges?branch=main
     :alt: Code coverage by Coveralls
+.. |code_style| image:: https://img.shields.io/badge/code%20style-black-000000.svg
+    :target: https://github.com/psf/black
+    :alt: Code style black
 .. |status| image:: https://img.shields.io/pypi/status/webchanges.svg
     :target: https://pypi.org/project/webchanges/
     :alt: Package stability
-.. |security| image:: https://img.shields.io/badge/security-bandit-yellow.svg
+.. |security| image:: https://img.shields.io/badge/security-bandit-green.svg
     :target: https://github.com/PyCQA/bandit
     :alt: Security Status

{webchanges-3.28.0rc0 → webchanges-3.28.2}/webchanges.egg-info/SOURCES.txt

@@ -4,6 +4,7 @@ README.rst
 pyproject.toml
 requirements.txt
 webchanges/__init__.py
+webchanges/__main__.py
 webchanges/cli.py
 webchanges/command.py
 webchanges/config.py