webcheck-strings 1.600__tar.gz → 1.601__tar.gz

{webcheck_strings-1.600 → webcheck_strings-1.601}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: webcheck_strings
-Version: 1.600
+Version: 1.601
 Summary: Monitor text strings on websites
 Home-page: http://ssb22.user.srcf.net/setup/webcheck.html
 Author: Silas S. Brown
@@ -88,7 +88,7 @@ If the site lists new items but does not support RSS, you can also *extract* ite
 
 Besides checking `http://`, `https://` and `gemini://` URLs, you can check for:
 
-  * DNS changes (useful if you’re maintaining a hosts file somewhere due to unreliable DNS or an awkward proxy situation): URLs starting `dns://` will return a list of all current IPs, each enclosed in parentheses. So for example to be alerted if `93.184.216.34` ceases to be one of the IP addresses of `example.com`, use `dns://example.com (93.184.216.34)`
+  * DNS changes (useful if you’re maintaining a hosts file somewhere due to unreliable DNS or an awkward proxy situation): URLs starting `dns://` will return a list of all current IPs, each enclosed in parentheses. So for example to be alerted if `93.184.215.14` ceases to be one of the IP addresses of `example.com`, use `dns://example.com (93.184.215.14)`
   * Server reachability: if a server has been unreachable for a long time and you want to be alerted if it ever becomes reachable again, you can place `up://` before the URL (e.g. `up://http://www.example.com`) which will return `yes` or `no` and not report an error if the server is not reachable.
   * Misguided blocking of the Lynx browser. Prepend `blocks-lynx://` to a URL to have Webcheck try to fetch it with a Lynx user agent and return `yes` if it gets an error or timeout only with that agent, or `no` otherwise, so you can, at least in a `noscript` tag, warn your visitors that the server to which you link has accidentally started discriminating against blind and other users of the text-only Lynx browser due to misguided security settings—some “example” security configurations of web servers incorrectly assume Lynx implies automation and block it, and staff have been known to copy such examples without review or without realising what blocking Lynx implies.
   * Items in the HTTP HEAD response: prepend `head://` to a URL. This might be useful for checking `Last-Modified` to see if a large download has changed, like `wget -N` but without needing to keep a copy of the file on the machine where your WebCheck runs. 

{webcheck_strings-1.600 → webcheck_strings-1.601}/setup.py

@@ -1,4 +1,4 @@
-from setuptools import setup, find_packages;setup(name='webcheck_strings',version='1.600',entry_points={'console_scripts':['webcheck=webcheck.__init__:main']},license='Apache 2',platforms='any',url='http://ssb22.user.srcf.net/setup/webcheck.html',author='Silas S. Brown',author_email='ssb22@cam.ac.uk',description='Monitor text strings on websites',long_description=r'''WebCheck
+from setuptools import setup, find_packages;setup(name='webcheck_strings',version='1.601',entry_points={'console_scripts':['webcheck=webcheck.__init__:main']},license='Apache 2',platforms='any',url='http://ssb22.user.srcf.net/setup/webcheck.html',author='Silas S. Brown',author_email='ssb22@cam.ac.uk',description='Monitor text strings on websites',long_description=r'''WebCheck
 ========
 
 From http://ssb22.user.srcf.net/setup/webcheck.html 
@@ -71,7 +71,7 @@ If the site lists new items but does not support RSS, you can also *extract* ite
 
 Besides checking `http://`, `https://` and `gemini://` URLs, you can check for:
 
-  * DNS changes (useful if you’re maintaining a hosts file somewhere due to unreliable DNS or an awkward proxy situation): URLs starting `dns://` will return a list of all current IPs, each enclosed in parentheses. So for example to be alerted if `93.184.216.34` ceases to be one of the IP addresses of `example.com`, use `dns://example.com (93.184.216.34)`
+  * DNS changes (useful if you’re maintaining a hosts file somewhere due to unreliable DNS or an awkward proxy situation): URLs starting `dns://` will return a list of all current IPs, each enclosed in parentheses. So for example to be alerted if `93.184.215.14` ceases to be one of the IP addresses of `example.com`, use `dns://example.com (93.184.215.14)`
   * Server reachability: if a server has been unreachable for a long time and you want to be alerted if it ever becomes reachable again, you can place `up://` before the URL (e.g. `up://http://www.example.com`) which will return `yes` or `no` and not report an error if the server is not reachable.
   * Misguided blocking of the Lynx browser. Prepend `blocks-lynx://` to a URL to have Webcheck try to fetch it with a Lynx user agent and return `yes` if it gets an error or timeout only with that agent, or `no` otherwise, so you can, at least in a `noscript` tag, warn your visitors that the server to which you link has accidentally started discriminating against blind and other users of the text-only Lynx browser due to misguided security settings—some “example” security configurations of web servers incorrectly assume Lynx implies automation and block it, and staff have been known to copy such examples without review or without realising what blocking Lynx implies.
   * Items in the HTTP HEAD response: prepend `head://` to a URL. This might be useful for checking `Last-Modified` to see if a large download has changed, like `wget -N` but without needing to keep a copy of the file on the machine where your WebCheck runs. 

{webcheck_strings-1.600 → webcheck_strings-1.601}/webcheck/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # (compatible with both Python 2 and Python 3)
 
-# webcheck.py v1.6 (c) 2014-24 Silas S. Brown.
+# webcheck.py v1.601 (c) 2014-24 Silas S. Brown.
 # See webcheck.html for description and usage instructions
 
 # Licensed under the Apache License, Version 2.0 (the "License");
@@ -103,7 +103,7 @@ def read_input():
   days = 0 ; extraHeaders = []
   url = mainDomain = None
   lines = read_input_file()
-  lastList = None
+  lastList = None # refs a list within ret, of [(days,text,elseLogic)]
   while lines:
     line = line_withComment = " ".join(lines.pop().split())
     if " #" in line: line = line[:line.index(" #")].strip()
@@ -167,7 +167,7 @@ def read_input():
     if isElse:
       assert lastList, "else without suitable rule before it"
       lastList[-1] = lastList[-1][:2] + ((url,[(0,text,None)]),) # must be days=0 because don't want to re-check the days count when just retrieved and failed something possibly on same URL ('else:' can be used for simple retrying)
-      lastList = lastList[-1][2][1] # so 'else' can be used as 'else if'
+      lastList = lastList[-1][2][1] # ref the above list (the one after url), so 'else' can be used as 'else if' and next iteration extends it
     else:
       lastList = ret.setdefault({
         # domains to treat as equivalent for rate reduce
@@ -419,13 +419,13 @@ def doJob(opener,delayer,url,checklist,extraHeaders):
           out=check(t[1:],content,"Source of "+url,"")
       elif not t or t.startswith('#'):
           parseRSS(url,content,t.replace('#','',1).strip())
-          out = None
+          out = None # (no else: after real RSS, but you can do else: for no extracted items: that goes through check() above)
       else:
         if textContent==None:
           textContent,errmsg=htmlStrings(content)
         out=check(t,textContent,url,errmsg)
-      if out:
-        if item[2]: toRet.append(item[2])
+      if out: # something to alert, unless else:
+        if item[2]: toRet.append(item[2]) # else:
         else: sys.stdout.write(out) # don't use 'print' or may have problems with threads
   return toRet
 
@@ -714,7 +714,7 @@ def check(text,content,url,errmsg):
     if comment: comment="\n  "+paren(comment)
     text = text.strip()
     assert text # or should have gone to parseRSS instead
-    if text.startswith('{') and text.endswith('}') and '...' in text: extract(url,content,text[1:-1].split('...'),orig_comment)
+    if text.startswith('{') and text.endswith('}') and '...' in text: return extract(url,content,text[1:-1].split('...'),orig_comment)
     elif text.startswith("!"): # 'not', so alert if DOES contain
         if len(text)==1: return # TODO: print error?
         if myFind(text[1:],content):
@@ -820,8 +820,8 @@ def extract(url,content,startEndMarkers,comment):
     c = content[i+len(start):j].decode('utf-8').strip()
     if c: items.append(('Auto-extracted text:','',c,"")) # NB the 'title' field must not be empty (unless we relocate that logic to parseRSS instead of handleRSS)
     i = j+len(end)
-  if not items: print ("No items were extracted from "+url+" via "+S(start)+"..."+S(end)+" (check that site changes haven't invalidated this extraction rule)")
-  handleRSS(url,items,comment,"extracted")
+  if not items: return ("No items were extracted from "+url+" via "+S(start)+"..."+S(end)+" (check that site changes haven't invalidated this extraction rule)")
+  else: handleRSS(url,items,comment,"extracted")
 
 def myFind(text,content):
   text,content = B(text),B(content)

{webcheck_strings-1.600 → webcheck_strings-1.601}/webcheck_strings.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: webcheck-strings
-Version: 1.600
+Version: 1.601
 Summary: Monitor text strings on websites
 Home-page: http://ssb22.user.srcf.net/setup/webcheck.html
 Author: Silas S. Brown
@@ -88,7 +88,7 @@ If the site lists new items but does not support RSS, you can also *extract* ite
 
 Besides checking `http://`, `https://` and `gemini://` URLs, you can check for:
 
-  * DNS changes (useful if you’re maintaining a hosts file somewhere due to unreliable DNS or an awkward proxy situation): URLs starting `dns://` will return a list of all current IPs, each enclosed in parentheses. So for example to be alerted if `93.184.216.34` ceases to be one of the IP addresses of `example.com`, use `dns://example.com (93.184.216.34)`
+  * DNS changes (useful if you’re maintaining a hosts file somewhere due to unreliable DNS or an awkward proxy situation): URLs starting `dns://` will return a list of all current IPs, each enclosed in parentheses. So for example to be alerted if `93.184.215.14` ceases to be one of the IP addresses of `example.com`, use `dns://example.com (93.184.215.14)`
   * Server reachability: if a server has been unreachable for a long time and you want to be alerted if it ever becomes reachable again, you can place `up://` before the URL (e.g. `up://http://www.example.com`) which will return `yes` or `no` and not report an error if the server is not reachable.
   * Misguided blocking of the Lynx browser. Prepend `blocks-lynx://` to a URL to have Webcheck try to fetch it with a Lynx user agent and return `yes` if it gets an error or timeout only with that agent, or `no` otherwise, so you can, at least in a `noscript` tag, warn your visitors that the server to which you link has accidentally started discriminating against blind and other users of the text-only Lynx browser due to misguided security settings—some “example” security configurations of web servers incorrectly assume Lynx implies automation and block it, and staff have been known to copy such examples without review or without realising what blocking Lynx implies.
   * Items in the HTTP HEAD response: prepend `head://` to a URL. This might be useful for checking `Last-Modified` to see if a large download has changed, like `wget -N` but without needing to keep a copy of the file on the machine where your WebCheck runs.