flnews_post_proc 1.7

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 2c864ce301d84082376b2d6662eab044a8b9386d7fc59bea42ffa457811687d7
+  data.tar.gz: deb232cd41c1d3e8c96052eace342f022d440039beeb5b644e0697ecc3209888
+SHA512:
+  metadata.gz: 523e37695eaef8c553b9458d7eba015e54ac709370368de54377f6b91c6a837a225f2f885e2c88b28aaf887ac33d4344f6d2b5b600e17f45e847c32327941ec9
+  data.tar.gz: 8c2c7747e1209268bedca5468c4cbc19251bab962c0783990444e0739042b7bf22444d714b466e656825ba0d8b76642d5eebf985a31c407049fc6a6c0d31e7e3

data/README.md

@@ -0,0 +1,381 @@
+ flnews_post_proc
+=======================
+ Post-Processor for the flnews newsreader
+------------------------------------------
+
+**IMPORTANT**  
+Please give priority to the documentation which is delivered with the gem  
+file, as the markdown syntax plays tricks, sometimes, and I may not be  
+quick enough to correct this file. This applies especially to the section   
+[_Configuration_](#CONFIGURATION), below.  
+
+The gem-file contains different formats of the man-page in English and French. 
+See section [_Source Code_](Source-Code), below, for instructions to access 
+these texts.  Or see the same files [on the  
+Web](https://www.uplawski.eu/div/flnews/). 
+
+SYNOPSIS
+=======================
+
+An article is piped-in to the post-processor. This is normally done  
+automatically as soon as the variable “post_proc“ in the flnews configuration  
+file is set to the flnews_post_proc executable, i.e.:
+
+**post_proc : /usr/local/bin/flnews_post_proc**
+
+Locally stored articles can be piped in to the post-processor, as needed  
+for testing purposes, with a command-line like the following:
+
+**flnews_post_proc < article**
+
+INSTALLATION
+=============
+You must have a Ruby interpreter installed (see [ruby-lang.org](https://www.ruby-lang.org) ).  
+Your Linux distribution will probably let you choose one from its  
+software resources. Use the package management utility of your choice.
+
+While you can download the current gem file for flnews_post_proc from  
+*[rubygems.org](https://www.rubygems.org)*, it is probably best to use the “gem”-utility  
+for the installation. It should be available from the same source as your  
+Ruby interpreter.
+
+With the interpreter and the gem utility in place, the installation effort is  
+reduced to a simple command:  
+user@machine:~$ **sudo gem install flnews_post_proc**
+
+This way, the gem file will be downloaded from rubygems.org and installed in  
+the designated path for ruby-gems. If the installation completes without  
+errors, the command **flnews_post_proc** will become immediately available.
+
+For other uses of the gem utility, see the output of  
+user@machine:~$ **gem --help**
+
+DESCRIPTION
+=======================
+The flnews newsreader is sufficient for Usenet access, i.e. to receive and read  
+articles from -, as well as to write and post articles to newsgroups.
+
+When you compare news-clients, you will always notice the differences and  
+choose the software that you prefer. flnews however, has the charm that you can  
+influence how the program itself works but also modify posts that flnews  
+produces, just before the program will transmit them to the chosen nntp-server.
+
+The flnews_post_proc can add and change details of a post, in ways that are  
+currently not possible with flnews alone.  As the program is configurable, it  
+can probably respond to the needs of some Usenet users. You should, however,  
+rather take it as an example for what can be done and an inspiration for your  
+own creations.
+
+The limits of a basic newsreader — what the program can do 
+----------------------------------------------------------
+While the articles that flnews creates, are complete and ready to be posted,  
+some users may not always agree with the result and for arbitrary reasons:
+
+* There may be inconveniences when you post to different newsgroups in  
+  different languages, as an introductory line which refers to a previous  
+  post can only be set once in the flnews configuration. The consequence  
+  can be that your post to a french newsgroup begins with an introduction  
+  in English.  
+  The post-processor program can set an introductory line specifically  
+  chosen for one or several newsgroups.  
+
+* The same conflict arises, when you have set a standard signature text and  
+  would like to replace it against another, based on the newsgroup you are  
+  about to post to.  
+  The post-processor program sets specific signatures as configured for one  
+  or several newsgroups. You can even define a list of signatures and have
+  the program pick one from the list.
+
+* Some custom headers may serve to convey additional information to  
+  interested readers of your post, like GnuPG key IDs, your language skills  
+  or the like. The signature may be a better choice than custom headers.  
+  You are free. I just mention face and x-face but prefer that you do not  
+  remember I did.  
+  Custom headers may be defined in the configuration file for the program  
+  and will then be added to each outgoing post.
+
+* The Archive- and X-No-Archive headers are sometimes set to avoid that an  
+  article be saved and stays available to search-engines (Google, notably).   
+  Test-  postings, for example, do probably not justify at all that they would  
+  be  referenced in search results. The post-processor program can impose the  
+  Archive- and X-No-Archive header for all posts to certain newsgroups.  
+
+  **ATTN** As of 2024, the header “X-No-Archive“ has lost most of its utility  
+  and it is the decision of server operators to honor it or not.  
+
+* If a news post contains many references to either other posts or Web  
+  pages, the text can be cluttered with URLs.  
+  The post-processor can identify marked text fragments and transform them  
+  into footnotes, which will be attached as a list at the bottom of the  
+  post.  
+  This works almost like the references-link in Wikipedia, but the delimiter  
+  can be determined in the configuration file.  
+  Example (with %=):  
+  _This is an object %=and this becomes the footnote, describing the object  
+  further=%_ 
+
+* The program can attempt to correct ill formatted references. This is an  
+  option, you can disable it by setting the option VFY_URLS to “no” in the  
+  configuration. Only URLs starting with http or https are currently 
+  handled.
+
+Dialog to override settings
+---------------------------
+You can have a dialog displayed just before the post-processor is invoked, to  
+**disable** some configuration options. Provided that either YAD, Zenity,  
+Whiptail or only xterm are available on your computer, you can choose from   
+the following options. You **cannot** use the dialog to enable options which    
+have not yet been set in the configuration.
+
+* Signatures, as set in the configuration **can be ignored**. Either  
+  a default signature will appear as set in flnews or none.  
+* Custom Headers, if configured, can be **omitted**.  
+* Archive- and X-No-Archive headers, if set for the current newsgroup, **can be
+  ignored**.  
+* The verification & correction of references **can be disabled**.  
+* Logging can be **switched off**, if set.
+
+Pushing Esc or the cancle-button of the dialog interrupts the process, flnews  
+will not post the article.
+
+You can disable the dialog, which ensures that all configured options will be  
+applied, without the need for further interaction (see below, option  
+OVERRIDE_CONFIG). 
+
+CONFIGURATION
+===============
+On first execution of the program, a copy of the original configuration file is  
+created in */home/[user]/.flnews_post_proc.conf*  
+It is this file which is used from then on. If you delete it, it will be  
+recreated, on the next occasion, but your own changes will be lost.  
+
+The configuration file is in YAML syntax and full of explanations. The  
+variables defined in this file can be classified as belonging to one of two  
+categories:
+
+* Variables describing values originally set by flnews, which should be used or  
+  replaced. The important elements are usually matched in a capture group.  
+
+* Variables defining the new or altered content.  
+
+*ATTENTION*: Most problems that occurred with recent versions of
+*flnews_post_proc* were caused by inadequate option values and very often ill
+chosen or mal formatted regular expressions. PSE double check your
+configuration, if the program does not seem to work normally.
+
+**FUP_NAME**   
+A Regular Expression, describing the string which contains the name of  
+previous poster who is the author of a quoted post.  This string is  
+recognized in the original article and may be used with the fitting element  
+rom *GROUP_INTRO*, below. The Regexp format is that of the Regexp class in  
+Ruby, noted as a String. Beware to mask a backslash '\\' by another one,  
+ike in the example. A capture-group '()' serves to extract the name from the  
+match result.
+
+Leave this field empty to keep the default from the flnews configuration  
+intact.
+
+ CONTENT: A String equivalent of a regular expression.
+
+ DEFAULT: EMPTY
+
+ EXAMPLE1: _"On \\d+.\\d+.\\d{2,4} at \\d+:\\d+ **(.*)** wrote:"_
+
+ EXAMPLE2: _"**(.*)** wrote:"_
+
+**FUP_GROUP**  
+ A Regular Expression, describing the string which contains the newsgroup    
+ where the previous post, that you are referring to in the followup, had been  
+ published.
+
+ Leave this field empty to ignore the precise group.
+
+ CONTENT: A String equivalent of a regular expression.
+
+ DEFAULT: EMPTY
+
+ EXAMPLE: "wrote in **(.*)**:"
+
+**GROUP_INTROS**  
+ Introductory strings, referring to the previous poster who is the author of a  
+ quoted post. If you match the newsgroup of the post (see FUP_GROUP), you can  
+ use these variables in the result.  
+ Currently only %fup_name% and %fup_group% are reproduced in the resulting  
+ introductory string.
+
+ CONTENT: A newsgroup or regexp per line, followed by a colon, a space and a String
+
+ DEFAULT: As configured in flnews
+ 
+ EXAMPLE: _alt.test: "Thus spoke #\{fup_name\} on that baleful #\{fup_date\}:"_
+
+**GROUP_SIGS**  
+ A signature line per Newsgroup.  
+ If a file is named instead, the program will pick randomly one signature from
+ that file. The file should contain signatures, separated by 1 empty line. 
+
+ ATTN! In multi line signatures, you have to use \r\n for line breaks.
+
+ CONTENT: A newsgroup or regexp per line, followed by a colon, a space and a String.  
+
+ DEFAULT: As configured in flnews
+
+ EXAMPLE: alt.test: _"Signature for alt.test\\\\r\\\\nsecond line"_
+ EXAMPLE: comp.*: /home/user/.my_sigs
+
+**CUSTOM_HEADERS**  
+ Additional headers for the outgoing article 
+
+ CONTENT: 1 line per header : a dash and space, then a String, comprising the  
+ name of the header, ending in a colon and the value of the header. 
+
+ DEFAULT: undefined
+
+ EXAMPLE (2 headers):  
+  _\- 'X-My-Header: nothing fancy'_  
+  _\- 'X-Another-Header: care not!'_ 
+
+**NO_ARCHIVE_GROUPS**  
+
+ The newsgroups, where the headers “Archive: no” and “X-No-Archive: YES” shall
+ be set.  
+
+ CONTENT: a dash and space, then a String, containing the name of the group or   
+ a regexp.
+
+ DEFAULT: empty
+
+ EXAMPLE (1 group, 1 hierarchy):  
+ _\- "alt.test"_  
+ _\- "^news.*"_
+
+
+**DEBUG_LOG**  
+ The name of a file, where debug messages are written. Setting this  
+ variable will enable the log. Leave empty to disable logging.
+
+ CONTENT: The name of a writable file, which will be created if inexistent  
+ and overwritten if need be.
+
+ DEFAULT: empty
+
+ EXAMPLE: _'/tmp/a_log-file.txt'_
+
+**LOG LEVEL**  
+ One of debug, fatal, error, info, warn
+
+**REFERENCES_SEPARATOR**  
+A sequence of 2 or more symbols marking the end of the message-body and the  
+beginning of a list of “references” or “footnotes”. It will only appear, if  
+the original message-body contains text marked for use as such a footnote. See  
+*REFERENCES_DELIMITER*.
+
+ If the option is not defined or empty, the list of footnotes will appear  
+ below the last line of the message body and no separator will be inserted.  
+
+ CONTENT: A quoted symbol or sequence of symbols.
+
+ DEFAULT: empty
+
+ EXAMPLE: _'---------'_
+
+**REFERENCES_DELIMITER**  
+ A symbol or sequence of symbols marking the beginning of a text which will  
+ serve as footnote (or reference). The **reversed sequence** musst be used to   
+ mark the end of the text. The presence of this sequence or symbol in the  
+ original message body will cause the enclosed text to be moved below the  
+ message body.  
+ The *REFERENCES_SEPARATOR*, if defined, will separate the message from the  
+ list  of footnotes.  
+
+ If this option is not defined or empty, footnotes are not created.
+
+ CONTENT a quoted symbol or sequence of symbols.
+
+ DEFAULT: none/empty
+
+ EXAMPLE: _'%?'_
+
+**REFERENCE_FORMAT**  
+ A format-string, using %s for a number, replacing the reference text in the   
+ message body.
+
+ DEFAULT: " %s)"  -> becomes 1) ... 2)  ... 3)
+
+ EXAMPLE: _"(%s)"  -> becomes (1) ... (2)  ... (3)_
+
+**VFY_URLS**
+A Boolean constant. It determines if the program shall verify and possibly  
+try to correct URLs. Even if URLs are identified as such, only a few  
+manipulations are attempted :  
+
+* Angular brackets '<' and '>' are added, if missing  
+* Slashes are added, if they are found missing after "http(s):"  
+
+If the variable is not set, a value 'yes' is assumed.
+
+CONTENT: One of YES, yes, NO, no, and other variations of case.
+
+DEFAULT: yes
+
+EXAMPLE: ... I let you guess.
+
+**OVERRIDE_CONFIG**
+GROUP_SIGS, XNAY_GROUPS, CUSTOM_HEADERS and DEBUG_LOG.  
+A dialog may be displayed which allows you to disable any of these  
+four options, so that the defaults from flnews prevail.
+
+ATTN! Canceling the dialog or pushing the Esc-key does interrupt   
+the process. flnews will not post the article.
+
+Set this option to no, NO or similar to disable the dialog.
+
+DEFAULT: yes
+
+EXAMPLE: No
+
+Other Information
+=================
+
+Testing
+-------
+The effects that the execution of the program will have on a posting can be  
+verified in two ways:
+
+1. By piping-in a post that had previously been saved to a file:  
+   **:~$ /usr/local/bin/[post-processor] < [test-article]**  
+   This will show the resulting new version of the article on screen, but you  
+   can also pipe the output into another file. This is a great way to test a  
+   program during development or to test your own configuration of the program.  
+
+2. By posting directly into a test-newsgroup (like alt.test or similar). This  
+   is mandatory before you really post to thematic newsgroups and when the  
+   settings of the post-processor will affect the article.  
+
+
+Source-Code
+-----------
+The gem-file that you get with the gem-utility or from rubygems.org contains  
+all the code of the program and some documentation (this page notably). To read  
+its content, you must
+
+1. untar the gem-file with tar _-xf flnews_post_proc-0.1.gem_
+2. uncompress the data.gz archive: _gunzip data.gz_
+3. untar the resultig data.tar archive: _tar -xf data.tar_
+
+This creates the directories bin, doc and lib.
+
+License
+-------
+flnews_post_proc is distributed under the conditions of the WTFPL-2.0 or later  
+License (see http://www.wtfpl.net/txt/copying/ or license-text in the doc  
+directory of the gem-file).
+
+Author
+-----
+flnews_post_proc has been developed by  
+Michael Uplawski <michael.uplawski@uplawski.eu>
+
+Ω
+==
+

data/bin/flnews_post_proc

@@ -0,0 +1,128 @@
+#!/bin/env ruby
+#encoding: UTF-8
+
+=begin
+/***************************************************************************
+ *   ©2023-2024, Michael Uplawski <michael.uplawski@uplawski.eu>           *
+ *                                                                         *
+ *   This program is free software; you can redistribute it and/or modify  *
+ *   it under the terms of the DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE *
+ *                                                                         *
+ *   This program is distributed in the hope that it will be useful,       *
+ *   but WITHOUT ANY WARRANTY; without even the implied warranty of        *
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.                  *
+ *                                                                         *
+ ***************************************************************************/
+=end
+# This program is Linux only
+if ! /linux/ =~ RUBY_PLATFORM
+  msg = "ERROR ! This script #{$0} cannot be used with #{RUBY_PLATFORM}"
+  msg << "\n\tABORTING! Bye."
+  STDERR.puts msg
+  exit false
+end
+
+# needs the ruby-filemagic gem
+# require 'filemagic'
+# require 'marcel'
+require 'tempfile'
+
+require_relative '../lib/override'
+require_relative '../lib/flnews_post_proc'
+require_relative '../lib/basic_logging'
+require_relative '../lib/configuration'
+require_relative '../lib/version'
+
+self.extend(BasicLogging)
+
+def usage
+  usage = "\nWhat do you want me to do? Where is the article to post-process?"
+  usage << "\nUsage: "
+  usage << "\n\t#{$0} < article.text"
+  info usage
+  STDERR.puts usage
+  exit false 
+end
+
+msg = PROGNAME.dup << ' ' << PROGVERSION << ' starting' 
+msg = "–––––– " << msg << " ––––––"
+info msg
+# Get a configuration 
+#   this call results in the configuration file to be
+#   read and interpreted. As the log depends on the
+#   configuration, errors may render the debug-log 
+#   unavailable. See STDERR in these cases, i.e.
+#   call the program on the command-line and pipe-in
+#   a message. Is this okay?
+debug 'creating config'
+config = Configuration::instance
+
+# Store the path to the main executable
+#   ... Forgot what this should be good for.
+# config.set(:BINDIR, File::dirname(__FILE__) )
+
+# Only if data is on STDIN
+if (!STDIN.tty?)
+  # read from STDIN
+  artext = ARGF.read
+
+  # There is content, create the post-processor.
+  if !artext.strip.empty? 
+=begin
+# unused
+    if (['news', 'rfc822'].none? {|m|  mime.start_with?('message/' << m) } )
+      # Not a pure news message
+      # Add more reactions, as you stumble upon them ...
+      warn ("Input is of wrong mime-type " << mime.split(';')[0])
+
+      # Javascript okay
+      if (['javascript'].any? {|m| mime.start_with?('application/' << m)} )
+        warn 'Input contains java-script code!'
+      # C okay
+      elsif(['x-c'].any? {|m| mime.start_with?('text/' << m)} )
+        warn 'Input contains C- or C++ code!'
+      # Ruby okay
+      elsif(['x-ruby'].any? {|m| mime.start_with?('text/' << m) })
+        warn 'Input contains Ruby-code!'
+      else
+        warn 'Input contains other things than plain-text, but I cannot say, what.'
+      end
+    end
+=end
+    #----------->
+    # Allow to override the configuration,
+    # if not disabled (default is true)
+    if config.OVERRIDE_CONFIG != false
+      cdlg = OverrideDlg.new
+      discarded = cdlg.show
+      if discarded && !discarded.empty?    
+        debug('options overriden ' << discarded)
+        OverrideDlg.cvars.each do |v|    
+          if discarded.include?(v.to_s)
+            debug('removing ' << v.to_s)
+            config.set(v, nil)
+          end
+        end
+      end
+      debug('new config: ' << config.inspect)
+    end
+    #<--------------
+    # Do it:
+    pp = PostProcessor.new(artext)    
+    # and get a result.
+    article = pp.article
+    if article 
+      # -------------> The main objective <------
+      puts article
+      # <------------- over and out ------>
+      exit true
+    end
+    # whatever.
+    exit false
+  else
+    error( "Cannot read the article, no content" ) 
+  end
+else
+  usage() 
+end
+# Ω