ruby-grafana-reporter 0.8.0 → 0.9.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b3d762755e84f775f00d04fe0333389ae011fbf0ed1b13942cd006599e60e0e2
-  data.tar.gz: 29c72d42855e2de7ecd43cbced9d6339c36312deebe05fc5210b80928e0d7894
+  metadata.gz: 57a3edcb1aa6c41ed2a26688dc6a8a80fba3b9169aa79ee061eeb2d2c0df1cb9
+  data.tar.gz: 897ae84eb2672b645528e118ff7107e2fab7a4f334f63a4c87bb29a020cf93a8
 SHA512:
-  metadata.gz: 2efa68036513d9d7c73ad13296e369bd0c945bf97dd5101f288b3c646443bbc95af0d96f876ab1c23d1424d2f450c1618d24e58705ecbfd406b31adfdd1c7871
-  data.tar.gz: b37f3d569f59a2ae22bea51894cde019ca395c253a4c62d83cbbfd9cd692010e327996c98fb1f0711f8f4ba396cd1eda35b203ebec7c92d95a855218aeb09fdd
+  metadata.gz: b3889cace3794d62bfd50978a24c4eb39cda1148398a4bed8c4354d5c51ad339faa7da7a81b4f001cb29c383ac98334494520a9cbeec2b44019dcafae883bfdb
+  data.tar.gz: 9a7fbf743f0e48e114deaa1d518ba993ea0318229ab391d90481bad43f859b20fca9f8ffaafe6799729a81ace22673d4904797963c776cf9a22c942289f35c23

data/README.md

@@ -17,26 +17,17 @@ Reporting Service for Grafana
   * [Installation](#installation)
   * [Grafana integration](#grafana-integration)
 * [Advanced information](#advanced-information)
-  * [Webservice](#webservice)
+  * [Use grafana variables in templates](#use-grafana-variables-in-templates)
+  * [Webservice endpoints](#webservice-endpoints)
+  * [API endpoints](#api-endpoints)
   * [Using ERB templates](#using-erb-templates)
   * [Using webhooks](#using-webhooks)
-  * [Developing your own plugin](#developing-your-own-plugin)
+  * [Developing your own datasource plugin](#developing-your-own-datasource-plugin)
 * [Roadmap](#roadmap)
 * [Contributing](#contributing)
 * [Licensing](#licensing)
 * [Acknowledgements](#acknowledgements)
 
-## Your support is appreciated!
-
-Hey there! I provide you this software free of charge. I have already
-spend a lot of my private time in developing, maintaining and supporting it.
-
-If you enjoy my work, feel free to
-
-[![buymeacoffee](https://az743702.vo.msecnd.net/cdn/kofi3.png?v=0)](https://ko-fi.com/divinity666)
-
-Thanks for your support and keeping this project alive!
-
 ## About the project
 
 [Grafana](https://github.com/grafana/grafana) is a great tool for monitoring and
@@ -62,11 +53,12 @@ By default (an extended version of) Asciidoctor is enabled as template language.
 
 ### Features
 
-* Supports creation of reports for multiple [grafana](https://github.com/grafana/grafana)
-dashboards (and also multiple grafana installations!) in one resulting report
-* PDF (default), HTML and many other report formats are supported
-* Easy-to-use configuration wizard, including fully automated functionality to create a
-demo report for your dashboard
+* Use Asciidoctor or ERB template language for defining templates
+* Render your templates as PDF (default), HTML, CSV, text files and any other format
+you can think of
+* Create reports containing data from multiple [grafana](https://github.com/grafana/grafana)
+dashboards or even multiple grafana installations
+* Easy-to-use configuration wizard including automated demo report creation
 * Include dynamic content from grafana (find here a reference for all
 [asciidcotor reporter calls](FUNCTION_CALLS.md)):
   * panels as images
@@ -77,8 +69,7 @@ database queries
   * webservice to be called directly from grafana
   * standalone command line tool, e.g. to be automated with `cron` or `bash` scrips
   * microservice from standard asciidoctor docker container without any dependencies
-* Use webhook callbacks on before, on cancel and on finishing a report (see
-configuration file) to combine them with your services
+* Integrate it in your toolchain with webhooks and API calls
 * Solid as a rock - no matter if you do mistakes in your configuration or grafana does no
 longer match templates: the ruby-grafana-reporter webservice will always return properly.
 * Full [API documentation](https://rubydoc.info/gems/ruby-grafana-reporter) available
@@ -234,6 +225,21 @@ The main endpoint to call for report generation is configured in the previous ch
 
 However, if you would like to see, currently running report generations and previously generated reports, you may want to call the endpoint `/overview`.
 
+### API endpoints
+
+If you want to call the reporter programatically, you might want to get machine
+usable endpoints. Therefore the following API endpoints are available:
+
+    POST /api/v1/render
+    GET /api/v1/status
+    DELETE /api/v1/cancel
+
+Those endpoints have to be called with the same URL parameters, as the webservice
+endpoints.
+
+If you want to receive the report itself, you'll have to use the `/view_report`
+call from the webservice endpoints.
+
 ### Using ERB templates
 
 By default the configuration wizard will setup the reporter with the asciidoctor
@@ -280,6 +286,12 @@ grafana-reporter:
   callbacks:
     all:
       - http://<<your_callback_url>>
+    on_before_create:
+      - http://<<your_callback_url>>
+    on_after_cancel:
+      - http://<<your_callback_url>>
+    on_after_finish:
+      - http://<<your_callback_url>>
 ````
 
 Remember to restart the reporter, if it is running as a webservice.
@@ -288,7 +300,7 @@ After having done so, your callback url will be called for each event with
 a JSON body including all necessary information of the report. For details see
 [callback](https://rubydoc.info/gems/ruby-grafana-reporter/GrafanaReporter/ReportWebhook#callback-instance_method).
 
-### Developing your own plugin
+### Developing your own datasource plugin
 
 The reporter is designed to allow easy integration of your own plugins,
 without having to modify the reporter base source on github (or anywhere
@@ -372,6 +384,12 @@ This is just a collection of things, I am heading for in future, without a sched
 If you'd like to contribute, please fork the repository and use a feature
 branch. Pull requests are warmly welcome.
 
+If you enjoy my work, and can't support with code contributions, feel free to
+
+[![buymeacoffee](https://az743702.vo.msecnd.net/cdn/kofi3.png?v=0)](https://ko-fi.com/divinity666)
+
+Thanks for your support and keeping this project alive!
+
 ## Licensing
 
 The code in this project is licensed under MIT license.

data/lib/VERSION.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
 # Version information
-GRAFANA_REPORTER_VERSION = [0, 8, 0].freeze
+GRAFANA_REPORTER_VERSION = [0, 9, 4].freeze
 # Release date
-GRAFANA_REPORTER_RELEASE_DATE = '2024-06-08'
+GRAFANA_REPORTER_RELEASE_DATE = '2024-08-22'

data/lib/grafana/dashboard.rb

@@ -62,8 +62,7 @@ module Grafana
         begin
           @variables << Variable.new(item, self)
         rescue => e
-          # TODO: show this message as a warning - needs test cleanup
-          @grafana.logger.debug(e.message)
+          @grafana.logger.warn(e.message)
         end
       end
     end

data/lib/grafana/grafana_alerts_datasource.rb

@@ -23,6 +23,7 @@ module Grafana
 
       result = webrequest.execute(query_description[:timeout])
       return unless result
+      return result.body unless result.code.to_s == "200"
 
       json = JSON.parse(result.body)
 

data/lib/grafana/variable.rb

@@ -240,7 +240,7 @@ module Grafana
         if !@config['current'].nil?
           self.raw_value = @config['current']['value']
         else
-          raise GrafanaError.new("Grafana variable with type '#{@config['type']}' and name '#{@config['name']}' cannot be handled properly by the reporter. Check your results and raise a ticket on github.")
+          raise GrafanaError.new("Dashboard variable with type '#{@config['type']}' and name '#{@config['name']}' cannot be handled properly by the reporter in queries. Check your resulting report and raise a ticket on github if you face issues.")
         end
       end
     end

data/lib/grafana_reporter/application/application.rb

@@ -134,6 +134,7 @@ module GrafanaReporter
           begin
             template_ext = config.report_class.default_template_extension
             report_ext = config.report_class.default_result_extension
+            report_ext = 'zip' if config.default_document_attributes["convert-backend"] != "pdf" and not config.default_document_attributes["convert-backend"].nil?
             default_to_file = File.basename(config.template.to_s.gsub(/(?:\.#{template_ext})?$/, ".#{report_ext}"))
 
             to_file = config.to_file

data/lib/grafana_reporter/application/webservice.rb

@@ -144,27 +144,56 @@ module GrafanaReporter
           attrs[k] = v.length == 1 ? v[0] : v
         end
 
-        case request.split("\r\n")[0]
-        when %r{^GET /render[? ]}
-          return render_report(attrs)
+        parsed_url = request.split("\r\n")[0].match(%r{(?<verb>GET|POST|DELETE) (?<subpath>/.*?)(?:api/v1/(?<api_action>render|status|cancel))?(?<html_action>render|overview|view_report|cancel_report|view_log)?[ ?]})
+        parsed_url = {} if not parsed_url
 
-        when %r{^GET /overview[? ]}
+        case parsed_url
+        # API calls
+        when -> (h) { h['verb'] == 'POST' && h['api_action'] == 'render' }
+          return render_report(attrs, parsed_url['subpath'], true)
+
+        when -> (h) { h['verb'] == 'GET' && h['api_action'] == 'status' }
+          return report_status(attrs)
+
+        when -> (h) { h['verb'] == 'DELETE' && h['api_action'] == 'cancel' }
+          return cancel_report(attrs, parsed_url['subpath'], true)
+
+        # HTML calls
+        when -> (h) { h['verb'] == 'GET' && h['html_action'] == 'render' }
+          return render_report(attrs, parsed_url['subpath'])
+
+        when -> (h) { h['verb'] == 'GET' && h['html_action'] == 'overview' }
           # show overview for current reports
-          return get_reports_status_as_html(@reports)
+          return get_reports_status_as_html(@reports, parsed_url['subpath'])
 
-        when %r{^GET /view_report[? ]}
-          return view_report(attrs)
+        when -> (h) { h['verb'] == 'GET' && h['html_action'] == 'view_report' }
+          return view_report(attrs, parsed_url['subpath'])
 
-        when %r{^GET /cancel_report[? ]}
-          return cancel_report(attrs)
+        when -> (h) { h['verb'] == 'GET' && h['html_action'] == 'cancel_report' }
+          return cancel_report(attrs, parsed_url['subpath'])
 
-        when %r{^GET /view_log[? ]}
+        when -> (h) { h['verb'] == 'GET' && h['html_action'] == 'view_log' }
           return view_log(attrs)
         end
 
         raise WebserviceUnknownPathError, request.split("\r\n")[0]
       end
 
+      def report_status(attrs)
+        report = @reports.select { |r| r.object_id.to_s == attrs['report_id'].to_s }.first
+        raise WebserviceGeneralRenderingError, 'report_status has been called without valid id' if report.nil?
+
+        response = {
+          report_id: report.object_id,
+          progress: report.progress,
+          state: report.status,
+          done: report.done,
+          execution_time: report.execution_time
+        }
+
+        http_response(200, 'OK', JSON.generate(response), "Content-Type": "application/json")
+      end
+
       def view_log(attrs)
         # view report if already available, or show status view
         report = @reports.select { |r| r.object_id.to_s == attrs['report_id'].to_s }.first
@@ -175,7 +204,7 @@ module GrafanaReporter
         http_response(200, 'OK', content, "Content-Type": 'text/plain')
       end
 
-      def cancel_report(attrs)
+      def cancel_report(attrs, subpath, as_json=false)
         # view report if already available, or show status view
         report = @reports.select { |r| r.object_id.to_s == attrs['report_id'].to_s }.first
         raise WebserviceGeneralRenderingError, 'cancel_report has been called without valid id' if report.nil?
@@ -183,28 +212,34 @@ module GrafanaReporter
         report.cancel! unless report.done
 
         # redirect to view_report page
-        http_response(302, 'Found', nil, Location: "/view_report?report_id=#{report.object_id}")
+        return http_response(302, 'Found', nil, Location: "#{subpath}view_report?report_id=#{report.object_id}") if not as_json
+
+        http_response(200, 'OK', nil)
       end
 
-      def view_report(attrs)
+      def view_report(attrs, subpath)
         # view report if already available, or show status view
         report = @reports.select { |r| r.object_id.to_s == attrs['report_id'].to_s }.first
         raise WebserviceGeneralRenderingError, 'view_report has been called without valid id' if report.nil?
 
         # show report status
-        return get_reports_status_as_html([report]) if !report.done || !report.error.empty?
+        return get_reports_status_as_html([report], subpath) if !report.done || !report.error.empty?
 
         # provide report
-        @logger.debug("Returning PDF report at #{report.path}")
+        @logger.debug("Returning report file #{report.path}")
         content = File.read(report.path, mode: 'rb')
         return http_response(200, 'OK', content, "Content-Type": 'application/pdf') if content.start_with?('%PDF')
 
+        return http_response(200, 'OK', content, "Content-Type": 'application/octet-stream',
+                                                 "Content-Disposition": 'attachment; '\
+                                                 "filename=report_#{attrs['report_id']}.zip") if content.start_with?('PK')
+
         http_response(200, 'OK', content, "Content-Type": 'application/octet-stream',
                                           "Content-Disposition": 'attachment; '\
-                                                                 "filename=report_#{attrs['report_id']}.zip")
+                                          "filename=report_#{attrs['report_id']}.#{report.class.default_result_extension}")
       end
 
-      def render_report(attrs)
+      def render_report(attrs, subpath, as_json=false)
         # build report
         template_file = "#{@config.templates_folder}#{attrs['var-template']}"
 
@@ -222,10 +257,13 @@ module GrafanaReporter
         end
         @reports << report
 
-        http_response(302, 'Found', nil, Location: "/view_report?report_id=#{report.object_id}")
+        return http_response(302, 'Found', nil, Location: "#{subpath}view_report?report_id=#{report.object_id}") if not as_json
+
+        response = {report_id: report.object_id}
+        return http_response(200, 'OK', JSON.generate(response))
       end
 
-      def get_reports_status_as_html(reports)
+      def get_reports_status_as_html(reports, subpath)
         i = reports.length
 
         # TODO: make reporter HTML results customizable
@@ -245,14 +283,14 @@ module GrafanaReporter
               <td><%= report.status %> (<%= (report.progress * 100).to_i %>%)</td>
               <td><%= report.error.join('<br>') %></td>
               <td><% if !report.done && !report.cancel %>
-                <a href="/cancel_report?report_id=<%= report.object_id %>">Cancel</a>
+                <a href="<%= subpath %>cancel_report?report_id=<%= report.object_id %>">Cancel</a>
               <% end %>
               &nbsp;
               <% if (report.status == 'finished') || (report.status == 'cancelled') %>
-                <a href="/view_report?report_id=<%= report.object_id %>">View</a>
+                <a href="<%= subpath %>view_report?report_id=<%= report.object_id %>">View</a>
               <% end %>
               &nbsp;
-              <a href="/view_log?report_id=<%= report.object_id %>">Log</a></td></tr>
+              <a href="<%= subpath %>view_log?report_id=<%= report.object_id %>">Log</a></td></tr>
             <% end.join('') %>
             <tbody>
           </table>
@@ -267,8 +305,8 @@ module GrafanaReporter
       end
 
       def http_response(code, text, body, opts = {})
-        "HTTP/1.1 #{code} #{text}\r\n#{opts.map { |k, v| "#{k}: #{v}" }.join("\r\n")}"\
-        "#{body ? "\r\nContent-Length: #{body.to_s.bytesize}" : ''}\r\n\r\n#{body}"
+        "HTTP/1.1 #{code} #{text}\r\n#{opts.map { |k, v| "#{k}: #{v}" }.join("\r\n")}#{opts.length > 0 ? "\r\n" : ""}"\
+        "#{body ? "Content-Length: #{body.to_s.bytesize}" : ''}\r\n\r\n#{body}"
       end
     end
   end

data/lib/grafana_reporter/asciidoctor/panel_query_table_include_processor.rb

@@ -80,6 +80,7 @@ module GrafanaReporter
       # @see ProcessorMixin#build_demo_entry
       def build_demo_entry(panel)
         return nil unless panel
+        return nil unless panel.model['targets']
 
         ref_id = nil
         panel.model['targets'].each do |item|

data/lib/grafana_reporter/asciidoctor/panel_query_value_inline_macro.rb

@@ -74,6 +74,7 @@ module GrafanaReporter
       # @see ProcessorMixin#build_demo_entry
       def build_demo_entry(panel)
         return nil unless panel
+        return nil unless panel.model['targets']
 
         ref_id = nil
         panel.model['targets'].each do |item|

data/lib/grafana_reporter/asciidoctor/report.rb

@@ -41,36 +41,29 @@ module GrafanaReporter
         ::Asciidoctor.convert_file(@template, extension_registry: registry, backend: attrs['convert-backend'],
                                               to_file: path, attributes: attrs, header_footer: true)
 
-        # store report including als images as ZIP file, if the result is not a PDF
+        # store report including all images as ZIP file, if the result is not a PDF
         if attrs['convert-backend'] != 'pdf'
           # build zip file
-          zip_file = Tempfile.new('gf_zip')
           buffer = Zip::OutputStream.write_buffer do |zipfile|
             # add report file
-            zipfile.put_next_entry("#{path.gsub(@config.reports_folder, '')}.#{attrs['convert-backend']}")
-            zipfile.write File.read(path)
+            zipfile.put_next_entry("#{path.gsub(@config.reports_folder, '').gsub(/\.[\w\d]+$/, '')}.#{attrs['convert-backend']}")
+            zipfile.write File.open(path, 'rb') { |f| f.read }
 
             # add image files
             @image_files.each do |file|
               zipfile.put_next_entry(file.path.gsub(@config.images_folder, ''))
-              zipfile.write File.read(file.path)
+              zipfile.write File.open(file.path, 'rb') { |f| f.read }
             end
           end
-          File.open(zip_file, 'wb') do |f|
-            f.write buffer.string
-          end
 
-          # replace original file with zip file
-          zip_file.rewind
+          # write zip file
           begin
-            File.write(path, zip_file.read)
+            File.open(path, 'wb') do |f|
+              f.write buffer.string
+            end
           rescue StandardError => e
-            logger.fatal("Could not overwrite report file '#{path}' with ZIP file. (#{e.message}).")
+            logger.fatal("Could not overwrite file '#{path}' with zipped file. (#{e.message}).")
           end
-
-          # cleanup temporary zip file
-          zip_file.close
-          zip_file.unlink
         end
 
         clean_image_files

data/lib/grafana_reporter/asciidoctor/sql_table_include_processor.rb

@@ -84,6 +84,8 @@ module GrafanaReporter
           end
         end
         return nil unless ref_id
+        # FIXME this filters out e.g. prometheus in demo reports, as the query method returns a Hash instead of a string
+        return nil unless panel.query(ref_id).is_a?(String)
 
         "|===\ninclude::grafana_sql_table:#{panel.dashboard.grafana.datasource_by_model_entry(panel.model['datasource']).id}"\
         "[sql=\"#{panel.query(ref_id).gsub(/"/, '\"').gsub("\r\n", ' ').gsub("\n", ' ').gsub(/\\/, '\\\\')}\",filter_columns=\"time\","\

data/lib/grafana_reporter/asciidoctor/sql_value_inline_macro.rb

@@ -89,6 +89,8 @@ module GrafanaReporter
           end
         end
         return nil unless ref_id
+        # FIXME this filters out e.g. prometheus in demo reports, as the query method returns a Hash instead of a string
+        return nil unless panel.query(ref_id).is_a?(String)
 
         "grafana_sql_value:#{panel.dashboard.grafana.datasource_by_model_entry(panel.model['datasource']).id}"\
         "[sql=\"#{panel.query(ref_id).gsub(/"/, '\"').gsub("\r\n", ' ').gsub("\n", ' ').gsub(/\\/, '\\\\')}\",from=\"now-1h\","\

data/lib/grafana_reporter/console_configuration_wizard.rb

@@ -319,8 +319,10 @@ default-document-attributes:
     end
 
     def user_input(text, default)
+      $stdout.sync = true
       print "#{text} [#{default}]: "
-      input = gets.gsub(/\n$/, '')
+      $stdout.sync = false
+      input = gets.gsub(/[\n\r]*$/, '')
       input = default if input.empty?
       input
     end

data/lib/ruby_grafana_reporter.rb

@@ -1,8 +1,6 @@
 # frozen_string_literal: true
 
-require 'rubygems'
-require 'rubygems/name_tuple'
-require 'rubygems/ext'
+require 'rubygems' # for OCRAN
 require 'net/http'
 require 'fileutils'
 require 'yaml'
@@ -16,7 +14,7 @@ require 'date'
 require 'time'
 require 'logger'
 require 'asciidoctor'
-require 'asciidoctor/extensions'
+require 'asciidoctor/converter/html5' # for OCRAN
 require 'asciidoctor-pdf'
 require 'zip'
 require_relative 'VERSION'

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ruby-grafana-reporter
 version: !ruby/object:Gem::Version
-  version: 0.8.0
+  version: 0.9.4
 platform: ruby
 authors:
 - Christian Kohlmeyer
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2024-06-08 00:00:00.000000000 Z
+date: 2024-08-22 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: asciidoctor