geoblacklight_admin 0.1.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7684e47be28824d2066af09187c41a5f443cf3ead2ea32d83c22c7bb1b254c2d
-  data.tar.gz: d444226fbbacfcc2b344e595c8920758e0a75b7981c7202041063fd26d89fec1
+  metadata.gz: f18e2477b4c86505e2db4f96deecdd1cb1c7a3da7e6a5ec914d62cb03d96462f
+  data.tar.gz: 65686ca5de9a294a3232e87c9639b0ee4dec85e2eb6f4d858f0dd47da81b3e97
 SHA512:
-  metadata.gz: 89901b678a8a2e240d63297a8a23ecce61e24c15d2e622b333710dcc4ad4d976ec205b583b1abf4373fb398962812895599650864ddb170137c01ca00f2f756d
-  data.tar.gz: c07f011ff04015800cec00397e7bbf65ef33a8e0511e56e300930dfe1eb9d2c110148666e805f863e7bfc18aa7fe311887f9abc5c5839387a5afb0dbcd0f2d4f
+  metadata.gz: 9a55943084756c68c07211e72de617cf1440f7128912bac6549ffcc95ac51733d2e3d21c165fb8f24acd6dda315c9a316056bda088e5eae65081552c9e6ce302
+  data.tar.gz: '090b8cdcac146e2fc720004f923dc73e4cedb3a00c75ea13d90f1ec0c3786b57be9ae65a9d29858e09dea66eb43bfcfbf68b3e0288c5414ff3b0b04bbe41cc92'

data/README.md

@@ -1,13 +1,14 @@
 # geoblacklight_admin
 
-GeoBlacklight Admin is a [GeoBlacklight](https://github.com/geoblacklight/geoblacklight) plugin, built on [Kithe](https://github.com/sciencehistory/kithe), that provides a complex web-form for editing documents and an CSV-based import/export workflow for OpenGeoMetadata's [Aardvark schema](https://opengeometadata.org/ogm-aardvark/). It's a Rubygem port of the Big Ten Academic Alliance's production workflow tool [GEOMG](https://github.com/geobtaa/geomg).
+![CI](https://github.com/geobtaa/geoblacklight_admin/actions/workflows/ci.yml/badge.svg)
 
-## Warning: Pre-Alpha
+GeoBlacklight Admin is a [GeoBlacklight](https://github.com/geoblacklight/geoblacklight) plugin, built on [Kithe](https://github.com/sciencehistory/kithe), that provides a complex web-form for editing documents and an CSV-based import/export workflow for OpenGeoMetadata's [Aardvark schema](https://opengeometadata.org/ogm-aardvark/). GBL Admin is based on the Big Ten Academic Alliance's production workflow tool [GEOMG](https://github.com/geobtaa/geomg).
 
-> :warning: This gem is not ready for public adoption yet, but hopefully someday soon (Late Summer 2023?). Feel free to kick the tires if you are curious and know GeoBlacklight's codebase/stack well.
+[![GeoBlackliht Admin](https://raw.githubusercontent.com/geobtaa/geoblacklight_admin/develop/docs/gbl_admin_screenshot.png)](https://youtu.be/lWjcr-Ow228 "GeoBlacklight Admin")
 
 ## Requirements
 
+* Rails v6.1 (not v7 yet)
 * Blacklight v7 (not v8)
 * GeoBlacklight v4 (not v3)
 * Solr v8.4+
@@ -15,43 +16,26 @@ GeoBlacklight Admin is a [GeoBlacklight](https://github.com/geoblacklight/geobla
 * Redis (for Sidekiq)
 * OpenGeoMetadata's Aardvark Schema
 
-## Install Notes
+## Installation
 
-### Terminal 1 - Drop/Create application PG database
-```bash
-psql postgres
-DROP DATABASE geoblacklight_development;
-CREATE DATABASE geoblacklight_development;
-```
-
-### Terminal 2 - Bundle and run generator
-```bash
-bundle install
-bundle exec rake engine_cart:generate
-```
+### PostgreSQL
 
-When the .internal_test_app is built, edit `geoblacklight_admin_helper.rb`, and uncomment this Pagy include. Why including Pagy here breaks the entire generator build is beyond me...
+You need a PostgreSQL database to use this project.
 
-```bash
-  # @TODO:
-  # Cannot generate app if uncommented...
-  # Uncomment after app is generated to fix view errors
-  # include ::Pagy::Frontend
-```
+* Homebrew: https://wiki.postgresql.org/wiki/Homebrew
+* Docker: https://www.docker.com/blog/how-to-use-the-postgres-docker-official-image/
 
-### Terminal 2 - Seed and spin up server
-```bash
-cd .internal_test_app
-bundle exec rake db:seed
+### Install Template
 
-# JS was breaking in dev mode, had to add the yarn package below - EWL 6/5/23
-yarn add @babel/plugin-proposal-private-methods --dev
+Use Ruby v3.2 and Rails v6.1.7.4 to bootstrap a new GeoBlacklight + GBL Admin application using the template script:
 
-# Run the app server
+```bash
+rails _6.1.7.4_ new gbl_admin -m https://raw.githubusercontent.com/geobtaa/geoblacklight_admin/develop/template.rb
+cd gbl_admin
 bundle exec rake gbl_admin:server
 ```
 
-You're now done generating the test app and populating the Elements / FormElements tables with the basic Aardvark controls.
+You have now generated the .internal_test_app and populated the Elements / FormElements tables for OMG Aardvark support.
 
 ### View App in Browser
 
@@ -59,12 +43,48 @@ You're now done generating the test app and populating the Elements / FormElemen
 2. Click on the "Sign in" link
 3. Enter email: admin@geoblacklight.org and password: 123456
 4. Click on the "GBL Admin" link
-5. Import some CSV
+5. Import some CSV (test/fixtures/files/btaa_sample_records.csv)
 
 -----
 
-### TODOs - Running list of things to accomplish
+## Run Project for Local Development
+Drop and recreate databases (or engine_cart:generate will fail)
 
+### Drop/Create application PG database
+```bash
+psql postgres
+DROP DATABASE geoblacklight_development;
+CREATE DATABASE geoblacklight_development;
+```
+
+```bash
+cd project root
+bundle install
+bundle exec rake engine_cart:regenerate
+```
+
+### Run Solr
+```bash
+bin/rails geoblacklight:solr
+```
+
+### Run App
+```bash
+cd .internal_test_app
+bundle exec rails server
+```
+
+### Lint App
+```bash
+standardrb .
+```
+
+### Test App
+```bash
+RAILS_ENV=test bundle exec rails test
+```
+
+## TODOs
 * ~~SolrWrapper - Add persist option~~
 * ~~BlacklightApi returns not auth'd message (not requiring auth for now (not sensitive data))~~
 * ~~Facet links need /admin nesting~~
@@ -78,31 +98,10 @@ You're now done generating the test app and populating the Elements / FormElemen
 * ~~Routes - Get devise user~~
 * ~~No route matches [GET] "/users/sign_out"~~
 * ~~Bookmarks need to be Admin::Bookmarks~~
-* GitHub Actions / CI integration
-* Port the GEOMG test suite
+* ~~GitHub Actions / CI integration~~
+* ~~Port the GEOMG test suite~~
+* ~~Project gem dependency injection redundancy...~~
+* DRY up Engine routing
 * Remove legacy GEOMG / B1G everywhere...
 * Send GBLADMIN JavaScript pack to NPM like Blacklight
-* Project gem dependency injection redundancy...
-* Likely a lot more polish to be uncovered...
-
-
-# To Run Project
-drop and create databases (or engine_cart will fail)
-cd project root
-bundle exec rake engine_cart:regenerate
-
-## Run Solr
-bin/rails geoblacklight:solr
-
-## Run App
-cd .internal_test_app
-bundle exec rails server
-
-## Test App
-bundle exec rails test
-cd .internal_test_app
-rake db:seed
-rake geomg:solr:reindex
-
-# @TODO: Add chosen.js css
-# @TODO: Fix missing select form-control class
+* Likely some more polish to be uncovered...

data/Rakefile

@@ -24,14 +24,6 @@ RuboCop::RakeTask.new(:rubocop)
 require "solr_wrapper/rake_task"
 require "engine_cart/rake_task"
 require "geoblacklight_admin/version"
-
-desc "Run test suite"
-task "ci" do
-  ENV["RAILS_ENV"] = "test"
-  system("RAILS_ENV=test bundle exec rake test") || false
-  # system("RAILS_ENV=test bundle exec rake geomg:solr:reindex") || false
-end
-
 require "rake/testtask"
 
 # Searches for files ending in _test.rb in the test directory
@@ -45,12 +37,35 @@ end
 # Will use test as defaut task if rake is run by itself
 task default: :test
 
+desc "Run test suite"
+task ci: ["geoblacklight:generate"] do
+  within_test_app do
+    system "RAILS_ENV=test bin/rails db:migrate"
+    system "RAILS_ENV=test rake db:seed"
+    system "RAILS_ENV=test rails webpacker:compile"
+  end
+
+  # Run RSpec tests with Coverage
+  Rake::Task["geoblacklight:coverage"].invoke
+end
+
 namespace :geoblacklight do
+  desc "Run tests with coverage"
+  task :coverage do
+    ENV["COVERAGE"] = "true"
+    # Rake::Task["spec"].invoke
+    Rake::Task["test"].invoke
+  end
+
+  desc "Create the test rails app"
+  task generate: ["engine_cart:generate"] do
+    # Intentionally Empty Block
+  end
+
   namespace :internal do
     task seed: ["engine_cart:generate"] do
       within_test_app do
-        # @TODO - Seed Elements / FormElements
-        # system "bundle exec rake gbl_admin:seed"
+        system "bundle exec rake db:seed"
         system "bundle exec rake geoblacklight:downloads:mkdir"
       end
     end

data/app/assets/javascripts/geoblacklight_admin.js

@@ -13,7 +13,7 @@
 
 //= require jquery3
 //= require rails-ujs
-//= require cocoon
+//= require @nathanvda/cocoon/cocoon.js
 //= require chosen-jquery/lib/chosen.jquery.js
 
 // VENDOR

data/app/assets/stylesheets/geoblacklight_admin/modules/_forms.scss

@@ -26,3 +26,111 @@ label, legend {
 html {
   scroll-padding-top: 4rem; /* height of your sticky header */
 }
+
+// Simple Form / Bootstrap 5 / Select
+//
+// GBL is using BS4 now, but SimpleForm is already on BS5
+// Adding support for BS5's .form-select class until we upgrade to BS5
+$form-select-padding-y:             $input-padding-y;
+$form-select-padding-x:             $input-padding-x;
+$form-select-font-family:           $input-font-family;
+$form-select-font-size:             $input-font-size;
+$form-select-indicator-padding:     $form-select-padding-x * 3; // Extra padding for background-image
+$form-select-font-weight:           $input-font-weight;
+$form-select-line-height:           $input-line-height;
+$form-select-color:                 $input-color;
+$form-select-bg:                    $input-bg;
+$form-select-disabled-color:        null;
+$form-select-disabled-bg:           $gray-200;
+$form-select-disabled-border-color: $gray-800;
+$form-select-bg-position:           right $form-select-padding-x center;
+$form-select-bg-size:               16px 12px; // In pixels because image dimensions
+$form-select-indicator-color:       $gray-800;
+$form-select-indicator:             url("data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16'><path fill='none' stroke='#{$form-select-indicator-color}' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' d='M2 5l6 6 6-6'/></svg>");
+
+$form-select-feedback-icon-padding-end: $form-select-padding-x * 2.5 + $form-select-indicator-padding;
+$form-select-feedback-icon-position:    center right $form-select-indicator-padding;
+$form-select-feedback-icon-size:        $input-height-inner-half $input-height-inner-half;
+
+$form-select-border-width:        $input-border-width;
+$form-select-border-color:        $input-border-color;
+$form-select-border-radius:       $border-radius;
+
+$form-select-focus-border-color:  $input-focus-border-color;
+$form-select-focus-width:         $input-focus-width;
+$form-select-focus-box-shadow:    0 0 0 $form-select-focus-width $input-btn-focus-color;
+
+$form-select-padding-y-sm:        $input-padding-y-sm;
+$form-select-padding-x-sm:        $input-padding-x-sm;
+$form-select-font-size-sm:        $input-font-size-sm;
+
+$form-select-padding-y-lg:        $input-padding-y-lg;
+$form-select-padding-x-lg:        $input-padding-x-lg;
+$form-select-font-size-lg:        $input-font-size-lg;
+
+$form-select-transition:          $input-transition;
+
+.form-select {
+  display: block;
+  width: 100%;
+  padding: $form-select-padding-y $form-select-indicator-padding $form-select-padding-y $form-select-padding-x;
+  // stylelint-disable-next-line property-no-vendor-prefix
+  -moz-padding-start: subtract($form-select-padding-x, 3px); // See https://github.com/twbs/bootstrap/issues/32636
+  font-family: $form-select-font-family;
+  @include font-size($form-select-font-size);
+  font-weight: $form-select-font-weight;
+  line-height: $form-select-line-height;
+  color: $form-select-color;
+  background-color: $form-select-bg;
+  background-image: escape-svg($form-select-indicator);
+  background-repeat: no-repeat;
+  background-position: $form-select-bg-position;
+  background-size: $form-select-bg-size;
+  border: $form-select-border-width solid $form-select-border-color;
+  @include border-radius($form-select-border-radius, 0);
+  @include transition($form-select-transition);
+  appearance: none;
+
+  &:focus {
+    border-color: $form-select-focus-border-color;
+    outline: 0;
+    @if $enable-shadows {
+      @include box-shadow($form-select-box-shadow, $form-select-focus-box-shadow);
+    } @else {
+      // Avoid using mixin so we can pass custom focus shadow properly
+      box-shadow: $form-select-focus-box-shadow;
+    }
+  }
+
+  &[multiple],
+  &[size]:not([size="1"]) {
+    padding-right: $form-select-padding-x;
+    background-image: none;
+  }
+
+  &:disabled {
+    color: $form-select-disabled-color;
+    background-color: $form-select-disabled-bg;
+    border-color: $form-select-disabled-border-color;
+  }
+
+  // Remove outline from select box in FF
+  &:-moz-focusring {
+    color: transparent;
+    text-shadow: 0 0 0 $form-select-color;
+  }
+}
+
+.form-select-sm {
+  padding-top: $form-select-padding-y-sm;
+  padding-bottom: $form-select-padding-y-sm;
+  padding-left: $form-select-padding-x-sm;
+  @include font-size($form-select-font-size-sm);
+}
+
+.form-select-lg {
+  padding-top: $form-select-padding-y-lg;
+  padding-bottom: $form-select-padding-y-lg;
+  padding-left: $form-select-padding-x-lg;
+  @include font-size($form-select-font-size-lg);
+}

data/app/controllers/admin/documents_controller.rb

@@ -10,6 +10,8 @@ module Admin
     # GET /documents
     # GET /documents.json
     def index
+      @request = "#{request.protocol}#{request.host}:#{request.port}"
+
       query_params = {
         q: params["q"],
         f: params["f"],
@@ -18,7 +20,7 @@ module Admin
         sort: params["sort"] || "score desc",
         daterange: params["daterange"] || nil
       }
-      @documents = BlacklightApi.new(**query_params)
+      @documents = BlacklightApi.new(@request, **query_params)
 
       respond_to do |format|
         format.html { render :index }
@@ -26,38 +28,44 @@ module Admin
 
         # JSON - BTAA Aardvark
         format.json_btaa_aardvark do
-          ExportJsonJob.perform_later(current_user, query_params.merge!({format: "json_btaa_aardvark"}),
+          ExportJsonJob.perform_later(@request, current_user, query_params.merge!({format: "json_btaa_aardvark"}),
             ExportJsonService)
           head :no_content
         end
 
         # JSON - GBL Aardvark
         format.json_aardvark do
-          ExportJsonJob.perform_later(current_user, query_params.merge!({format: "json_aardvark"}), ExportJsonService)
+          ExportJsonJob.perform_later(@request, current_user, query_params.merge!({format: "json_aardvark"}), ExportJsonService)
           head :no_content
         end
 
         # JSON - GBL v1
         format.json_gbl_v1 do
-          ExportJsonJob.perform_later(current_user, query_params.merge!({format: "json_gbl_v1"}), ExportJsonService)
+          ExportJsonJob.perform_later(@request, current_user, query_params.merge!({format: "json_gbl_v1"}), ExportJsonService)
+          head :no_content
+        end
+
+        # JSON - FILE
+        format.json_file do
+          ExportJsonBulkJob.perform_later(@request, current_user, query_params.merge!({format: "json_file"}), ExportJsonService)
           head :no_content
         end
 
         # CSV - B1G
         format.csv do
-          ExportJob.perform_later(current_user, query_params, ExportCsvService)
+          ExportJob.perform_later(@request, current_user, query_params, ExportCsvService)
           head :no_content
         end
 
         # CSV Document Downloads - B1G
         format.csv_document_downloads do
-          ExportJob.perform_later(current_user, query_params, ExportCsvDocumentDownloadsService)
+          ExportJob.perform_later(@request, current_user, query_params, ExportCsvDocumentDownloadsService)
           head :no_content
         end
 
         # CSV Document Access Links - B1G
         format.csv_document_access_links do
-          ExportJob.perform_later(current_user, query_params, ExportCsvDocumentAccessLinksService)
+          ExportJob.perform_later(@request, current_user, query_params, ExportCsvDocumentAccessLinksService)
           head :no_content
         end
       end
@@ -65,6 +73,7 @@ module Admin
 
     # Fetch documents from array of friendlier_ids
     def fetch
+      @request = "#{request.protocol}#{request.host}:#{request.port}"
       @documents = Document.where(friendlier_id: params["ids"])
 
       respond_to do |format|
@@ -73,42 +82,42 @@ module Admin
 
         # JSON - BTAA Aardvark
         format.json_btaa_aardvark do
-          ExportJsonJob.perform_later(current_user, {ids: @documents.pluck(:friendlier_id), format: "json_btaa_aardvark"},
+          ExportJsonJob.perform_later(@request, current_user, {ids: @documents.pluck(:friendlier_id), format: "json_btaa_aardvark"},
             ExportJsonService)
           head :no_content
         end
 
         # JSON - GBL Aardvark
         format.json_aardvark do
-          ExportJsonJob.perform_later(current_user, {ids: @documents.pluck(:friendlier_id), format: "json_aardvark"},
+          ExportJsonJob.perform_later(@request, current_user, {ids: @documents.pluck(:friendlier_id), format: "json_aardvark"},
             ExportJsonService)
           head :no_content
         end
 
         # JSON - GBL v1
         format.json_gbl_v1 do
-          ExportJsonJob.perform_later(current_user, {ids: @documents.pluck(:friendlier_id), format: "json_gbl_v1"},
+          ExportJsonJob.perform_later(@request, current_user, {ids: @documents.pluck(:friendlier_id), format: "json_gbl_v1"},
             ExportJsonService)
           head :no_content
         end
 
         # CSV - B1G
         format.csv do
-          ExportJob.perform_later(current_user, {ids: @documents.pluck(:friendlier_id), format: "csv"},
+          ExportJob.perform_later(@request, current_user, {ids: @documents.pluck(:friendlier_id), format: "csv"},
             ExportCsvService)
           head :no_content
         end
 
         # CSV Document Downloads - B1G
         format.csv_document_downloads do
-          ExportJob.perform_later(current_user,
+          ExportJob.perform_later(@request, current_user,
             {ids: @documents.pluck(:friendlier_id), format: "csv_document_downloads"}, ExportCsvDocumentDownloadsService)
           head :no_content
         end
 
         # CSV Document Downloads - B1G
         format.csv_document_access_links do
-          ExportJob.perform_later(current_user,
+          ExportJob.perform_later(@request, current_user,
             {ids: @documents.pluck(:friendlier_id), format: "csv_document_access_links"}, ExportCsvDocumentAccessLinksService)
           head :no_content
         end

data/app/controllers/admin/search_controller.rb

@@ -4,7 +4,8 @@
 module Admin
   class SearchController < Admin::AdminController
     def index
-      @facet_options = BlacklightApiFacets.new.facets
+      @request = "#{request.protocol}#{request.host}:#{request.port}"
+      @facet_options = BlacklightApiFacets.new(@request).facets
     end
   end
 end

data/app/controllers/admin/users_controller.rb

@@ -4,7 +4,7 @@
 module Admin
   class UsersController < Admin::AdminController
     def index
-      @users = User.all
+      @users = User.where(admin: true)
     end
   end
 end

data/app/indexers/document_indexer.rb

@@ -28,7 +28,7 @@ class DocumentIndexer < Kithe::Indexer
       rec.updated_at.utc.iso8601 if rec&.updated_at
     end
 
-    # - GEOMG
+    # - GBL ADMIN
     to_field "b1g_geom_import_id_ssi", obj_extract("import_id")
   end
 end

data/app/jobs/export_job.rb

@@ -6,8 +6,9 @@ require "csv"
 class ExportJob < ApplicationJob
   queue_as :default
 
-  def perform(current_user, query_params, export_service)
+  def perform(request, current_user, query_params, export_service)
     logger.debug("\n\n Background Job: ♞")
+    logger.debug("Request: #{request.inspect}")
     logger.debug("User: #{current_user.inspect}")
     logger.debug("Query: #{query_params.inspect}")
     logger.debug("Export Service: #{export_service.inspect}")
@@ -17,7 +18,7 @@ class ExportJob < ApplicationJob
     ActionCable.server.broadcast("export_channel", {data: "Hello from Export Job!"})
 
     # Query params into Doc ids
-    document_ids = query_params[:ids] || crawl_query(query_params)
+    document_ids = query_params[:ids] || crawl_query(request, query_params)
 
     logger.debug("Document Ids: #{document_ids}")
 
@@ -56,15 +57,16 @@ class ExportJob < ApplicationJob
     })
   end
 
-  def crawl_query(query_params, doc_ids = [])
+  def crawl_query(request, query_params, doc_ids = [])
     logger.debug("\n\n CRAWL Query: #{query_params}")
-    api_results = BlacklightApiIds.new(query_params)
+    logger.debug("\n\n CRAWL Query Request: #{request}")
+    api_results = BlacklightApiIds.new(request, query_params)
     logger.debug("API Results: #{api_results.results.inspect}")
 
     doc_ids << api_results.results.pluck("id")
 
     unless api_results.meta["pages"]["next_page"].nil?
-      crawl_query(query_params.merge!({page: api_results.meta["pages"]["next_page"]}),
+      crawl_query(request, query_params.merge!({page: api_results.meta["pages"]["next_page"]}),
         doc_ids)
     end
 

data/app/jobs/export_json_bulk_job.rb

@@ -0,0 +1,95 @@
+# frozen_string_literal: true
+
+require "csv"
+
+# ExportJsonBulkJob
+class ExportJsonBulkJob < ApplicationJob
+  queue_as :default
+
+  def perform(request, current_user, query_params, export_service)
+    logger.debug("\n\n Background Job: ♞")
+    logger.debug("Request: #{request.inspect}")
+    logger.debug("User: #{current_user.inspect}")
+    logger.debug("Query: #{query_params.inspect}")
+    logger.debug("Export Service: #{export_service.inspect}")
+    logger.debug("\n\n")
+
+    # Test broadcast
+    ActionCable.server.broadcast("export_channel", {data: "Hello from Export Job!"})
+
+    # Query params into Doc ids
+    document_ids = query_params[:ids] || crawl_query(request, query_params)
+
+    logger.debug("Document Ids: #{document_ids}")
+
+    # Send progress
+    documents = export_service.call(document_ids)
+
+    begin
+      # Array of JSON
+      @json_array = []
+
+      documents.each do |doc|
+        json_output = Admin::DocumentsController.render("_json_file.jbuilder",
+          locals: {document: doc})
+
+        json_obj = JSON.parse(json_output)
+        Rails.logger.debug json_obj
+
+        # Remove nil/null values from JSON
+        json_obj.compact!
+
+        @json_array << JSON.pretty_generate(json_obj)
+
+      rescue NoMethodError => e
+        Rails.logger.debug { "==== Error! - #{doc.friendlier_id} ====" }
+        Rails.logger.debug e.inspect
+        next
+      end
+    end
+
+    # Write into tempfile
+    @tempfile = Tempfile.new(["export-#{Time.zone.today}", ".json"]).tap do |file|
+      file.write("[#{@json_array.join(",")}]")
+    end
+    @tempfile.rewind
+
+    # Create notification
+    # Message: "Download Type|Row Count|Button Label"
+    notification = ExportNotification.with(message: "JSON FILE |#{ActionController::Base.helpers.number_with_delimiter(documents.size)} rows|JSON")
+
+    # Deliver notification
+    notification.deliver(current_user)
+
+    # Attach JSON file (can only attach after persisted)
+    notification.record.file.attach(io: @tempfile, filename: "geomg-export-#{Time.zone.today}.json",
+      content_type: "application/json")
+
+    # Update UI
+    ActionCable.server.broadcast("export_channel", {
+      data: "Notification ready!",
+      actions: [
+        {
+          method: "RefreshNotifications",
+          payload: current_user.notifications.unread.count
+        }
+      ]
+    })
+  end
+
+  def crawl_query(request, query_params, doc_ids = [])
+    logger.debug("\n\n CRAWL Query: #{query_params}")
+    logger.debug("\n\n CRAWL Query Request: #{request}")
+    api_results = BlacklightApiIds.new(request, query_params)
+    logger.debug("API Results: #{api_results.results.inspect}")
+
+    doc_ids << api_results.results.pluck("id")
+
+    unless api_results.meta["pages"]["next_page"].nil?
+      crawl_query(request, query_params.merge!({page: api_results.meta["pages"]["next_page"]}),
+        doc_ids)
+    end
+
+    doc_ids
+  end
+end