source_monitor 0.1.3 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. checksums.yaml +4 -4
  2. data/.gitignore +1 -0
  3. data/CHANGELOG.md +22 -0
  4. data/Gemfile.lock +2 -2
  5. data/app/assets/builds/source_monitor/application.css +2076 -0
  6. data/app/assets/builds/source_monitor/application.js +2758 -0
  7. data/app/assets/builds/source_monitor/application.js.map +7 -0
  8. data/app/assets/javascripts/source_monitor/application.js +4 -0
  9. data/app/assets/javascripts/source_monitor/controllers/confirm_navigation_controller.js +49 -0
  10. data/app/assets/javascripts/source_monitor/controllers/select_all_controller.js +36 -0
  11. data/app/controllers/source_monitor/import_sessions_controller.rb +791 -0
  12. data/app/controllers/source_monitor/sources_controller.rb +5 -36
  13. data/app/helpers/source_monitor/application_helper.rb +17 -0
  14. data/app/jobs/source_monitor/import_opml_job.rb +150 -0
  15. data/app/jobs/source_monitor/import_session_health_check_job.rb +93 -0
  16. data/app/models/source_monitor/import_history.rb +35 -0
  17. data/app/models/source_monitor/import_session.rb +34 -0
  18. data/app/views/source_monitor/import_sessions/_header.html.erb +12 -0
  19. data/app/views/source_monitor/import_sessions/_sidebar.html.erb +23 -0
  20. data/app/views/source_monitor/import_sessions/health_check/_progress.html.erb +20 -0
  21. data/app/views/source_monitor/import_sessions/health_check/_row.html.erb +44 -0
  22. data/app/views/source_monitor/import_sessions/show.html.erb +15 -0
  23. data/app/views/source_monitor/import_sessions/show.turbo_stream.erb +1 -0
  24. data/app/views/source_monitor/import_sessions/steps/_configure.html.erb +53 -0
  25. data/app/views/source_monitor/import_sessions/steps/_confirm.html.erb +121 -0
  26. data/app/views/source_monitor/import_sessions/steps/_health_check.html.erb +82 -0
  27. data/app/views/source_monitor/import_sessions/steps/_navigation.html.erb +29 -0
  28. data/app/views/source_monitor/import_sessions/steps/_preview.html.erb +172 -0
  29. data/app/views/source_monitor/import_sessions/steps/_upload.html.erb +42 -0
  30. data/app/views/source_monitor/sources/_form.html.erb +8 -138
  31. data/app/views/source_monitor/sources/_form_fields.html.erb +142 -0
  32. data/app/views/source_monitor/sources/_import_history_panel.html.erb +53 -0
  33. data/app/views/source_monitor/sources/index.html.erb +7 -1
  34. data/config/coverage_baseline.json +91 -15
  35. data/config/routes.rb +6 -0
  36. data/db/migrate/20251124090000_create_import_sessions.rb +18 -0
  37. data/db/migrate/20251124153000_add_health_fields_to_import_sessions.rb +14 -0
  38. data/db/migrate/20251125094500_create_import_histories.rb +19 -0
  39. data/lib/source_monitor/health/import_source_health_check.rb +55 -0
  40. data/lib/source_monitor/health.rb +1 -0
  41. data/lib/source_monitor/import_sessions/entry_normalizer.rb +30 -0
  42. data/lib/source_monitor/import_sessions/health_check_broadcaster.rb +103 -0
  43. data/lib/source_monitor/sources/params.rb +52 -0
  44. data/lib/source_monitor/version.rb +1 -1
  45. data/tasks/completed/codebase_audit_2025.md +1396 -0
  46. data/tasks/completed/engine-asset-configuration.md +203 -0
  47. data/tasks/completed/opml-import-wizard/opml-import-wizard-product-brief.md +58 -0
  48. data/tasks/completed/opml-import-wizard/opml-import-wizard-tech-brief.md +75 -0
  49. data/tasks/completed/opml-import-wizard/task-01/instructions.md +81 -0
  50. data/tasks/completed/opml-import-wizard/task-01/requirements.md +19 -0
  51. data/tasks/completed/opml-import-wizard/task-02/instructions.md +83 -0
  52. data/tasks/completed/opml-import-wizard/task-02/requirements.md +18 -0
  53. data/tasks/completed/opml-import-wizard/task-03/instructions.md +58 -0
  54. data/tasks/completed/opml-import-wizard/task-03/requirements.md +18 -0
  55. data/tasks/completed/opml-import-wizard/task-04/instructions.md +84 -0
  56. data/tasks/completed/opml-import-wizard/task-04/requirements.md +17 -0
  57. data/tasks/completed/opml-import-wizard/task-05/instructions.md +50 -0
  58. data/tasks/completed/opml-import-wizard/task-05/requirements.md +17 -0
  59. data/tasks/completed/opml-import-wizard/task-06/instructions.md +92 -0
  60. data/tasks/completed/opml-import-wizard/task-06/requirements.md +21 -0
  61. data/tasks/completed/phase_17_01_complexity_audit_2025-10-12.md +62 -0
  62. data/tasks/completed/phase_17_02_complexity_findings_2025-10-12.md +74 -0
  63. data/tasks/completed/phase_17_03_refactor_plan_2025-10-12.md +37 -0
  64. data/tasks/completed/phase_21_01_log_consolidation_2025-10-15.md +30 -0
  65. data/tasks/completed/release_checklist.md +23 -0
  66. data/tasks/completed/routes_refactor_evaluation.md +109 -0
  67. data/tasks/completed/source_monitor_rename_plan.md +70 -0
  68. data/tasks/completed/tasks.md +952 -0
  69. data/tasks/ideas.md +10 -0
  70. metadata +59 -3
  71. /data/tasks/{prd-setup-workflow-streamlining.md → completed/prd-setup-workflow-streamlining.md} +0 -0
  72. /data/tasks/{tasks-setup-workflow-streamlining.md → completed/tasks-setup-workflow-streamlining.md} +0 -0
data/tasks/ideas.md ADDED
@@ -0,0 +1,10 @@
1
+ - [ ] job queues stats cards needs to be rethought
2
+ - [X] ~~*add opml import/export*~~ [2025-11-25]
3
+ - [ ] clean upgrade strategy
4
+ - [ ] growl notifications are spamming the user when taking bulk actions
5
+ - [ ] add ability to manage per-source scraping rate limits to override configuration defaults
6
+ - [ ] ensure adaptive scheduling only reacts to truly new content (fetching a feed should use hashing on the summary or scrape to determine the item has really been updated, otherwise don't update anything)
7
+ - [ ] automatically extract transcripts from content that is mostly video or audio
8
+ - [ ] explore using this for markdown conversion (could be optional adapter for sources): https://github.com/Goldziher/html-to-markdown/blob/main/packages%2Fruby%2FREADME.md
9
+ - [ ] sources show page needs pagination for items
10
+ - [ ] sources show page should truncate item content to first 100 characters (with a stimulus expand link under the truncated content to show full body)
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: source_monitor
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.3
4
+ version: 0.2.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - dchuk
@@ -250,13 +250,18 @@ files:
250
250
  - README.md
251
251
  - Rakefile
252
252
  - app/assets/builds/.keep
253
+ - app/assets/builds/source_monitor/application.css
254
+ - app/assets/builds/source_monitor/application.js
255
+ - app/assets/builds/source_monitor/application.js.map
253
256
  - app/assets/config/source_monitor_manifest.js
254
257
  - app/assets/images/source_monitor/.keep
255
258
  - app/assets/javascripts/source_monitor/application.js
256
259
  - app/assets/javascripts/source_monitor/controllers/async_submit_controller.js
260
+ - app/assets/javascripts/source_monitor/controllers/confirm_navigation_controller.js
257
261
  - app/assets/javascripts/source_monitor/controllers/dropdown_controller.js
258
262
  - app/assets/javascripts/source_monitor/controllers/modal_controller.js
259
263
  - app/assets/javascripts/source_monitor/controllers/notification_controller.js
264
+ - app/assets/javascripts/source_monitor/controllers/select_all_controller.js
260
265
  - app/assets/javascripts/source_monitor/turbo_actions.js
261
266
  - app/assets/stylesheets/source_monitor/application.tailwind.css
262
267
  - app/assets/svgs/source_monitor/.keep
@@ -266,6 +271,7 @@ files:
266
271
  - app/controllers/source_monitor/dashboard_controller.rb
267
272
  - app/controllers/source_monitor/fetch_logs_controller.rb
268
273
  - app/controllers/source_monitor/health_controller.rb
274
+ - app/controllers/source_monitor/import_sessions_controller.rb
269
275
  - app/controllers/source_monitor/items_controller.rb
270
276
  - app/controllers/source_monitor/logs_controller.rb
271
277
  - app/controllers/source_monitor/scrape_logs_controller.rb
@@ -279,6 +285,8 @@ files:
279
285
  - app/helpers/source_monitor/application_helper.rb
280
286
  - app/jobs/source_monitor/application_job.rb
281
287
  - app/jobs/source_monitor/fetch_feed_job.rb
288
+ - app/jobs/source_monitor/import_opml_job.rb
289
+ - app/jobs/source_monitor/import_session_health_check_job.rb
282
290
  - app/jobs/source_monitor/item_cleanup_job.rb
283
291
  - app/jobs/source_monitor/log_cleanup_job.rb
284
292
  - app/jobs/source_monitor/schedule_fetches_job.rb
@@ -290,6 +298,8 @@ files:
290
298
  - app/models/source_monitor/application_record.rb
291
299
  - app/models/source_monitor/fetch_log.rb
292
300
  - app/models/source_monitor/health_check_log.rb
301
+ - app/models/source_monitor/import_history.rb
302
+ - app/models/source_monitor/import_session.rb
293
303
  - app/models/source_monitor/item.rb
294
304
  - app/models/source_monitor/item_content.rb
295
305
  - app/models/source_monitor/log_entry.rb
@@ -303,6 +313,18 @@ files:
303
313
  - app/views/source_monitor/dashboard/_stats.html.erb
304
314
  - app/views/source_monitor/dashboard/index.html.erb
305
315
  - app/views/source_monitor/fetch_logs/show.html.erb
316
+ - app/views/source_monitor/import_sessions/_header.html.erb
317
+ - app/views/source_monitor/import_sessions/_sidebar.html.erb
318
+ - app/views/source_monitor/import_sessions/health_check/_progress.html.erb
319
+ - app/views/source_monitor/import_sessions/health_check/_row.html.erb
320
+ - app/views/source_monitor/import_sessions/show.html.erb
321
+ - app/views/source_monitor/import_sessions/show.turbo_stream.erb
322
+ - app/views/source_monitor/import_sessions/steps/_configure.html.erb
323
+ - app/views/source_monitor/import_sessions/steps/_confirm.html.erb
324
+ - app/views/source_monitor/import_sessions/steps/_health_check.html.erb
325
+ - app/views/source_monitor/import_sessions/steps/_navigation.html.erb
326
+ - app/views/source_monitor/import_sessions/steps/_preview.html.erb
327
+ - app/views/source_monitor/import_sessions/steps/_upload.html.erb
306
328
  - app/views/source_monitor/items/_details.html.erb
307
329
  - app/views/source_monitor/items/_details_wrapper.html.erb
308
330
  - app/views/source_monitor/items/index.html.erb
@@ -317,7 +339,9 @@ files:
317
339
  - app/views/source_monitor/sources/_empty_state_row.html.erb
318
340
  - app/views/source_monitor/sources/_fetch_interval_heatmap.html.erb
319
341
  - app/views/source_monitor/sources/_form.html.erb
342
+ - app/views/source_monitor/sources/_form_fields.html.erb
320
343
  - app/views/source_monitor/sources/_health_status_badge.html.erb
344
+ - app/views/source_monitor/sources/_import_history_panel.html.erb
321
345
  - app/views/source_monitor/sources/_row.html.erb
322
346
  - app/views/source_monitor/sources/edit.html.erb
323
347
  - app/views/source_monitor/sources/index.html.erb
@@ -348,6 +372,9 @@ files:
348
372
  - db/migrate/20251015100000_create_source_monitor_log_entries.rb
349
373
  - db/migrate/20251022100000_create_source_monitor_health_check_logs.rb
350
374
  - db/migrate/20251108120116_refresh_fetch_status_constraint.rb
375
+ - db/migrate/20251124090000_create_import_sessions.rb
376
+ - db/migrate/20251124153000_add_health_fields_to_import_sessions.rb
377
+ - db/migrate/20251125094500_create_import_histories.rb
351
378
  - docs/configuration.md
352
379
  - docs/deployment.md
353
380
  - docs/gh-cli-workflow.md
@@ -384,10 +411,13 @@ files:
384
411
  - lib/source_monitor/fetching/retry_policy.rb
385
412
  - lib/source_monitor/fetching/stalled_fetch_reconciler.rb
386
413
  - lib/source_monitor/health.rb
414
+ - lib/source_monitor/health/import_source_health_check.rb
387
415
  - lib/source_monitor/health/source_health_check.rb
388
416
  - lib/source_monitor/health/source_health_monitor.rb
389
417
  - lib/source_monitor/health/source_health_reset.rb
390
418
  - lib/source_monitor/http.rb
419
+ - lib/source_monitor/import_sessions/entry_normalizer.rb
420
+ - lib/source_monitor/import_sessions/health_check_broadcaster.rb
391
421
  - lib/source_monitor/instrumentation.rb
392
422
  - lib/source_monitor/items/item_creator.rb
393
423
  - lib/source_monitor/items/retention_pruner.rb
@@ -446,6 +476,7 @@ files:
446
476
  - lib/source_monitor/setup/verification/solid_queue_verifier.rb
447
477
  - lib/source_monitor/setup/verification/telemetry_logger.rb
448
478
  - lib/source_monitor/setup/workflow.rb
479
+ - lib/source_monitor/sources/params.rb
449
480
  - lib/source_monitor/sources/turbo_stream_presenter.rb
450
481
  - lib/source_monitor/turbo_streams/stream_responder.rb
451
482
  - lib/source_monitor/version.rb
@@ -459,8 +490,33 @@ files:
459
490
  - postcss.config.js
460
491
  - source_monitor.gemspec
461
492
  - stylelint.config.js
462
- - tasks/prd-setup-workflow-streamlining.md
463
- - tasks/tasks-setup-workflow-streamlining.md
493
+ - tasks/completed/codebase_audit_2025.md
494
+ - tasks/completed/engine-asset-configuration.md
495
+ - tasks/completed/opml-import-wizard/opml-import-wizard-product-brief.md
496
+ - tasks/completed/opml-import-wizard/opml-import-wizard-tech-brief.md
497
+ - tasks/completed/opml-import-wizard/task-01/instructions.md
498
+ - tasks/completed/opml-import-wizard/task-01/requirements.md
499
+ - tasks/completed/opml-import-wizard/task-02/instructions.md
500
+ - tasks/completed/opml-import-wizard/task-02/requirements.md
501
+ - tasks/completed/opml-import-wizard/task-03/instructions.md
502
+ - tasks/completed/opml-import-wizard/task-03/requirements.md
503
+ - tasks/completed/opml-import-wizard/task-04/instructions.md
504
+ - tasks/completed/opml-import-wizard/task-04/requirements.md
505
+ - tasks/completed/opml-import-wizard/task-05/instructions.md
506
+ - tasks/completed/opml-import-wizard/task-05/requirements.md
507
+ - tasks/completed/opml-import-wizard/task-06/instructions.md
508
+ - tasks/completed/opml-import-wizard/task-06/requirements.md
509
+ - tasks/completed/phase_17_01_complexity_audit_2025-10-12.md
510
+ - tasks/completed/phase_17_02_complexity_findings_2025-10-12.md
511
+ - tasks/completed/phase_17_03_refactor_plan_2025-10-12.md
512
+ - tasks/completed/phase_21_01_log_consolidation_2025-10-15.md
513
+ - tasks/completed/prd-setup-workflow-streamlining.md
514
+ - tasks/completed/release_checklist.md
515
+ - tasks/completed/routes_refactor_evaluation.md
516
+ - tasks/completed/source_monitor_rename_plan.md
517
+ - tasks/completed/tasks-setup-workflow-streamlining.md
518
+ - tasks/completed/tasks.md
519
+ - tasks/ideas.md
464
520
  homepage: https://github.com/dchuk/source_monitor
465
521
  licenses:
466
522
  - MIT