fetch_util 0.7.0 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (69) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +27 -0
  3. data/lib/fetch_util/assets/extract.js +1 -1
  4. data/lib/fetch_util/browser/site_stabilization/communities/reddit.rb +71 -0
  5. data/lib/fetch_util/browser/site_stabilization/communities.rb +13 -0
  6. data/lib/fetch_util/browser/site_stabilization/{bitbucket_cloud_pull_resource_modules.rb → forges/bitbucket/pulls/resource_modules.rb} +5 -5
  7. data/lib/fetch_util/browser/site_stabilization/{bitbucket_cloud_pull_resources.rb → forges/bitbucket/pulls/resources.rb} +3 -34
  8. data/lib/fetch_util/browser/site_stabilization/{bitbucket_cloud_threads.rb → forges/bitbucket/threads/index.rb} +3 -34
  9. data/lib/fetch_util/browser/site_stabilization/forges/gitea/stabilization.rb +31 -0
  10. data/lib/fetch_util/browser/site_stabilization/forges/observation_state.rb +59 -0
  11. data/lib/fetch_util/browser/site_stabilization/forges.rb +109 -0
  12. data/lib/fetch_util/browser/site_stabilization/marketplaces/ebay.rb +56 -0
  13. data/lib/fetch_util/browser/site_stabilization/marketplaces.rb +13 -0
  14. data/lib/fetch_util/browser/site_stabilization/{facebook_stabilization.rb → social/facebook.rb} +7 -0
  15. data/lib/fetch_util/browser/site_stabilization/{social_platforms.rb → social/instagram.rb} +6 -8
  16. data/lib/fetch_util/browser/site_stabilization/social.rb +15 -0
  17. data/lib/fetch_util/browser/site_stabilization/{travel_and_lodging.rb → travel/lodging.rb} +7 -9
  18. data/lib/fetch_util/browser/site_stabilization/travel.rb +13 -0
  19. data/lib/fetch_util/browser/site_stabilization.rb +10 -97
  20. data/lib/fetch_util/browser/stabilization/page_flow.rb +7 -5
  21. data/lib/fetch_util/browser.rb +3 -0
  22. data/lib/fetch_util/raw_docs_fallback.rb +5 -1
  23. data/lib/fetch_util/regulatory/cache_store.rb +5 -3
  24. data/lib/fetch_util/regulatory/cache_validation.rb +24 -0
  25. data/lib/fetch_util/regulatory/tdm_policy.rb +1 -1
  26. data/lib/fetch_util/regulatory.rb +1 -0
  27. data/lib/fetch_util/version.rb +1 -1
  28. metadata +56 -48
  29. data/lib/fetch_util/browser/site_stabilization/community_and_marketplace.rb +0 -117
  30. data/lib/fetch_util/browser/site_stabilization/gitea_family_stabilization.rb +0 -60
  31. /data/lib/fetch_util/browser/site_stabilization/{azure_devops_pr_commit_details_script.rb → forges/azure_devops/pull_requests/commit_details_script.rb} +0 -0
  32. /data/lib/fetch_util/browser/site_stabilization/{azure_devops_pr_product_state_script.rb → forges/azure_devops/pull_requests/product_state_script.rb} +0 -0
  33. /data/lib/fetch_util/browser/site_stabilization/{azure_devops_pr_request_helpers_script.rb → forges/azure_devops/pull_requests/request_helpers_script.rb} +0 -0
  34. /data/lib/fetch_util/browser/site_stabilization/{azure_devops_pr_request_state_script.rb → forges/azure_devops/pull_requests/request_state_script.rb} +0 -0
  35. /data/lib/fetch_util/browser/site_stabilization/{azure_devops_pr_stabilization.rb → forges/azure_devops/pull_requests/stabilization.rb} +0 -0
  36. /data/lib/fetch_util/browser/site_stabilization/{azure_devops_pr_state.rb → forges/azure_devops/pull_requests/state.rb} +0 -0
  37. /data/lib/fetch_util/browser/site_stabilization/{bitbucket_cloud_pull_diff_product_state.rb → forges/bitbucket/pulls/diff/product_state.rb} +0 -0
  38. /data/lib/fetch_util/browser/site_stabilization/{bitbucket_cloud_pull_diff_request_state.rb → forges/bitbucket/pulls/diff/request_state.rb} +0 -0
  39. /data/lib/fetch_util/browser/site_stabilization/{bitbucket_cloud_pull_diff_state.rb → forges/bitbucket/pulls/diff/state.rb} +0 -0
  40. /data/lib/fetch_util/browser/site_stabilization/{bitbucket_cloud_pull_resource_state.rb → forges/bitbucket/pulls/resource_state.rb} +0 -0
  41. /data/lib/fetch_util/browser/site_stabilization/{bitbucket_cloud_thread_state.rb → forges/bitbucket/threads/state.rb} +0 -0
  42. /data/lib/fetch_util/browser/site_stabilization/{gerrit_change_product_state_script.rb → forges/gerrit/changes/product_state_script.rb} +0 -0
  43. /data/lib/fetch_util/browser/site_stabilization/{gerrit_change_request_state_script.rb → forges/gerrit/changes/request_state_script.rb} +0 -0
  44. /data/lib/fetch_util/browser/site_stabilization/{gerrit_change_stabilization.rb → forges/gerrit/changes/stabilization.rb} +0 -0
  45. /data/lib/fetch_util/browser/site_stabilization/{gerrit_change_state.rb → forges/gerrit/changes/state.rb} +0 -0
  46. /data/lib/fetch_util/browser/site_stabilization/{gerrit_file_resource_fetch_state_script.rb → forges/gerrit/resources/fetch_state_script.rb} +0 -0
  47. /data/lib/fetch_util/browser/site_stabilization/{gerrit_file_resource_product_state_script.rb → forges/gerrit/resources/product_state_script.rb} +0 -0
  48. /data/lib/fetch_util/browser/site_stabilization/{gerrit_file_resource_request_state_script.rb → forges/gerrit/resources/request_state_script.rb} +0 -0
  49. /data/lib/fetch_util/browser/site_stabilization/{gerrit_file_resource_route_state_script.rb → forges/gerrit/resources/route_state_script.rb} +0 -0
  50. /data/lib/fetch_util/browser/site_stabilization/{gerrit_file_resource_stabilization.rb → forges/gerrit/resources/stabilization.rb} +0 -0
  51. /data/lib/fetch_util/browser/site_stabilization/{gerrit_file_resource_state.rb → forges/gerrit/resources/state.rb} +0 -0
  52. /data/lib/fetch_util/browser/site_stabilization/{gerrit_file_resource_validation_state_script.rb → forges/gerrit/resources/validation_state_script.rb} +0 -0
  53. /data/lib/fetch_util/browser/site_stabilization/{gitea_family_pull_resource_product_state.rb → forges/gitea/pulls/product_state.rb} +0 -0
  54. /data/lib/fetch_util/browser/site_stabilization/{gitea_family_pull_resources.rb → forges/gitea/pulls/resources.rb} +0 -0
  55. /data/lib/fetch_util/browser/site_stabilization/{gitea_family_pull_resource_state.rb → forges/gitea/pulls/state.rb} +0 -0
  56. /data/lib/fetch_util/browser/site_stabilization/{gitea_family_threads.rb → forges/gitea/threads/index.rb} +0 -0
  57. /data/lib/fetch_util/browser/site_stabilization/{gitea_family_thread_product_state.rb → forges/gitea/threads/product_state.rb} +0 -0
  58. /data/lib/fetch_util/browser/site_stabilization/{gitea_family_thread_timeline_state.rb → forges/gitea/threads/timeline_state.rb} +0 -0
  59. /data/lib/fetch_util/browser/site_stabilization/{gitea_family_visibility_script.rb → forges/gitea/visibility_script.rb} +0 -0
  60. /data/lib/fetch_util/browser/site_stabilization/{github_pull_resource_stabilization.rb → forges/github/pull_resource_stabilization.rb} +0 -0
  61. /data/lib/fetch_util/browser/site_stabilization/{github_pull_resources.rb → forges/github/pull_resources.rb} +0 -0
  62. /data/lib/fetch_util/browser/site_stabilization/{github_threads.rb → forges/github/threads.rb} +0 -0
  63. /data/lib/fetch_util/browser/site_stabilization/{gitlab_merge_request_resource_state.rb → forges/gitlab/merge_requests/resource_state.rb} +0 -0
  64. /data/lib/fetch_util/browser/site_stabilization/{gitlab_merge_request_resource_state_script.rb → forges/gitlab/merge_requests/resource_state_script.rb} +0 -0
  65. /data/lib/fetch_util/browser/site_stabilization/{gitlab_merge_request_resource_visibility_script.rb → forges/gitlab/merge_requests/resource_visibility_script.rb} +0 -0
  66. /data/lib/fetch_util/browser/site_stabilization/{gitlab_merge_request_resources.rb → forges/gitlab/merge_requests/resources.rb} +0 -0
  67. /data/lib/fetch_util/browser/site_stabilization/{gitlab_repo.rb → forges/gitlab/repository.rb} +0 -0
  68. /data/lib/fetch_util/browser/site_stabilization/{gitlab_stabilization.rb → forges/gitlab/stabilization.rb} +0 -0
  69. /data/lib/fetch_util/browser/site_stabilization/{gitlab_threads.rb → forges/gitlab/threads.rb} +0 -0
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 579a3f35c401910fa8fb6713370113b8158b170438716be04b224542fd19b46d
4
- data.tar.gz: 80c3444cfe7cf2ad43ebaa47a46ec0ffb28fb12915c188672beba142e30bfd6a
3
+ metadata.gz: efdb71da51e7961572bc62718b2936ac971de3433778654c00f0d154610da7f1
4
+ data.tar.gz: c26c226218f84e2e9f666b050fe8b536cdd95b8edff7a66b01227145b0d4c3f9
5
5
  SHA512:
6
- metadata.gz: 54999188b4a18ae81c17733689c7f050e85b4d98dd30b5f65933d17745b8e399add16972d10186edd3bc3d5bce8438a06a72373ce61739f87d9293800c787b42
7
- data.tar.gz: e1c52a6939a13a9e1c70d310c0dfb4ce206c7f4161979d4ca5c8fa35b0715105d985a43cd30aa1e0874ab66713bd58617e992761836c81374db063048dc43f39
6
+ metadata.gz: ab18dc0611e5acda138a795c634634ee96f20555bb958d8024f4796b347e69b19414db343490208bbaeb78a66bcc4a356e5907bceff23b6dcefb49d7038d3d1a
7
+ data.tar.gz: 2f3b05a8846238e92e3abaee4662677f7d4a9908b515d4e4faecd7a6d4d110387aae7e66ae7d222a0b7c74892c7945aa20aa72026aed9e30d982d349741ac0ae
data/CHANGELOG.md CHANGED
@@ -2,6 +2,33 @@
2
2
 
3
3
  ## Unreleased
4
4
 
5
+ ## v0.8.0 - 2026-09-24
6
+
7
+ ### Changed
8
+
9
+ - Separate host-agnostic forge extraction and browser stabilization into product-owned families, leaving forums, Q&A, social-news, and wiki profiles under `profiles/community/` without changing runtime behavior.
10
+ - Mirror forge, community, and social ownership in integration specs and fixtures, replace mixed site-stabilization files with explicit family loaders, and protect the existing method ownership and stabilization phase order with executable contracts.
11
+ - Group article extraction, homepage/list strategies, generic list records and sections, dynamic collection recovery, and Markdown helpers into cohesive source directories without changing runtime bundle order or behavior.
12
+ - Infer visible social threads, feeds, posts, and public profiles through shared structural ownership, retaining every materialized record and nested reply in source order without presentation-count caps.
13
+ - Retire redundant or synthetic host handlers only where shared extraction proves better on the same page; restore six focused article handlers after broader frozen-page testing exposed lost bodies, paywall teasers, or focal liveblog ownership.
14
+
15
+ ### Fixed
16
+
17
+ - Preserve source-owned article sections, author and publication context, image captions, supplemental resources, public paywall teasers, and complete code examples when generic reader or list arbitration would omit them.
18
+ - Retain all visible records in directories, numbered collections, image cards, community feeds, and legal indexes without replacing substantial articles with unrelated portal links.
19
+ - Keep Markdown links distinct by destination, escape generated link labels correctly, and retain complete RawDocs leaf fragments and validated regulatory cache payloads.
20
+ - Distinguish public content from cookie, login, verification, access-error, and empty browser shells; preserve visible denial diagnostics and avoid fabricated content or misleading access warnings.
21
+ - Prefer page-owned SPA data and source-corroborated article headlines and excerpts while excluding related rails, inline controls, and hidden challenge text.
22
+ - Reject blank navigation results and opaque URLs during browser stabilization instead of allowing empty acquisitions or GitLab-thread predicate errors.
23
+
24
+ ### Performance
25
+
26
+ - Reuse extraction-local structured-data snapshots and skip speculative list scans for complete source-owned code-reference articles without raising timeouts or truncating code samples.
27
+
28
+ ### Known Limitations
29
+
30
+ - An immutable 600-website comparison with the local v0.7.0 release found 54 reviewed improvements, 496 byte-identical ties, 30 semantic ties, 17 mixed outcomes, one baseline-better page, one v0.7.0-only extraction failure, and one page unavailable in every version. Strict no-regression acceptance did not pass; the mixed and unavailable cases are not counted as wins.
31
+
5
32
  ## v0.7.0 - 2026-09-20
6
33
 
7
34
  ### Changed