WebSearcher 0.7.0__tar.gz → 0.7.2__tar.gz

websearcher-0.7.2/.github/dependabot.yml

@@ -0,0 +1,10 @@
+version: 2
+updates:
+  - package-ecosystem: github-actions
+    directory: /
+    schedule:
+      interval: weekly
+  - package-ecosystem: uv
+    directory: /
+    schedule:
+      interval: weekly

websearcher-0.7.2/.github/pull_request_template.md

@@ -0,0 +1,5 @@
+## Summary
+<!-- What does this PR do and why? -->
+
+## Test plan
+<!-- How was this tested? -->

websearcher-0.7.2/.github/workflows/publish.yml

@@ -0,0 +1,33 @@
+name: Publish
+
+on:
+  push:
+    tags: ["v*"]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+      - uses: astral-sh/setup-uv@v8.1.0
+      - run: uv build
+      - uses: actions/upload-artifact@v7
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/project/WebSearcher/
+    permissions:
+      id-token: write
+      contents: read
+    steps:
+      - uses: actions/download-artifact@v8
+        with:
+          name: dist
+          path: dist/
+      - uses: pypa/gh-action-pypi-publish@release/v1

websearcher-0.7.2/.github/workflows/test.yml

@@ -0,0 +1,26 @@
+name: Tests
+
+on:
+  push:
+    branches: [dev, master]
+  pull_request:
+    branches: [dev, master]
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.12", "3.13", "3.14"]
+    steps:
+      - uses: actions/checkout@v6
+      - uses: astral-sh/setup-uv@v8.1.0
+      - run: uv python install ${{ matrix.python-version }}
+      - run: uv sync --all-groups --python ${{ matrix.python-version }}
+      - run: uv run ruff check .
+      - run: uv run ruff format --check .
+      - run: uv run pyrefly check
+      - run: uv run pytest --cov --cov-report=term-missing

websearcher-0.7.2/.gitignore

@@ -0,0 +1,22 @@
+.venv/
+.archive/
+.claude
+TODO.md
+
+build/
+data/
+notebooks/
+scripts/ads-no-subtype/
+
+*.egg-info
+*__pycache__
+
+# Caches
+.pytest_cache/
+.ruff_cache/
+.coverage
+
+# Environment files
+.env
+.env.*
+!.env.example

websearcher-0.7.2/.pre-commit-config.yaml

@@ -0,0 +1,16 @@
+repos:
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.15.5
+    hooks:
+      - id: ruff-format
+      - id: ruff
+        args: [--fix]
+  - repo: local
+    hooks:
+      - id: pyrefly-check
+        name: pyrefly check
+        entry: uv run pyrefly check
+        language: system
+        types_or: [python, pyi]
+        pass_filenames: false
+        require_serial: true

websearcher-0.7.2/CHANGELOG.md

@@ -0,0 +1,199 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/),
+and this project adheres to [Semantic Versioning](https://semver.org/).
+
+## [Unreleased]
+
+## [0.7.1] - 2026-05-03
+
+- Added component type registry: consolidates header-text mappings, parser dispatch, and labels (supersedes `cmpt_mappings.py`)
+- Added `find_by_selectors` utility; applied to classifiers
+- Added `Selector` NamedTuple in `utils` for tag/attrs selector lists
+- Added pyrefly type checking; fixed all type errors across the codebase
+- Added structural tests for component type registry; resolved "Ver más" collision
+- Refactored ad parser and classifier; tidied carousel scope; edited local parser/classifier selectors
+- Restored `knowledge_panel` cmpt-attr check on `jscontroller qTdDb`
+- Moved `ClassifyHeaderText` into `main.py` as `ClassifyMainHeader`; removed dead `ClassifyHeaderComponent`
+- Updated CI: ruff lint + format checks, pyrefly check, coverage; matrix narrowed to 3.12-3.14
+- Switched publish workflow to tag-based with split build + publish jobs
+- Added Dependabot config, PR template, pre-commit ruff + pyrefly hooks
+- Bumped support floor: `requires-python = ">=3.12"`, ruff/pyrefly `target-version = py312`
+- Bumped security floors: `requests>=2.33.0`, `lxml>=6.1.0`, `pytest>=9.0.3`; pulled Dependabot patches for `pygments`
+- Fixed `ResponseOutput` not subscriptable for dict-style access
+- Added `CHANGELOG.md` and changelog guide (migrated from README)
+- Added `docs/plans/` development history with landing page
+
+## [0.7.0] - 2026-03-15
+
+- **Breaking:** `details` field is now always `dict | None` with a self-describing `type` key (e.g. `{"type": "menu", "items": [...]}`)
+- **Breaking:** `parse_serp()` now always returns a dict with `results` and `features` keys; the `extract_features` parameter has been removed
+- Standardized all models on Pydantic BaseModel (removed dataclasses)
+- Added `ResponseOutput` and `ParsedSERP` typed models
+- Removed `DetailsItem`, `DetailsList` classes
+- Normalized `local_results` sub_type for location-specific headers
+- Replaced `os` with `pathlib.Path` throughout
+- Consolidated `webutils.py` into `utils.py`
+- Added ruff formatting, linting, and pre-commit hooks
+- Added test coverage reporting (69%)
+- Added unit tests for utils, locations, models, and feature extractor
+- Replaced pandas with polars in demo scripts
+
+## [0.6.9] - 2026-02-22
+
+- Fixed bugs in component parsers (class comparison, assignment operator, set literal)
+- Fixed `return` in `finally` block in requests searcher
+- Added captcha detection to feature extractor
+- Added captcha handling and jittered delay to demo searches
+- Dropped pandas from core dependencies
+- Cleaned up legacy typing imports
+- Removed poetry.toml
+
+## [0.6.8] - 2026-02-20
+
+- Migrated from Poetry to uv for dependency management
+- Added Python 3.12-3.14 test matrix in GitHub Actions
+- Added `flights` classifier and `standard-4` layout
+- Added local service ad parser
+- Extracted bottom ads before main column
+- Fixed `return` in `finally` block warning in selenium searcher
+
+## [0.6.7] - 2026-02-06
+
+- Added `get_text_by_selectors()` to `webutils` -- centralizes multi-selector fallback pattern across 7 component parsers
+- Added `perspectives`, `recent_posts`, and `latest_from` component classifiers
+- Added `sub_type` to perspectives parser from header text
+- Added CI test workflow on push to dev branch
+- Added compressed test fixtures with `condense_fixtures.py` script
+- Updated dependency lower bounds for security patches (protobuf, orjson)
+- Updated GitHub Actions to checkout v6 and setup-python v6
+
+## [0.6.6] - 2025-12-05
+
+- Update packages with dependabot alerts (brotli, urllib3)
+
+## [0.6.5] - 2025-12-05
+
+- Add GitHub Actions section to README
+
+## [0.6.0] - 2025-03-28
+
+- Method for collecting data with selenium; requests no longer works without a redirect
+- Pull request [#72](https://github.com/gitronald/WebSearcher/pull/72)
+
+## [0.5.2] - 2025-03-09
+
+- Added support for Spanish component headers by text
+- Pull request [#74](https://github.com/gitronald/WebSearcher/pull/74)
+
+## [0.5.1] - 2025-03-07
+
+- Fixed canonical name -> UULE converter using `protobuf`, see [this gist](https://gist.github.com/gitronald/66cac42194ea2d489ff3a1e32651e736) for details
+- Added lang arg to specify language in se.search, uses hl URL param and does not change Accept-Language request header (which defaults to en-US), but works in tests.
+- Fixed null location/language arg input handling (again)
+- Pull Request [#76](https://github.com/gitronald/WebSearcher/pull/76)
+
+## [0.5.0] - 2025-02-03
+
+- configuration now using poetry v2
+
+## [0.4.9] - 2025-02-03
+
+- last version with poetry v1, future versions (`>=0.5.0`) will use [poetry v2](https://python-poetry.org/blog/announcing-poetry-2.0.1/) configs.
+
+## [0.4.2] - [0.4.8] - 2024-11-11 to 2025-02-03
+
+- varied parser updates, testing with py3.12.
+
+## [0.4.1] - 2024-08-26
+
+- Added notices component types, including query edits, suggestions, language tips, and location tips.
+
+## [0.4.0] - 2024-05-27
+
+- Restructured parser for component classes, split classifier into submodules for header, main, footer, etc., and rewrote extractors to work with component classes. Various bug fixes.
+
+## [0.3.13]
+
+- New footer parser, broader extraction coverage, various bug and deprecation fixes.
+
+## [0.3.12] - 2024-05-09
+
+- Added num_results to search args, added handling for local results text and labels (made by the SE), ignore hidden_survey type at extraction.
+
+## [0.3.11] - 2024-05-08
+
+- Added extraction of labels for ads (made by the SE), use model validation, cleanup and various bug fixes.
+
+## [0.3.10] - 2024-05-06
+
+- Updated component classifier for images, added exportable header text mappings, added gist on localized searches.
+
+## [0.3.9] - 2024-02-25
+
+- Small fixes for video url parsing
+
+## [0.3.8] - 2024-02-13
+
+- Using SERP pydantic model, added github pip publishing workflow
+
+## [0.3.7] - 2024-02-09
+
+- Fixed localization, parser and classifier updates and fixes, image subtypes, changed rhs component handling.
+
+## [0.3.0] - [0.3.6] - 2023-10-16 to 2023-12-08
+
+- Parser updates for SERPs from 2022 and 2023, standalone extractors file, added pydantic, reduced redundancies in outputs.
+
+## [2020.0.0], [2022.12.18], [2023.01.04] - 2022-12-19, 2022-12-21, and 2023-01-04
+
+- Various updates, attempt at date versioning that seemed like a good idea at the time ¯\\\_(ツ)\_/¯
+
+<!-- refs/tags/v2022.12.18 -->
+<!-- refs/tags/v2023.01.04 -->
+
+## [0.2.15]
+
+- Fix people-also-ask and hotel false positives, add flag for left-hand side bar
+
+## [0.2.14]
+
+- Add shopping ads carousel and three knowledge subtypes (flights, hotels, events)
+
+## [0.2.13]
+
+- Small fixes for knowledge subtypes, general subtypes, and ads
+
+## [0.2.12] - 2021-12-17
+
+- Try to brotli decompress by default
+
+## [0.2.11] - 2021-11-15
+
+- Fixed local result parser and no return in general extra details
+
+## [0.2.10]
+
+- a) Add right-hand-side knowledge panel and top image carousel, b) Add knowledge and general component subtypes, c) Updates to component classifier, footer, ad, and people_also_ask components
+
+## [0.2.9] - 2021-05-08
+
+- Various fixes for SERPs with a left-hand side bar, which are becoming more common and change other parts of the SERP layout.
+
+## [0.2.8] - 2021-03-08
+
+- Small fixes due to HTML changes, such as missing titles and URLs in general components
+
+## [0.2.7] - 2020-11-30
+
+- Added fix for parsing twitter cards, removed pandas dependencies and several unused functions, moving towards greater package simplicity.
+
+## [0.2.6] - 2020-11-14
+
+- Updated ad parser for latest format, still handles older ad format.
+
+## [0.2.5] - 2020-07-24
+
+-  Google Search, like most online platforms, undergoes changes over time. These changes often affect not just their outward appearance, but the underlying code that parsers depend on. This makes parsing a goal with a moving target. Sometime around February 2020, Google changed a few elements of their HTML structure which broke this parser. I created this patch for these changes, but have not tested its backwards compatibility (e.g. on SERPs collected prior to 2/2020). More generally, there's no guarantee on future compatibility. In fact, there is almost certainly the opposite: more changes will inevitably occur. If you have older data that you need to parse and the current parser doesn't work, you can try using `0.2.1`, or send a pull request if you find a way to make both work!

websearcher-0.7.2/CLAUDE.md

@@ -0,0 +1,73 @@
+# WebSearcher
+
+Parser library for Google search result pages. Each SERP is decomposed into a typed list of `results` (e.g. `general`, `top_stories`, `perspectives`, `local_results`, `available_on`, `videos`, `searches_related`, ...) by an `Extractor` + `ClassifyMain`/`ClassifyFooter` pipeline, then per-component parsers populate `title`, `url`, `text`, `cite`, `details`, etc.
+
+## Running code
+
+All Python execution goes through `uv` (the project's `.python-version` is 3.14):
+
+```bash
+uv run python scripts/...
+uv run pytest
+```
+
+Do not use `poetry`, `python`, or `python3` directly. Some skill files still reference `poetry` from before the migration — treat those as stale.
+
+## Inspection scripts
+
+Use these when diagnosing parser issues or auditing a demo dataset:
+
+| Script | When |
+|---|---|
+| `scripts/show_parsed.py "{query}" --data-dir {dir}` | See the parsed-results table for a SERP (rank, type, sub_type, title, url, details summary). Re-parses fresh on every call — always reflects current code. |
+| `scripts/show_serp.py "{query}" --data-dir {dir}` | Serve the saved HTML on localhost:8765 with Google's overlays/scroll-locks stripped. `--raw` keeps the original. |
+| `scripts/demo_screenshot.py --data-dir {dir}` | Render a screenshot of the SERP with component-type colored borders. |
+| `scripts/show_serp.py --list --data-dir {dir}` | List the queries in a dataset. |
+
+Default `--data-dir` resolves to `data/demo-ws-v{ws.__version__}/`. Pass an explicit path for older fixtures or `/tmp` extracts.
+
+## Demo data layout
+
+```
+data/demo-ws-v{version}/
+  serps.json     # JSONL: {qry, html, serp_id, ...}
+  parsed.json    # JSONL: {results: [...], features: {...}, ...} — pre-parsed output
+  searches.json  # JSONL: {qry, serp_id, timestamp, ...} — search metadata
+tests/fixtures/
+  serps-v0.6.7.json.bz2   # older bz2-compressed JSONL fixtures
+  serps-v0.6.8.json.bz2   # — useful for regressions; the "northern lights" SERP used in plan 018/019 lives here
+```
+
+## Skills (`.claude/skills/`)
+
+| Skill | Use case |
+|---|---|
+| `/parser-update` | 7-phase parser diagnose-and-fix workflow on demo data. Now references the inspection scripts above. |
+| `/compare-parsed` | Compare parsed output before/after uncommitted changes — regression check. |
+| `/compare-selectors` | Diff selectors between current and committed versions of a parser file. |
+| `/reparse` | Re-parse fixtures with current code and diff against pre-parsed output. |
+
+## Plans and TODO
+
+- `docs/plans/{NNN}-{slug}.md` — design specs with `status: draft|active|done|abandoned` frontmatter
+- `TODO.md` — gitignored, flat list of open/closed items, each pointing to a plan or marked `(no plan)`
+
+Conventions per global rules in `~/.claude/rules/plan-files.md` and `~/.claude/rules/todo-files.md`.
+
+## Schema conventions
+
+A `details` block on a parsed result uses a controlled vocabulary of `type` values. Try to reuse existing labels rather than invent new ones:
+
+- `text` — `{items: [str, ...]}` (people_also_ask, searches_related)
+- `hyperlinks` — `{items: [{url, text, ...}]}` (available_on, top_image_carousel, knowledge_rhs, footer image cards, general submenu)
+- `ratings` — rating data, see general's modern rating widget and shopping_ads
+- `place` — local_results: rating, n_reviews, price, category, address, phone, hours, review_snippet, website, directions
+- `panel`, `video` — used by knowledge / general video subtypes
+
+When a `details` block would have only null fields, set `details=None` instead of emitting a hollow dict.
+
+## Notable parser dependencies
+
+- `top_stories.parse_top_stories` is shared by `top_stories`, `perspectives`, `local_news`, `recent_posts`, and `latest_from`. One selector change ripples to all five. Verify with `/compare-parsed` after edits.
+- `parse_subtype_details` in `general.py` has its own elif chain for layout variants (subresult, submenu, submenu_rating, submenu_mini, submenu_scholarly, submenu_product). Some branches are dormant on modern SERPs; modern rating widget reads from `Y0A0hc`/`z3HNkc` aria-labels.
+- `classifiers/main.py` runs an ordered chain of classifiers; order matters when components match multiple. `available_on` was recently moved ahead of `knowledge_panel` because the streaming-providers widget was being absorbed.

{websearcher-0.7.0 → websearcher-0.7.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: WebSearcher
-Version: 0.7.0
+Version: 0.7.2
 Summary: Tools for conducting, collecting, and parsing web search
 Project-URL: homepage, http://github.com/gitronald/WebSearcher
 Project-URL: repository, http://github.com/gitronald/WebSearcher
@@ -8,14 +8,14 @@ Author-email: "Ronald E. Robertson" <rer@acm.org>
 License-Expression: GPL-3.0
 License-File: LICENSE
 Keywords: parser,search,web
-Requires-Python: >=3.10
+Requires-Python: >=3.12
 Requires-Dist: beautifulsoup4>=4.12.3
 Requires-Dist: brotli>=1.1.0
-Requires-Dist: lxml>=5.3.0
+Requires-Dist: lxml>=6.1.0
 Requires-Dist: orjson<4.0.0,>=3.11.5
 Requires-Dist: protobuf<7.0.0,>=6.33.5
 Requires-Dist: pydantic>=2.9.2
-Requires-Dist: requests>=2.32.4
+Requires-Dist: requests>=2.33.0
 Requires-Dist: selenium>=4.9.0
 Requires-Dist: tldextract>=5.1.2
 Requires-Dist: undetected-chromedriver>=3.5.5
@@ -29,41 +29,40 @@ This package provides tools for conducting algorithm audits of web search and
 includes a scraper built on `selenium` with tools for geolocating, conducting, 
 and saving searches. It also includes a modular parser built on `BeautifulSoup` 
 for decomposing a SERP into list of components with categorical classifications 
-and position-based specifications.
+and position-based specifications. 
+
+## Recent Changes
+
+- `0.7.1`: Added component type registry and pyrefly type checking; refreshed CI/tooling (lint, format, type-check, tag-based publish); bumped Python floor to 3.12
+- `0.7.0`: Breaking changes, standardized data models on Pydantic, typed `details` field, and removed `DetailsItem`/`DetailsList`
+
+See [CHANGELOG.md](CHANGELOG.md) for a longer history of changes by version.
 
 ## Table of Contents
 
 - [WebSearcher](#websearcher)
+  - [Tools for conducting and parsing web searches](#tools-for-conducting-and-parsing-web-searches)
+  - [Recent Changes](#recent-changes)
+  - [Table of Contents](#table-of-contents)
   - [Getting Started](#getting-started)
   - [Usage](#usage)
     - [Example Search Script](#example-search-script)
     - [Step by Step](#step-by-step)
+      - [1. Initialize Collector](#1-initialize-collector)
+      - [2. Conduct a Search](#2-conduct-a-search)
+      - [3. Parse Search Results](#3-parse-search-results)
+      - [4. Save HTML and Metadata](#4-save-html-and-metadata)
+      - [5. Save Parsed Results](#5-save-parsed-results)
   - [Localization](#localization)
   - [Contributing](#contributing)
+    - [Repair or Enhance a Parser](#repair-or-enhance-a-parser)
+    - [Add a Parser](#add-a-parser)
+    - [Testing](#testing)
+    - [Test Fixtures](#test-fixtures)
   - [GitHub Actions](#github-actions)
-  - [Recent Updates](#recent-updates)
-  - [Update Log](#update-log)
   - [Similar Packages](#similar-packages)
   - [License](#license)
 
----
-## Recent Updates
-
-### 0.7.0 (dev)
-
-- **Breaking:** `details` field is now always `dict | None` with a self-describing `type` key (e.g. `{"type": "menu", "items": [...]}`)
-- **Breaking:** `parse_serp()` now always returns a dict with `results` and `features` keys; the `extract_features` parameter has been removed
-- Standardized all models on Pydantic BaseModel (removed dataclasses)
-- Added `ResponseOutput` and `ParsedSERP` typed models
-- Removed `DetailsItem`, `DetailsList` classes
-- Normalized `local_results` sub_type for location-specific headers
-- Replaced `os` with `pathlib.Path` throughout
-- Consolidated `webutils.py` into `utils.py`
-- Added ruff formatting, linting, and pre-commit hooks
-- Added test coverage reporting (69%)
-- Added unit tests for utils, locations, models, and feature extractor
-- Replaced pandas with polars in demo scripts
-
 ---
 ## Getting Started
 
@@ -295,123 +294,6 @@ To release a new version:
 2. Once merged, the package is automatically published to PyPI
 
 ---
-## Update Log
-
-`0.7.0`
-- Standardize data models on Pydantic, typed details field, remove DetailsItem/DetailsList
-
-`0.6.9`
-- Fixed bugs in component parsers (class comparison, assignment operator, set literal)
-- Fixed `return` in `finally` block in requests searcher
-- Added captcha detection to feature extractor
-- Added captcha handling and jittered delay to demo searches
-- Dropped pandas from core dependencies
-- Cleaned up legacy typing imports
-- Removed poetry.toml
-
-`0.6.8`
-- Migrated from Poetry to uv for dependency management
-- Added Python 3.12-3.14 test matrix in GitHub Actions
-- Added `flights` classifier and `standard-4` layout
-- Added local service ad parser
-- Extracted bottom ads before main column
-- Fixed `return` in `finally` block warning in selenium searcher
-
-`0.6.7`
-- Added `get_text_by_selectors()` to `webutils` -- centralizes multi-selector fallback pattern across 7 component parsers
-- Added `perspectives`, `recent_posts`, and `latest_from` component classifiers
-- Added `sub_type` to perspectives parser from header text
-- Added CI test workflow on push to dev branch
-- Added compressed test fixtures with `condense_fixtures.py` script
-- Updated dependency lower bounds for security patches (protobuf, orjson)
-- Updated GitHub Actions to checkout v6 and setup-python v6
-
-`0.6.6`
-- Update packages with dependabot alerts (brotli, urllib3)
-
-`0.6.5`
-- Add GitHub Actions section to README
-
-`0.6.0`
-- Method for collecting data with selenium; requests no longer works without a redirect
-- Pull request [#72](https://github.com/gitronald/WebSearcher/pull/72)
-
-`0.5.2`
-- Added support for Spanish component headers by text
-- Pull request [#74](https://github.com/gitronald/WebSearcher/pull/74)
-
-`0.5.1`
-- Fixed canonical name -> UULE converter using `protobuf`, see [this gist](https://gist.github.com/gitronald/66cac42194ea2d489ff3a1e32651e736) for details
-- Added lang arg to specify language in se.search, uses hl URL param and does not change Accept-Language request header (which defaults to en-US), but works in tests.
-- Fixed null location/language arg input handling (again)
-- Pull Request [#76](https://github.com/gitronald/WebSearcher/pull/76)
-
-`0.5.0`
-- configuration now using poetry v2
-
-`0.4.9` - last version with poetry v1, future versions (`>=0.5.0`) will use [poetry v2](https://python-poetry.org/blog/announcing-poetry-2.0.1/) configs.
-
-`0.4.2` - `0.4.8` - varied parser updates, testing with py3.12.
-
-`0.4.1` - Added notices component types, including query edits, suggestions, language tips, and location tips.
-
-`0.4.0` - Restructured parser for component classes, split classifier into submodules for header, main, footer, etc., and rewrote extractors to work with component classes. Various bug fixes.
-
-`0.3.13` - New footer parser, broader extraction coverage, various bug and deprecation fixes.
-
-`0.3.12` - Added num_results to search args, added handling for local results text and labels (made by the SE), ignore hidden_survey type at extraction.
-
-`0.3.11` - Added extraction of labels for ads (made by the SE), use model validation, cleanup and various bug fixes.
-
-`0.3.10` - Updated component classifier for images, added exportable header text mappings, added gist on localized searches.
-
-`0.3.9` - Small fixes for video url parsing
-
-`0.3.8` - Using SERP pydantic model, added github pip publishing workflow
-
-`0.3.7` - Fixed localization, parser and classifier updates and fixes, image subtypes, changed rhs component handling.
-
-`0.3.0` - `0.3.6` - Parser updates for SERPs from 2022 and 2023, standalone extractors file, added pydantic, reduced redundancies in outputs.
-
-`2020.0.0`, `2022.12.18`, `2023.01.04` - Various updates, attempt at date versioning that seemed like a good idea at the time ¯\\\_(ツ)\_/¯
-
-<!-- refs/tags/v2022.12.18 -->
-<!-- refs/tags/v2023.01.04 -->
-
-`0.2.15` - Fix people-also-ask and hotel false positives, add flag for left-hand side bar
-
-`0.2.14` - Add shopping ads carousel and three knowledge subtypes (flights, hotels, events)
-
-`0.2.13` - Small fixes for knowledge subtypes, general subtypes, and ads
-
-`0.2.12` - Try to brotli decompress by default
-
-`0.2.11` - Fixed local result parser and no return in general extra details
-
-`0.2.10` - a) Add right-hand-side knowledge panel and top image carousel, b) Add knowledge and general component subtypes, c) Updates to component classifier, footer, ad, and people_also_ask components
-
-`0.2.9` - Various fixes for SERPs with a left-hand side bar, which are becoming more common and change other parts of the SERP layout.
-
-`0.2.8` - Small fixes due to HTML changes, such as missing titles and URLs in general components
-
-`0.2.7` - Added fix for parsing twitter cards, removed pandas dependencies and 
-several unused functions, moving towards greater package simplicity.
-
-`0.2.6` - Updated ad parser for latest format, still handles older ad format.
-
-`0.2.5` -  Google Search, like most online platforms, undergoes changes over time. 
-These changes often affect not just their outward appearance, but the underlying 
-code that parsers depend on. This makes parsing a goal with a moving target. 
-Sometime around February 2020, Google changed a few elements of their HTML 
-structure which broke this parser. I created this patch for these changes, 
-but have not tested its backwards compatibility (e.g. on SERPs collected prior to 
-2/2020). More generally, there's no guarantee on future compatibility. In fact, 
-there is almost certainly the opposite: more changes will inevitably occur. 
-If you have older data that you need to parse and the current parser doesn't work, 
-you can try using `0.2.1`, or send a pull request if you find a way to make both work!
-
-
----  
 ## Similar Packages
 
 Many of the packages I've found for collecting web search data via python are no longer maintained, but others are still ongoing and interesting or useful. The primary strength of WebSearcher is its parser, which provides a level of detail that enables examinations of SERP [composition](http://dl.acm.org/citation.cfm?doid=3178876.3186143) by recording the type and position of each result, and its modular design, which has allowed us to (itermittenly) maintain it for so long and to cover such a wide array of component types (currently 25 without considering `sub_types`). Feel free to add to the list of packages or services through a pull request if you are aware of others: