befave 0.1.0__tar.gz

befave-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 David Jackson
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

befave-0.1.0/PKG-INFO

@@ -0,0 +1,189 @@
+Metadata-Version: 2.4
+Name: befave
+Version: 0.1.0
+Summary: Keep favorites in sync across Chrome, Firefox, Safari and Opera via a deduplicated sqlite3 database
+Author: David Jackson
+Author-email: David Jackson <pyguy411@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+
+# befave
+
+Keep your favorites in sync across browsers. `befave` reads the bookmark export
+from Chrome, Firefox, Safari or Opera, merges everything into one sqlite3
+database with no duplicates, and writes it back out as a file any of those
+browsers can import. Two commands do this: `pull` reads a browser's exported
+file into the database, and `push` writes the database out as a file for a
+browser to import. (Browsers call these steps "export" and "import"; `befave`
+uses its own words to keep them apart.) Pull and push can be repeated in
+cycles, so favorites can round trip between browsers.
+
+## Why
+
+- Frustrated that the browser on your work computer doesn't have the same
+  favorites as your home computer? Export from each browser, pull both files
+  into `befave`, and push the merged set back to either one.
+- Added new favorites in a new browser after you pulled your favorites from
+  another one? Export the new browser's bookmarks, pull them (only the new
+  ones are added), then push the full set to any browser.
+- Switching browsers, or using several at once? Keep one master list and
+  load it into whichever browser you pick up.
+
+## Install
+
+```
+uv tool install befave   # once published
+# or from a checkout
+uv run befave
+```
+
+## Usage
+
+Run `befave` and it asks what it needs to know. Most questions are numbered
+choices: type the number and press Enter. Anything you pass as an argument is
+not asked again.
+
+```
+befave                      # asks: pull or push?
+befave pull                 # asks for the file(s), source and database
+befave pull --db a.sqlite3 --source home-firefox example_formats/firefox_bookmarks.html
+befave push                 # asks for the target browser, output file, source, database
+befave push out.html --browser firefox --source home-firefox --db a.sqlite3
+```
+
+When not run from a terminal, `befave` never prompts. It uses the default
+database and fails with a message if something required is missing.
+
+### Step by step
+
+**Pull**: read a browser's exported bookmarks file into the database.
+
+1. In your browser, export your bookmarks as an HTML file (see below).
+2. Run `befave` and choose `1` (pull), or run `befave pull`.
+3. Enter the path to the exported file (`.html` or `.zip`).
+4. **Which database?** Choose `1` to use the default database (its path is
+   shown), or `2` to type the path to a different one.
+5. **Which source is this?** A source is one browser profile on one computer,
+   such as `work-chrome`. Pick an existing one from the list, or choose "Add a
+   new source" and type a name. Use the same name every time you pull from that
+   browser, because `befave` compares each pull with the previous one to spot
+   favorites you deleted.
+6. If asked to confirm deletions or restores, choose `1` (Yes) or `2` (No); see
+   [Deletions](#deletions).
+
+**Push**: write the database out as a file a browser can import.
+
+1. Run `befave` and choose `2` (push), or run `befave push`.
+2. **Which browser will import this file?** Choose the browser. This only sets
+   the name of the toolbar folder in the file, so your toolbar favorites land
+   on that browser's real toolbar (see [What push does](#what-push-does)). It
+   does not change which favorites are included.
+3. Enter the path for the output `.html` file.
+4. Choose the database, as in pull.
+5. Choose the source the file is for, usually the browser profile you are about
+   to import it into. Pushing records what that source now has, so a favorite
+   you later delete there is recognized as a deletion and not re-added.
+6. In the browser, import the file with its bookmark manager's import option.
+
+### Getting an export file from your browser
+
+Use the browser's bookmark manager export (HTML). Safari's export can be a
+`.zip`, which `befave` opens itself. The browser is detected by reading the
+file, so you never say which one it came from.
+
+### Database
+
+The default is `~/.befave/favorites.db`; use `--db` to choose
+another file. One table, `bookmarks`, holds the URL, title, folder path, add
+and last-modified dates, tags (Firefox) and the icon (base64 data). The other
+tables (`sources`, `snapshots`, `source_items`, `tombstones`) track deletions.
+
+### Using more than one computer
+
+The database is a single file on one machine; `befave` does not sync over the
+network. To share favorites between, say, a work and a home computer, keep the
+database in a folder you already sync (Dropbox, iCloud Drive, a USB stick, etc.)
+and point both computers at it with `--db`:
+
+```
+befave pull --db ~/Dropbox/favorites.db --source work-chrome bookmarks.html
+befave push out.html --browser chrome --source work-chrome --db ~/Dropbox/favorites.db
+```
+
+Or copy `~/.befave/favorites.db` between machines. Use it from
+one computer at a time, and let the sync finish before running `befave` on
+another, so two copies don't diverge.
+
+## What pull does
+
+- **No duplicates.** URLs are compared after lowercasing the scheme and host,
+  dropping the `#fragment`, and removing `utm_*` tracking parameters. The path
+  and the rest of the query keep their case. The first URL processed wins,
+  including across files and runs; the same URL in several folders is saved once.
+- **Web links only.** Only `http`, `https` and `ftp` links are kept. Browser
+  internals such as `chrome://`, `about:`, `javascript:`, `moz-extension://`,
+  `place:` and `file://` are skipped.
+- **Browser defaults are skipped.** Firefox's "Mozilla Firefox" folder (and its
+  stray help/customize links) and the 12 links Safari ships in Favorites.
+  The lists are `BUNDLED_FOLDERS` and `SAFARI_DEFAULTS` in `puller.py`.
+- **Toolbar folders are normalized.** Chrome/Opera "Bookmarks Bar", Firefox
+  "Bookmarks Toolbar" and Safari "Favorites" are all stored as `Bookmarks Bar`.
+
+## Deletions
+
+Every pull and push is tied to a **source**: one browser profile on one
+computer (for example `work-chrome`). `befave` asks you to pick an existing
+source or add a new one, or you can pass `--source NAME`.
+
+Each pull is treated as the full list of that source's favorites. A favorite
+that was in the source's previous pull but is missing now counts as deleted
+and is removed from the database, so it is no longer pushed to your other
+browsers. A deleted favorite is not brought back by a pull of another source
+that still has it.
+
+Two safeguards ask for confirmation (`pull --yes` accepts them):
+
+- **Mass deletion:** if more than 30% of a source's favorites (for sources with
+  more than 10) are missing, it may be a partial export, such as a single folder.
+- **Restoring:** a file contains a favorite that was deleted earlier and that
+  source did not have in its last pull. Declining keeps it deleted.
+
+The first pull of a source never deletes anything.
+
+## What push does
+
+Writes a Netscape-format HTML file with the folder hierarchy rebuilt. The
+toolbar folder is named for the target browser (`--browser`):
+
+| Browser | Toolbar folder |
+|---|---|
+| chrome, opera | Bookmarks Bar |
+| firefox | Bookmarks Toolbar |
+| safari | Favorites |
+
+The file has no wrapper folder. Other root folders sit beside the toolbar.
+
+## Possible future feature: hosted sync
+
+Today you share the database file yourself (see above). A paid hosted service
+could do that for you: sign in, and your favorites sync between all your
+computers and browsers without moving files around. This is only an idea and
+nothing is built yet. If you would be interested, please get in touch at
+pyguy411@gmail.com.
+
+## Development
+
+Example exports live in `example_formats/`, named after their browser. They
+contain real bookmarks and are git-ignored: **never commit sample bookmarks or
+test databases.**
+
+```
+uv run befave -h
+uv build
+```
+
+## License
+
+MIT. See [LICENSE](LICENSE). The software is provided "as is", without warranty of any kind.

befave-0.1.0/README.md

@@ -0,0 +1,178 @@
+# befave
+
+Keep your favorites in sync across browsers. `befave` reads the bookmark export
+from Chrome, Firefox, Safari or Opera, merges everything into one sqlite3
+database with no duplicates, and writes it back out as a file any of those
+browsers can import. Two commands do this: `pull` reads a browser's exported
+file into the database, and `push` writes the database out as a file for a
+browser to import. (Browsers call these steps "export" and "import"; `befave`
+uses its own words to keep them apart.) Pull and push can be repeated in
+cycles, so favorites can round trip between browsers.
+
+## Why
+
+- Frustrated that the browser on your work computer doesn't have the same
+  favorites as your home computer? Export from each browser, pull both files
+  into `befave`, and push the merged set back to either one.
+- Added new favorites in a new browser after you pulled your favorites from
+  another one? Export the new browser's bookmarks, pull them (only the new
+  ones are added), then push the full set to any browser.
+- Switching browsers, or using several at once? Keep one master list and
+  load it into whichever browser you pick up.
+
+## Install
+
+```
+uv tool install befave   # once published
+# or from a checkout
+uv run befave
+```
+
+## Usage
+
+Run `befave` and it asks what it needs to know. Most questions are numbered
+choices: type the number and press Enter. Anything you pass as an argument is
+not asked again.
+
+```
+befave                      # asks: pull or push?
+befave pull                 # asks for the file(s), source and database
+befave pull --db a.sqlite3 --source home-firefox example_formats/firefox_bookmarks.html
+befave push                 # asks for the target browser, output file, source, database
+befave push out.html --browser firefox --source home-firefox --db a.sqlite3
+```
+
+When not run from a terminal, `befave` never prompts. It uses the default
+database and fails with a message if something required is missing.
+
+### Step by step
+
+**Pull**: read a browser's exported bookmarks file into the database.
+
+1. In your browser, export your bookmarks as an HTML file (see below).
+2. Run `befave` and choose `1` (pull), or run `befave pull`.
+3. Enter the path to the exported file (`.html` or `.zip`).
+4. **Which database?** Choose `1` to use the default database (its path is
+   shown), or `2` to type the path to a different one.
+5. **Which source is this?** A source is one browser profile on one computer,
+   such as `work-chrome`. Pick an existing one from the list, or choose "Add a
+   new source" and type a name. Use the same name every time you pull from that
+   browser, because `befave` compares each pull with the previous one to spot
+   favorites you deleted.
+6. If asked to confirm deletions or restores, choose `1` (Yes) or `2` (No); see
+   [Deletions](#deletions).
+
+**Push**: write the database out as a file a browser can import.
+
+1. Run `befave` and choose `2` (push), or run `befave push`.
+2. **Which browser will import this file?** Choose the browser. This only sets
+   the name of the toolbar folder in the file, so your toolbar favorites land
+   on that browser's real toolbar (see [What push does](#what-push-does)). It
+   does not change which favorites are included.
+3. Enter the path for the output `.html` file.
+4. Choose the database, as in pull.
+5. Choose the source the file is for, usually the browser profile you are about
+   to import it into. Pushing records what that source now has, so a favorite
+   you later delete there is recognized as a deletion and not re-added.
+6. In the browser, import the file with its bookmark manager's import option.
+
+### Getting an export file from your browser
+
+Use the browser's bookmark manager export (HTML). Safari's export can be a
+`.zip`, which `befave` opens itself. The browser is detected by reading the
+file, so you never say which one it came from.
+
+### Database
+
+The default is `~/.befave/favorites.db`; use `--db` to choose
+another file. One table, `bookmarks`, holds the URL, title, folder path, add
+and last-modified dates, tags (Firefox) and the icon (base64 data). The other
+tables (`sources`, `snapshots`, `source_items`, `tombstones`) track deletions.
+
+### Using more than one computer
+
+The database is a single file on one machine; `befave` does not sync over the
+network. To share favorites between, say, a work and a home computer, keep the
+database in a folder you already sync (Dropbox, iCloud Drive, a USB stick, etc.)
+and point both computers at it with `--db`:
+
+```
+befave pull --db ~/Dropbox/favorites.db --source work-chrome bookmarks.html
+befave push out.html --browser chrome --source work-chrome --db ~/Dropbox/favorites.db
+```
+
+Or copy `~/.befave/favorites.db` between machines. Use it from
+one computer at a time, and let the sync finish before running `befave` on
+another, so two copies don't diverge.
+
+## What pull does
+
+- **No duplicates.** URLs are compared after lowercasing the scheme and host,
+  dropping the `#fragment`, and removing `utm_*` tracking parameters. The path
+  and the rest of the query keep their case. The first URL processed wins,
+  including across files and runs; the same URL in several folders is saved once.
+- **Web links only.** Only `http`, `https` and `ftp` links are kept. Browser
+  internals such as `chrome://`, `about:`, `javascript:`, `moz-extension://`,
+  `place:` and `file://` are skipped.
+- **Browser defaults are skipped.** Firefox's "Mozilla Firefox" folder (and its
+  stray help/customize links) and the 12 links Safari ships in Favorites.
+  The lists are `BUNDLED_FOLDERS` and `SAFARI_DEFAULTS` in `puller.py`.
+- **Toolbar folders are normalized.** Chrome/Opera "Bookmarks Bar", Firefox
+  "Bookmarks Toolbar" and Safari "Favorites" are all stored as `Bookmarks Bar`.
+
+## Deletions
+
+Every pull and push is tied to a **source**: one browser profile on one
+computer (for example `work-chrome`). `befave` asks you to pick an existing
+source or add a new one, or you can pass `--source NAME`.
+
+Each pull is treated as the full list of that source's favorites. A favorite
+that was in the source's previous pull but is missing now counts as deleted
+and is removed from the database, so it is no longer pushed to your other
+browsers. A deleted favorite is not brought back by a pull of another source
+that still has it.
+
+Two safeguards ask for confirmation (`pull --yes` accepts them):
+
+- **Mass deletion:** if more than 30% of a source's favorites (for sources with
+  more than 10) are missing, it may be a partial export, such as a single folder.
+- **Restoring:** a file contains a favorite that was deleted earlier and that
+  source did not have in its last pull. Declining keeps it deleted.
+
+The first pull of a source never deletes anything.
+
+## What push does
+
+Writes a Netscape-format HTML file with the folder hierarchy rebuilt. The
+toolbar folder is named for the target browser (`--browser`):
+
+| Browser | Toolbar folder |
+|---|---|
+| chrome, opera | Bookmarks Bar |
+| firefox | Bookmarks Toolbar |
+| safari | Favorites |
+
+The file has no wrapper folder. Other root folders sit beside the toolbar.
+
+## Possible future feature: hosted sync
+
+Today you share the database file yourself (see above). A paid hosted service
+could do that for you: sign in, and your favorites sync between all your
+computers and browsers without moving files around. This is only an idea and
+nothing is built yet. If you would be interested, please get in touch at
+pyguy411@gmail.com.
+
+## Development
+
+Example exports live in `example_formats/`, named after their browser. They
+contain real bookmarks and are git-ignored: **never commit sample bookmarks or
+test databases.**
+
+```
+uv run befave -h
+uv build
+```
+
+## License
+
+MIT. See [LICENSE](LICENSE). The software is provided "as is", without warranty of any kind.

befave-0.1.0/pyproject.toml

@@ -0,0 +1,20 @@
+[project]
+name = "befave"
+version = "0.1.0"
+description = "Keep favorites in sync across Chrome, Firefox, Safari and Opera via a deduplicated sqlite3 database"
+readme = "README.md"
+license = "MIT"
+license-files = ["LICENSE"]
+requires-python = ">=3.9"
+dependencies = []
+
+[[project.authors]]
+name = "David Jackson"
+email = "pyguy411@gmail.com"
+
+[project.scripts]
+befave = "befave.cli:main"
+
+[build-system]
+requires = ["uv_build>=0.12.3,<0.13.0"]
+build-backend = "uv_build"

befave-0.1.0/pyproject.toml.orig

@@ -0,0 +1,19 @@
+[project]
+name = "befave"
+version = "0.1.0"
+description = "Keep favorites in sync across Chrome, Firefox, Safari and Opera via a deduplicated sqlite3 database"
+readme = "README.md"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [
+    { name = "David Jackson", email = "pyguy411@gmail.com" }
+]
+requires-python = ">=3.9"
+dependencies = []
+
+[project.scripts]
+befave = "befave.cli:main"
+
+[build-system]
+requires = ["uv_build>=0.12.3,<0.13.0"]
+build-backend = "uv_build"

befave-0.1.0/src/befave/__init__.py

@@ -0,0 +1 @@
+"""Pull favorites from any browser's bookmark export and push them to any browser."""

befave-0.1.0/src/befave/cli.py

@@ -0,0 +1,150 @@
+"""befave: interactive wrapper around the puller and pusher."""
+import argparse
+import os
+import shlex
+import sys
+
+from .pusher import BROWSER_TOOLBAR, run_push
+from .puller import DEFAULT_DB, list_sources, run_pull
+
+INTERACTIVE = sys.stdin.isatty()
+
+
+def ask(question, default=None, choices=None):
+    """Prompt until a non-empty answer (or the default) is given."""
+    hint = f" [{'/'.join(choices)}]" if choices else ""
+    hint += f" (default: {default})" if default else ""
+    while True:
+        try:
+            answer = input(f"{question}{hint}: ").strip()
+        except EOFError:
+            raise SystemExit("\naborted")
+        answer = answer or default
+        if answer and (not choices or answer.lower() in choices):
+            return answer.lower() if choices else answer
+        print("  please enter a valid answer")
+
+
+def choose(question, options):
+    """Show numbered options and return the index of the one picked."""
+    print(question)
+    for i, option in enumerate(options, 1):
+        print(f"  {i}. {option}")
+    numbers = [str(i) for i in range(1, len(options) + 1)]
+    return int(ask("Choose a number", choices=numbers)) - 1
+
+
+def need(value, missing_msg, prompt, **kw):
+    """Return value if given; otherwise prompt, or fail when not interactive."""
+    if value:
+        return value
+    if not INTERACTIVE:
+        raise SystemExit(f"befave: error: {missing_msg}")
+    return ask(prompt, **kw)
+
+
+def pick_db(db):
+    if db:
+        return db
+    if not INTERACTIVE:
+        return DEFAULT_DB
+    if choose("Which database should be used?",
+              [f"Use the default database: {DEFAULT_DB}", "Use a different database"]) == 0:
+        return DEFAULT_DB
+    return ask("Path to the database file")
+
+
+def pick_browser(browser):
+    if browser:
+        return browser
+    if not INTERACTIVE:
+        raise SystemExit("befave: error: --browser is required")
+    names = sorted(BROWSER_TOOLBAR)
+    return names[choose("Which browser will import this file?", names)]
+
+
+def pick_source(db, given=None):
+    """Return the named source, or let the user pick an existing one or add a new one."""
+    if given:
+        return given
+    if not INTERACTIVE:
+        raise SystemExit("befave: error: --source is required")
+    existing = list_sources(db)
+    if existing:
+        i = choose("Which source is this? (one browser profile on one computer)",
+                   existing + ["Add a new source"])
+        if i < len(existing):
+            return existing[i]
+    else:
+        print("No sources yet. A source is one browser profile on one computer.")
+    return ask("Name for the new source (e.g. work-chrome)")
+
+
+def confirm(msg):
+    if not INTERACTIVE:
+        print(f"  {msg} (not confirmed; pass --yes to accept)")
+        return False
+    return choose(msg, ["Yes", "No"]) == 0
+
+
+def do_pull(args):
+    files = args.files
+    if not files:
+        if not INTERACTIVE:
+            raise SystemExit("befave: error: no bookmark files given")
+        files = shlex.split(ask("Browser export file(s) to pull (.html or .zip, space separated)"))
+    db = pick_db(args.db)
+    db = os.path.expanduser(db)
+    return run_pull([os.path.expanduser(f) for f in files], db,
+                    lambda path: pick_source(db, args.source), args.yes, confirm)
+
+
+def do_push(args):
+    browser = pick_browser(args.browser)
+    output = need(args.output, "output file is required", "Output file (.html)")
+    db = pick_db(args.db)
+    db = os.path.expanduser(db)
+    return run_push(os.path.expanduser(output), browser, db, pick_source(db, args.source))
+
+
+def build_parser():
+    ap = argparse.ArgumentParser(
+        prog="befave",
+        description="Pull browser bookmark export files into a database, or push the "
+                    "database out as a file a browser can import. Any missing details "
+                    "are asked interactively.")
+    sub = ap.add_subparsers(dest="action", metavar="{pull,push}")
+
+    pull = sub.add_parser("pull", help="pull browser export file(s) into the database")
+    pull.add_argument("files", nargs="*", help="bookmark export(s): .html or .zip")
+    pull.add_argument("--db", help=f"database file (default: {DEFAULT_DB})")
+    pull.add_argument("--source", help="name of the browser profile the file(s) come from")
+    pull.add_argument("--yes", action="store_true",
+                      help="accept deletion and restore confirmations")
+    pull.set_defaults(func=do_pull)
+
+    push = sub.add_parser("push", help="push the database out as a file for a target browser")
+    push.add_argument("output", nargs="?", help="output .html file")
+    push.add_argument("--browser", choices=sorted(BROWSER_TOOLBAR), help="target browser")
+    push.add_argument("--db", help=f"database file (default: {DEFAULT_DB})")
+    push.add_argument("--source", help="name of the browser profile the file is for")
+    push.set_defaults(func=do_push)
+    return ap
+
+
+def main(argv=None):
+    ap = build_parser()
+    args = ap.parse_args(argv)
+    if not args.action:
+        if not INTERACTIVE:
+            ap.error("an action (pull or push) is required")
+        action = ["pull", "push"][choose("What would you like to do?", [
+            "pull - read a browser's exported bookmarks file into the database",
+            "push - write the database out as a file for a browser to import",
+        ])]
+        args = ap.parse_args([action])
+    return args.func(args)
+
+
+if __name__ == "__main__":
+    sys.exit(main())

befave-0.1.0/src/befave/puller.py

@@ -0,0 +1,302 @@
+"""Pull Netscape-format bookmark exports (.html or .zip) into a sqlite3 db."""
+import os
+import re
+import sqlite3
+import sys
+import time
+import zipfile
+from html.parser import HTMLParser
+from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit
+
+DEFAULT_DB = os.path.join(os.path.expanduser("~"), ".befave", "favorites.db")
+WEB_SCHEMES = ("http", "https", "ftp")
+
+# Folders browsers ship by default; the whole subtree is skipped.
+BUNDLED_FOLDERS = {"mozilla firefox"}
+# Links Safari pre-populates in its top-level Favorites folder.
+SAFARI_DEFAULTS = {
+    "https://www.apple.com/", "https://www.icloud.com/", "https://www.yahoo.com/",
+    "https://www.bing.com/", "https://www.google.com/?client=safari&channel=mac_bm",
+    "https://www.wikipedia.org/", "https://www.facebook.com/", "https://twitter.com/",
+    "https://www.linkedin.com/", "https://www.weather.com/", "https://www.yelp.com/",
+    "https://www.tripadvisor.com/",
+}
+
+BUNDLED_URL = re.compile(r"^https://www\.mozilla\.org/(?:[\w-]+/)?firefox/(?:customize|help)/?$")
+
+# All browsers' toolbar folders are stored under this one name.
+TOOLBAR = "Bookmarks Bar"
+
+SCHEMA = """
+CREATE TABLE IF NOT EXISTS bookmarks (
+    id INTEGER PRIMARY KEY,
+    url_norm TEXT NOT NULL UNIQUE,
+    url TEXT NOT NULL,
+    title TEXT,
+    folder TEXT,
+    add_date INTEGER,
+    last_modified INTEGER,
+    tags TEXT,
+    icon TEXT
+);
+CREATE TABLE IF NOT EXISTS sources (
+    id INTEGER PRIMARY KEY,
+    name TEXT NOT NULL UNIQUE
+);
+CREATE TABLE IF NOT EXISTS snapshots (
+    id INTEGER PRIMARY KEY,
+    source_id INTEGER NOT NULL REFERENCES sources(id),
+    taken_at INTEGER NOT NULL,
+    kind TEXT NOT NULL CHECK (kind IN ('pull', 'push'))
+);
+CREATE TABLE IF NOT EXISTS source_items (
+    snapshot_id INTEGER NOT NULL REFERENCES snapshots(id),
+    url_norm TEXT NOT NULL,
+    PRIMARY KEY (snapshot_id, url_norm)
+);
+CREATE TABLE IF NOT EXISTS tombstones (
+    url_norm TEXT PRIMARY KEY,
+    source_id INTEGER NOT NULL REFERENCES sources(id),
+    deleted_at INTEGER NOT NULL
+);
+"""
+
+# Guard against partial exports being mistaken for mass deletion.
+MASS_DELETE_FRACTION = 0.3
+MASS_DELETE_MIN_ITEMS = 10
+
+
+def normalize_url(url):
+    """Lowercase scheme/host, drop fragment, drop utm_* query parameters."""
+    url = url.strip()
+    try:
+        parts = urlsplit(url)
+    except ValueError:
+        return url
+    if not parts.scheme or not parts.netloc:
+        return urlunsplit((parts.scheme.lower(), parts.netloc, parts.path, parts.query, ""))
+    host = parts.netloc
+    if "@" in host:
+        userinfo, _, hostport = host.rpartition("@")
+        host = userinfo + "@" + hostport.lower()
+    else:
+        host = host.lower()
+    query = parts.query
+    if "utm_" in query.lower():
+        pairs = parse_qsl(query, keep_blank_values=True)
+        query = urlencode([(k, v) for k, v in pairs if not k.lower().startswith("utm_")])
+    return urlunsplit((parts.scheme.lower(), host, parts.path, query, ""))
+
+
+def to_int(value):
+    try:
+        return int(value)
+    except (TypeError, ValueError):
+        return None
+
+
+class BookmarkParser(HTMLParser):
+    """Walks <DL>/<H3>/<A> nesting; yields dicts in file order."""
+
+    def __init__(self):
+        super().__init__(convert_charrefs=True)
+        self.bookmarks = []
+        self.folders = []
+        self.pending_folder = None
+        self.capture = None  # "h3" or "a"
+        self.buf = []
+        self.attrs = {}
+        self.folder_attrs = {}
+
+    def handle_starttag(self, tag, attrs):
+        if tag == "h3":
+            self.capture, self.buf = "h3", []
+            self.folder_attrs = {k.lower(): v for k, v in attrs}
+        elif tag == "a":
+            self.capture, self.buf = "a", []
+            self.attrs = {k.lower(): v for k, v in attrs}
+        elif tag == "dl":
+            self.folders.append(self.pending_folder)
+            self.pending_folder = None
+
+    def handle_endtag(self, tag):
+        if tag == "h3" and self.capture == "h3":
+            name = "".join(self.buf).strip()
+            top_level = not any(self.folders)
+            if top_level and (
+                    self.folder_attrs.get("personal_toolbar_folder", "").lower() == "true"
+                    or name == "Favorites"  # Safari's toolbar folder
+            ):
+                name = TOOLBAR
+            self.pending_folder = name
+            self.capture = None
+        elif tag == "a" and self.capture == "a":
+            self._finish_link()
+        elif tag == "dl" and self.folders:
+            self.folders.pop()
+
+    def handle_data(self, data):
+        if self.capture:
+            self.buf.append(data)
+
+    def _finish_link(self):
+        a = self.attrs
+        self.capture = None
+        url = a.get("href")
+        if not url or not url.strip():
+            return
+        self.bookmarks.append({
+            "url": url.strip(),
+            "title": "".join(self.buf).strip(),
+            "folder": "/".join(f for f in self.folders if f),
+            "add_date": to_int(a.get("add_date")),
+            "last_modified": to_int(a.get("last_modified")),
+            "tags": a.get("tags"),
+            "icon": a.get("icon"),
+        })
+
+
+def is_bundled(b):
+    if any(p.lower() in BUNDLED_FOLDERS for p in b["folder"].split("/")):
+        return True
+    if BUNDLED_URL.match(b["url"]):
+        return True
+    return b["folder"] == TOOLBAR and b["url"] in SAFARI_DEFAULTS
+
+
+def read_export(path):
+    """Return the HTML text of an export file, unzipping if needed."""
+    if zipfile.is_zipfile(path):
+        with zipfile.ZipFile(path) as z:
+            names = [n for n in z.namelist()
+                     if n.lower().endswith((".html", ".htm")) and not n.startswith("__MACOSX")]
+            if not names:
+                raise ValueError(f"no html file found in {path}")
+            return z.read(names[0]).decode("utf-8", errors="replace")
+    with open(path, encoding="utf-8", errors="replace") as f:
+        return f.read()
+
+
+def parse_export(path):
+    text = read_export(path)
+    if "NETSCAPE-BOOKMARK-FILE-1" not in text[:500].upper():
+        raise ValueError(f"{path} is not a Netscape bookmark export")
+    parser = BookmarkParser()
+    parser.feed(text)
+    parser.close()
+    return parser.bookmarks
+
+
+def source_id(conn, name):
+    conn.execute("INSERT OR IGNORE INTO sources (name) VALUES (?)", (name,))
+    return conn.execute("SELECT id FROM sources WHERE name = ?", (name,)).fetchone()[0]
+
+
+def baseline_items(conn, sid):
+    """URLs in the source's latest snapshot (None if it has none)."""
+    row = conn.execute(
+        "SELECT id FROM snapshots WHERE source_id = ? ORDER BY id DESC LIMIT 1", (sid,)
+    ).fetchone()
+    if not row:
+        return None
+    return {r[0] for r in conn.execute(
+        "SELECT url_norm FROM source_items WHERE snapshot_id = ?", (row[0],))}
+
+
+def record_snapshot(conn, sid, kind, items):
+    cur = conn.execute(
+        "INSERT INTO snapshots (source_id, taken_at, kind) VALUES (?,?,?)",
+        (sid, int(time.time()), kind))
+    conn.executemany("INSERT INTO source_items VALUES (?,?)",
+                     [(cur.lastrowid, u) for u in items])
+
+
+def list_sources(db):
+    if not os.path.exists(db):
+        return []
+    conn = sqlite3.connect(db)
+    try:
+        return [r[0] for r in conn.execute("SELECT name FROM sources ORDER BY name")]
+    except sqlite3.OperationalError:
+        return []
+    finally:
+        conn.close()
+
+
+def import_file(conn, path, source, assume_yes=False, confirm=None):
+    """Pull one browser export file as a full snapshot of `source`.
+
+    Returns (found, added, deleted). Raises ValueError if a confirmation is declined.
+    """
+    confirm = confirm or (lambda msg: False)
+    ok = lambda msg: assume_yes or confirm(msg)
+    rows = {}
+    for b in parse_export(path):
+        if is_bundled(b) or urlsplit(b["url"]).scheme.lower() not in WEB_SCHEMES:
+            continue
+        rows.setdefault(normalize_url(b["url"]), b)
+    items = set(rows)
+
+    sid = source_id(conn, source)
+    baseline = baseline_items(conn, sid) or set()
+    vanished = baseline - items
+    if (len(baseline) > MASS_DELETE_MIN_ITEMS
+            and len(vanished) / len(baseline) > MASS_DELETE_FRACTION
+            and not ok(f"{len(vanished)} of {len(baseline)} bookmarks from '{source}' are "
+                       "missing from this file and would be deleted everywhere. "
+                       "Is this a partial export? Delete them anyway?")):
+        raise ValueError(f"{path}: pull cancelled (too many deletions)")
+
+    tombstoned = {r[0] for r in conn.execute("SELECT url_norm FROM tombstones")}
+    # A tombstoned URL the source already had is just a stale copy; one it did not
+    # have before was re-added (or comes from a source never pulled before).
+    revived = (items & tombstoned) - baseline
+    if revived and not ok(f"{len(revived)} bookmark(s) in this file were deleted earlier. "
+                          "Restore them?"):
+        revived = set()
+    blocked = (items & tombstoned) - revived
+
+    now = int(time.time())
+    for url in vanished:
+        conn.execute("INSERT OR REPLACE INTO tombstones VALUES (?,?,?)", (url, sid, now))
+        conn.execute("DELETE FROM bookmarks WHERE url_norm = ?", (url,))
+    for url in revived:
+        conn.execute("DELETE FROM tombstones WHERE url_norm = ?", (url,))
+    added = 0
+    for url in items - blocked:
+        b = rows[url]
+        cur = conn.execute(
+            "INSERT OR IGNORE INTO bookmarks "
+            "(url_norm, url, title, folder, add_date, last_modified, tags, icon) "
+            "VALUES (?,?,?,?,?,?,?,?)",
+            (url, b["url"], b["title"], b["folder"],
+             b["add_date"], b["last_modified"], b["tags"], b["icon"]),
+        )
+        added += cur.rowcount
+    record_snapshot(conn, sid, "pull", items)
+    conn.commit()
+    return len(rows), added, len(vanished)
+
+
+def run_pull(files, db, source=None, assume_yes=False, confirm=None):
+    """Pull each file into db as a snapshot of `source`; returns an exit status.
+
+    `source` may be a name or a callable taking the file path.
+    """
+    os.makedirs(os.path.dirname(os.path.abspath(db)), exist_ok=True)
+    conn = sqlite3.connect(db)
+    conn.executescript(SCHEMA)
+    status = 0
+    for path in files:
+        try:
+            name = source(path) if callable(source) else source
+            found, added, deleted = import_file(conn, path, name, assume_yes, confirm)
+        except (OSError, ValueError, zipfile.BadZipFile) as e:
+            conn.rollback()
+            print(f"error: {e}", file=sys.stderr)
+            status = 1
+            continue
+        print(f"{path}: {found} found, {added} new, {found - added} duplicates skipped, "
+              f"{deleted} deleted")
+    conn.close()
+    return status

befave-0.1.0/src/befave/pusher.py

@@ -0,0 +1,84 @@
+"""Push the favorites db to a Netscape bookmark HTML file that any browser can import."""
+import os
+import sqlite3
+import sys
+from html import escape
+
+from .puller import SCHEMA, TOOLBAR, baseline_items, record_snapshot, source_id
+
+# Name each browser uses for its toolbar folder on import.
+BROWSER_TOOLBAR = {
+    "chrome": "Bookmarks Bar",
+    "opera": "Bookmarks Bar",
+    "firefox": "Bookmarks Toolbar",
+    "safari": "Favorites",
+}
+
+
+def build_tree(rows):
+    root = {"folders": {}, "links": []}
+    for row in rows:
+        node = root
+        for name in filter(None, (row["folder"] or "").split("/")):
+            node = node["folders"].setdefault(name, {"folders": {}, "links": []})
+        node["links"].append(row)
+    return root
+
+
+def attr(name, value):
+    return f' {name}="{escape(str(value), quote=True)}"' if value not in (None, "") else ""
+
+
+def render(node, out, depth, toolbar_name=None):
+    pad = "    " * depth
+    out.append(f"{pad}<DL><p>")
+    for name, child in node["folders"].items():
+        if toolbar_name and name == TOOLBAR:
+            flag = "" if toolbar_name == "Favorites" else ' PERSONAL_TOOLBAR_FOLDER="true"'
+            out.append(f"{pad}    <DT><H3{flag}>{escape(toolbar_name)}</H3>")
+        else:
+            out.append(f"{pad}    <DT><H3>{escape(name)}</H3>")
+        render(child, out, depth + 1)
+    for r in node["links"]:
+        out.append(
+            f"{pad}    <DT><A{attr('HREF', r['url'])}{attr('ADD_DATE', r['add_date'])}"
+            f"{attr('LAST_MODIFIED', r['last_modified'])}{attr('TAGS', r['tags'])}"
+            f"{attr('ICON', r['icon'])}>{escape(r['title'] or r['url'])}</A>"
+        )
+    out.append(f"{pad}</DL><p>")
+
+
+def push(db, path, browser, source):
+    conn = sqlite3.connect(db)
+    conn.executescript(SCHEMA)
+    conn.row_factory = sqlite3.Row
+    rows = conn.execute("SELECT * FROM bookmarks ORDER BY id").fetchall()
+    out = [
+        "<!DOCTYPE NETSCAPE-Bookmark-file-1>",
+        '<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8">',
+        "<TITLE>Bookmarks</TITLE>",
+        "<H1>Bookmarks</H1>",
+    ]
+    root = build_tree(rows)
+    if TOOLBAR in root["folders"]:  # toolbar first
+        root["folders"] = {TOOLBAR: root["folders"].pop(TOOLBAR), **root["folders"]}
+    render(root, out, 0, BROWSER_TOOLBAR[browser])
+    with open(path, "w", encoding="utf-8", newline="\n") as f:
+        f.write("\n".join(out) + "\n")
+    # Browsers add imported bookmarks to what they already have, so the source's
+    # new baseline is its previous one plus everything just pushed.
+    sid = source_id(conn, source)
+    record_snapshot(conn, sid, "push",
+                    (baseline_items(conn, sid) or set()) | {r["url_norm"] for r in rows})
+    conn.commit()
+    conn.close()
+    return len(rows)
+
+
+def run_push(output, browser, db, source):
+    """Push db to output for the target browser; returns a process exit status."""
+    if not os.path.exists(db):
+        print(f"error: database not found: {db}", file=sys.stderr)
+        return 1
+    print(f"{output}: {push(db, output, browser, source)} bookmarks pushed")
+    return 0