fop-cli 5.4.0 → 6.0.0

package/README.md

@@ -12,7 +12,7 @@ A Rust port of the EasyList FOP tool for sorting and cleaning ad-blocking filter
 - **Git integration**: Commit changes directly to repositories (can be disabled)
 - **easylist_adservers.txt validation**: Ensures rules start with `|` or `/`
 - **:has-text() merging**: Combines rules with same base selector into single regex
-- **Parallel processing**: Processes files in parallel via Rayon, using up to 8 worker threads. Past that, extra workers cost memory without improving throughput; set `RAYON_NUM_THREADS` to override
+- **Parallel processing**: Processes files in parallel via Rayon, using up to 8 worker threads. Past that, extra workers mostly buy memory rather than speed; set `--threads=N` (or `RAYON_NUM_THREADS`) to override
 
 ## Extended Syntax Support
 
@@ -92,7 +92,7 @@ fop -n ~/easylist ~/easyprivacy ~/fanboy-addon
 | `--no-large-warning` | Disable large change warning prompt |
 | `--backup` | Create .backup files before modifying |
 | `--keep-empty-lines` | Keep empty lines in output |
-| `--ignore-dot-domains` | Don't skip rules without dot in domain |
+| `--ignore-dot-domains` | Don't mention rules whose domain has no dot (`\|\|cfd^`, `\|\|countly-`). Such rules are kept either way |
 | `--ignorefiles=` | Additional files to ignore (comma-separated, partial names) |
 | `--ignoredirs=` | Additional directories to ignore (comma-separated, partial names) |
 | `--ignore-all-but=` |  Only process these files, ignore all others (comma-separated) |
@@ -106,14 +106,19 @@ fop -n ~/easylist ~/easyprivacy ~/fanboy-addon
 | `--pr-show-changes` | Include rule changes (combines, merges, typos) in PR body |
 | `--fix-typos` | Fix cosmetic rule typos in all files during sort |
 | `--fix-typos-on-add` | Check cosmetic rule typos in git additions before commit |
+| `--check-rules-on-add` | Check git additions for rules that cannot work (empty selector, unbalanced brackets, malformed domain list, unknown or empty option, bare hostname, missing `||` anchor), and prompt before committing; with `--ci` it audits the committed diff and exits 1 on a defect |
+| `--remove-bad-rules` | Delete defective lines instead of prompting, then commit what is left. Advice (a bare hostname, an unanchored host) is reported but kept, since it is legal syntax (implies `--check-rules-on-add`) |
+| `--ignore-line-minimum` | Keep rules shorter than 3 characters instead of dropping them as malformed |
 | `--auto-fix` | Auto-fix typos without prompting (use with --fix-typos-on-add) |
 | `--only-sort-changed` | Only process files changed according to git |
 | `--check-banned-list=FILE` | Check for banned domains in git additions |
 | `--auto-banned-remove` | Auto-remove banned domains and commit |
 | `--ci` | CI mode - exit with error code on failures (banned domains) |
-| `--rebase-on-fail` | Auto `git pull --rebase --autostash` and retry when a push fails. **On by default** |
-| `--no-rebase-on-fail` | Don't auto-rebase; print the suggested `git pull --rebase` command and stop |
+| `--rebase-on-fail` | Auto `git pull --rebase --autostash` and retry when a push fails, up to 5 times when another push keeps landing first. Stops at a merge conflict with instructions to finish it. A commit that could not be published makes FOP exit 1. **On by default** |
+| `--threads=N` | Worker threads for the parallel pool. Defaults to the core count capped at 8; overrides `RAYON_NUM_THREADS`. |
+| `--no-rebase-on-fail` | Don't auto-rebase; print the suggested `git pull --rebase` command and stop (exit 1) |
 | `--commit-mask=N` | Mask URLs in commit messages (1=`[.]`, 2=`(.)`, 3=space, 4=preserve subdomain dot, 5=Unicode lookalike `․`) |
+| `--no-commit-mask` | Disable URL masking even when `.fopconfig` sets `commit-mask` (`--commit-mask=0` cannot: 0 falls through to level 1) |
 | `--commit-mask-users=u1,u2` | Restrict `--commit-mask` to these `git config user.name` values (lowercased) |
 | `--commit-mask-bare` | Also mask bare hostnames without `http(s)://`. Risks false positives on filenames |
 | `--commit-mask-exempt-hosts=h1,h2` | Additional apex hosts exempt from masking (e.g. self-hosted Gitea/Forgejo/GitLab) |
@@ -132,8 +137,9 @@ fop -n ~/easylist ~/easyprivacy ~/fanboy-addon
 | `--config-file=` | Custom config file path |
 | `--show-config` | Show applied configuration and exit |
 | `--git-binary=<path>` | Path to git binary (default: git in PATH) |
-| `--benchmark` | Benchmark sorting performance (3 iterations, dry-run) |
+| `--benchmark[=N]` | Time the sort: one untimed warm-up, then N runs (default 5), reporting the median, min and max. Files are left untouched: no diff is built, and timestamps and checksums are not written |
 | `--abp-convert` | Convert ABP extended selectors to uBO format |
+| `--adguard-convert` | Promote a `:has-text()` rule's separator to AdGuard's spelling (`##` → `#?#`, `#@#` → `#@?#`) |
 | `--convert-trusted` | Convert trusted scriptlets to non-trusted when value is safe |
 | `-h, --help` | Show help message |
 | `-V, --version` | Show version number |
@@ -173,7 +179,7 @@ backup = false
 # Keep empty lines in output
 keep-empty-lines = false
 
-# Don't skip rules without dot in domain
+# Don't mention rules whose domain has no dot
 ignore-dot-domains = false
 
 # Comment line prefixes
@@ -229,6 +235,7 @@ ci = false
 # reapplies your stashed changes and can leave conflicts to resolve. Set false
 # to be told the command to run instead of having it run for you.
 rebase-on-fail = true
+threads = 8                 # Worker threads; omit to size from the core count
 
 # Mask URLs in commit messages: 1=[.], 2=(.), 3=space, 4=preserve subdomain dot, 5=Unicode lookalike
 # github.com, gitlab.com, and codeberg.org (and subdomains) are always exempt so PR/issue links stay clickable.
@@ -262,6 +269,8 @@ direct-push-users =
 history = A: ,P: ,M: Update,M: Cleanup,M: Sort,M: Adjust
 ```
 
+A `.fopconfig` in the working directory travels with the repository, so it may not be yours: a pull request can add one. From there FOP ignores `git-binary`, ignores a `warning-output` outside the directory, and refuses an `output-diff` outside it. Set those on the command line, in `~/.fopconfig`, or with `--config-file`. Every other option applies as normal.
+
 Command line arguments override config file settings.
 
 ### Per-File Overrides
@@ -284,7 +293,7 @@ add-checksum = true
 add-timestamp = true
 ```
 
-Supported per-file options: `no-sort`, `alt-sort`, `parse-adguard`, `localhost`, `add-checksum`, `add-timestamp`, `no-ubo-convert`, `abp-convert`, `convert-trusted`, `keep-empty-lines`, `ignore-dot-domains`, `fix-typos`.
+Supported per-file options: `no-sort`, `alt-sort`, `parse-adguard`, `localhost`, `add-checksum`, `add-timestamp`, `no-ubo-convert`, `abp-convert`, `adguard-convert`, `convert-trusted`, `keep-empty-lines`, `ignore-dot-domains`, `fix-typos`, `ignore-line-minimum`.
 
 ### Commit Message URL Masking
 
@@ -348,8 +357,8 @@ If `commit-mask` is unset, or every URL was github/gitlab (a no-op mask), the pl
 | Platform | Binary | Optimization | Compatible Devices |
 |----------|--------|--------------|-------------------|
 | **Linux** | | | |
-| x86_64 | `linux-x86_64` | Baseline | All 64-bit Intel/AMD |
-| x86_64 | `linux-x86_64-v3` | AVX2 | Intel Haswell+ / AMD Excavator+ (~2015+) |
+| x86_64 | `linux-x86_64` | Baseline + PGO | All 64-bit Intel/AMD |
+| x86_64 | `linux-x86_64-v3` | AVX2 + PGO | Intel Haswell+ / AMD Excavator+ (~2015+) |
 | x86 | `linux-x86_32` | Baseline | 32-bit systems, older hardware |
 | ARM64 | `linux-arm64` | Baseline | Raspberry Pi 3/4/5, Orange Pi 3/4/5, all ARM64 |
 | ARM64 | `linux-arm64-n1` | Neoverse N1 | Pi 5, Orange Pi 5, AWS Graviton2+, Ampere Altra |
@@ -360,14 +369,32 @@ If `commit-mask` is unset, or every URL was github/gitlab (a no-op mask), the pl
 | **Windows** | | | |
 | x86_64 | `windows-x86_64.exe` | Baseline | All 64-bit Windows |
 | x86_64 | `windows-x86_64-v3.exe` | AVX2 | Intel Haswell+ / AMD Excavator+ (~2015+) |
+| x86_64 | `windows-x86_64-msvc.exe` | MSVC + PGO | All 64-bit Windows. A trial build, see below |
 | x86 | `windows-x86_32.exe` | Baseline | 32-bit Windows |
 | ARM64 | `windows-arm64-v2.exe` | Cortex-A78 | Surface Pro X, Snapdragon laptops |
 
 ### Which binary should I use?
 
-**Linux/Windows x86_64:**
-- Use `-v3` for CPUs from ~2015+ (Haswell, Ryzen) - ~10-20% faster due to AVX2
-- Use baseline if unsure or on older CPUs
+**Linux x86_64:**
+- Use the baseline. It is built with profile-guided optimisation, trained by
+  sorting `test-lists/` so the profile reflects real filter lists, and it runs
+  on every x86_64 CPU: PGO reorders and inlines, it adds no instructions.
+- `-v3` adds AVX2 targeting on top of the same PGO, worth under 1% here and
+  within measurement noise. The hot path is byte scanning, which `memchr`
+  already dispatches to AVX2 at runtime whatever the build targets, so the
+  flag only vectorises colder code. Take it if you want to measure it on your
+  own lists.
+
+**Windows x86_64:**
+- Use the baseline. `-v3` adds AVX2 targeting, for the same negligible gain
+  as on Linux.
+- Neither carries PGO, and not for want of trying: rustup ships no profiler
+  runtime for `x86_64-pc-windows-gnu`, the mingw target these are built with,
+  so it cannot be instrumented at all. Only `x86_64-pc-windows-msvc` can.
+- `-msvc` is that build: MSVC rather than mingw, with PGO, and a static CRT so
+  it still needs no `VCRUNTIME140.dll`. It is published for comparison while
+  the two are measured against each other; `npm install` continues to fetch
+  the mingw build. Try it if you want the speedup on Windows today.
 
 **Linux ARM64 (Raspberry Pi / Orange Pi):**
 - Use `linux-arm64` for Pi 3, Pi 4, Orange Pi 3/4, or if unsure
@@ -379,8 +406,8 @@ If `commit-mask` is unset, or every URL was github/gitlab (a no-op mask), the pl
 
 | Binary | Target | RUSTFLAGS |
 |--------|--------|-----------|
-| `linux-x86_64` | Native | - |
-| `linux-x86_64-v3` | Native | `-C target-cpu=x86-64-v3` |
+| `linux-x86_64` | Native | `-Cprofile-use` (PGO) |
+| `linux-x86_64-v3` | Native | `-C target-cpu=x86-64-v3 -Cprofile-use` (PGO) |
 | `linux-x86_32` | `i686-unknown-linux-gnu` | - |
 | `linux-arm64` | `aarch64-unknown-linux-gnu` | - |
 | `linux-arm64-n1` | `aarch64-unknown-linux-gnu` | `-C target-cpu=neoverse-n1` |
@@ -389,6 +416,7 @@ If `commit-mask` is unset, or every URL was github/gitlab (a no-op mask), the pl
 | `macos-arm64` | `aarch64-apple-darwin` | `-C target-cpu=apple-m1` |
 | `windows-x86_64.exe` | `x86_64-pc-windows-gnu` | - |
 | `windows-x86_64-v3.exe` | `x86_64-pc-windows-gnu` | `-C target-cpu=x86-64-v3` |
+| `windows-x86_64-msvc.exe` | `x86_64-pc-windows-msvc` | `-C target-feature=+crt-static -Cprofile-use` (PGO) |
 | `windows-x86_32.exe` | `i686-pc-windows-gnu` | - |
 | `windows-arm64-v2.exe` | `aarch64-pc-windows-msvc` | `-C target-cpu=cortex-a78` |
 

package/install.js

@@ -7,7 +7,7 @@ const path = require('path');
 const { execSync } = require('child_process');
 
 // Configuration - UPDATE THESE FOR YOUR RELEASE
-const VERSION = '5.4.0';
+const VERSION = '6.0.0';
 const GITHUB_REPO = 'ryanbr/fop-rs'; // Change to your repo
 const BINARY_NAME = 'fop';
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "fop-cli",
-  "version": "5.4.0",
+  "version": "6.0.0",
   "description": "Filter Orderer and Preener - A tool for sorting and cleaning ad-blocking filter lists",
   "keywords": [
     "adblock",