scrape-cli 1.3.5__tar.gz → 1.3.7__tar.gz

{scrape_cli-1.3.5 → scrape_cli-1.3.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scrape_cli
-Version: 1.3.5
+Version: 1.3.7
 Summary: It's a command-line tool to extract HTML elements using an XPath query or CSS3 selector.
 Author-email: Andrea Borruso <aborruso@gmail.com>
 Project-URL: Homepage, https://github.com/aborruso/scrape-cli
@@ -285,6 +285,39 @@ Output:
 
 `-j` automatically wraps the result in `<html>/<body>` before conversion, so you don't need to pass `-b`. For elements, the output is the same you would get from `scrape -be ... | xq .` (the underlying converter is `xmltodict`, the same library used by `xq`). Expressions that return only strings, such as `//a/@href` or `//p/text()`, give the string in `body` when there is one result and an array when there are more.
 
+Use `--json-array` when a script needs one shape for any number of results. The output is a flat list with no `html`/`body` envelope, one item per result, in the order of the expressions. Strings stay strings and elements become objects with the tag as key, so `scrape --json-array -e "//a/@href" -e "b"` gives `["x", "y", {"b": "tre"}]`, and no results give `[]`. It implies `-j` and has the same limits.
+
+With mixed expressions, each result keeps its own form and its place in the order of the expressions. XPath function values are strings, so `count()` gives `"2"`, not `2`:
+
+```bash
+scrape --json-array -e "h1" -e "//a/@href" -e "count(//a)" page.html
+# [{"h1": "News"}, "/one", "/two", "2"]
+```
+
+The list shape makes `jq` pipelines simple, with the same code for zero, one or many results:
+
+```bash
+# Count the results
+scrape --json-array -e "a" page.html | jq 'length'
+
+# Loop over attribute values; with no results, the loop does not run
+scrape --json-array -e "//a/@href" page.html | jq -r '.[]' | while read -r href; do
+  echo "got $href"
+done
+
+# Read fields of each element, as TSV
+scrape --json-array -e "a" page.html | jq -r '.[].a | [.["@href"], .["#text"]] | @tsv'
+
+# Filter elements by attribute
+scrape --json-array -e "a" page.html | jq -c '.[] | select(.a["@class"] == "ext")'
+
+# With mixed expressions, keep only the strings
+scrape --json-array -e "h1" -e "//a/@href" page.html | jq -c 'map(strings)'
+
+# Turn a count() into a number
+scrape --json-array -e "count(//a)" page.html | jq '.[0] | tonumber'
+```
+
 `-j` is mutually exclusive with `-t`, `-x` (`--check-existence`) and `-a` (`--argument`).
 
 `-j` exits with an error when a matched element has an attribute name that is not valid XML, such as `@click` or `:bind`.
@@ -299,6 +332,68 @@ Some notes on the commands:
 - `-t` to extract only text content (useful for LLMs and text processing)
 
 
+### Several fields per element
+
+Separate `-e` expressions lose the link between fields: if an article has no date, every following date ends up on the wrong article. Use `--row` to select the elements, and one `--field NAME=EXPR` for each value, relative to the row:
+
+```bash
+scrape --row "article" --field "title=h2" --field "url=.//a/@href" --field "date=time/@datetime" page.html
+```
+
+```json
+[
+  {"title": "First story", "url": "/one", "date": "2026-01-01"},
+  {"title": "Second story", "url": "/two", "date": null}
+]
+```
+
+- Every object has the same keys, and every value is a string or `null`. `null` means no match, `""` a match with no text.
+- A field gives only its first match. Elements give their cleaned text, and XPath functions a string.
+- Field expressions are XPath or CSS, as with `-e`. Write attributes as `.//a/@href`: `a/@href` looks among direct children only, and `//a` is refused because it would read the whole document. In functions, write `string(.//h2)`.
+- `--absolute-urls` and `--base-url` apply to `href` and `src` fields.
+- `--row` cannot be combined with `-e`, `-t`, `-x`, `-a`, `-b`, `-j` or `--json-array`.
+
+### JSON Lines
+
+Use `--jsonl` to get one JSON value per line: the objects of `--row`, or otherwise the items of `--json-array`. With no results the output is empty. It cannot be combined with `-t`, `-x`, `-a`, `-b` or `-j`.
+
+```bash
+scrape --jsonl --row "article" --field "title=h2" --field "date=time/@datetime" page.html
+# {"title": "First story", "date": "2026-01-01"}
+# {"title": "Second story", "date": null}
+
+# Articles with no date, with jq
+scrape --jsonl --row "article" --field "title=h2" --field "date=time/@datetime" page.html \
+  | jq -r 'select(.date == null) | .title'
+
+# Query with DuckDB
+scrape --jsonl --row "article" --field "title=h2" --field "date=time/@datetime" page.html \
+  | duckdb -c "select title, date from read_json('/dev/stdin', format='newline_delimited')"
+```
+
+The whole page is read and the output built in memory: `--jsonl` changes the format, not the memory use.
+
+### Absolute URLs
+
+Links and images often have relative values. Use `--absolute-urls` to get absolute `href` and `src` values, in every output mode except `-t`:
+
+```bash
+scrape --absolute-urls -e "//a/@href" https://www.iana.org/help/example-domains
+```
+
+With stdin or a local file there is no page URL, so pass it with `--base-url`, which implies `--absolute-urls`:
+
+```bash
+curl -s https://www.iana.org/help/example-domains \
+  | scrape --base-url https://www.iana.org/help/example-domains -e "//a/@href"
+```
+
+- The base is the first `<base href>` of the page, resolved against the page URL, as in browsers. With a URL as input, the page URL is the final one after redirects.
+- With no absolute base, the command exits with code 1.
+- Selection uses the original values, so `a[href^="/"]` still matches.
+- Other attributes, such as `srcset`, and XPath functions, such as `string(//a/@href)`, are not changed. Use `//a/@href`.
+- `--absolute-urls` and `--base-url` cannot be combined with `-r`, and are ignored with `-t` and `-x`.
+
 ## License
 
 [MIT](LICENSE)

{scrape_cli-1.3.5 → scrape_cli-1.3.7}/README.md

@@ -268,6 +268,39 @@ Output:
 
 `-j` automatically wraps the result in `<html>/<body>` before conversion, so you don't need to pass `-b`. For elements, the output is the same you would get from `scrape -be ... | xq .` (the underlying converter is `xmltodict`, the same library used by `xq`). Expressions that return only strings, such as `//a/@href` or `//p/text()`, give the string in `body` when there is one result and an array when there are more.
 
+Use `--json-array` when a script needs one shape for any number of results. The output is a flat list with no `html`/`body` envelope, one item per result, in the order of the expressions. Strings stay strings and elements become objects with the tag as key, so `scrape --json-array -e "//a/@href" -e "b"` gives `["x", "y", {"b": "tre"}]`, and no results give `[]`. It implies `-j` and has the same limits.
+
+With mixed expressions, each result keeps its own form and its place in the order of the expressions. XPath function values are strings, so `count()` gives `"2"`, not `2`:
+
+```bash
+scrape --json-array -e "h1" -e "//a/@href" -e "count(//a)" page.html
+# [{"h1": "News"}, "/one", "/two", "2"]
+```
+
+The list shape makes `jq` pipelines simple, with the same code for zero, one or many results:
+
+```bash
+# Count the results
+scrape --json-array -e "a" page.html | jq 'length'
+
+# Loop over attribute values; with no results, the loop does not run
+scrape --json-array -e "//a/@href" page.html | jq -r '.[]' | while read -r href; do
+  echo "got $href"
+done
+
+# Read fields of each element, as TSV
+scrape --json-array -e "a" page.html | jq -r '.[].a | [.["@href"], .["#text"]] | @tsv'
+
+# Filter elements by attribute
+scrape --json-array -e "a" page.html | jq -c '.[] | select(.a["@class"] == "ext")'
+
+# With mixed expressions, keep only the strings
+scrape --json-array -e "h1" -e "//a/@href" page.html | jq -c 'map(strings)'
+
+# Turn a count() into a number
+scrape --json-array -e "count(//a)" page.html | jq '.[0] | tonumber'
+```
+
 `-j` is mutually exclusive with `-t`, `-x` (`--check-existence`) and `-a` (`--argument`).
 
 `-j` exits with an error when a matched element has an attribute name that is not valid XML, such as `@click` or `:bind`.
@@ -282,6 +315,68 @@ Some notes on the commands:
 - `-t` to extract only text content (useful for LLMs and text processing)
 
 
+### Several fields per element
+
+Separate `-e` expressions lose the link between fields: if an article has no date, every following date ends up on the wrong article. Use `--row` to select the elements, and one `--field NAME=EXPR` for each value, relative to the row:
+
+```bash
+scrape --row "article" --field "title=h2" --field "url=.//a/@href" --field "date=time/@datetime" page.html
+```
+
+```json
+[
+  {"title": "First story", "url": "/one", "date": "2026-01-01"},
+  {"title": "Second story", "url": "/two", "date": null}
+]
+```
+
+- Every object has the same keys, and every value is a string or `null`. `null` means no match, `""` a match with no text.
+- A field gives only its first match. Elements give their cleaned text, and XPath functions a string.
+- Field expressions are XPath or CSS, as with `-e`. Write attributes as `.//a/@href`: `a/@href` looks among direct children only, and `//a` is refused because it would read the whole document. In functions, write `string(.//h2)`.
+- `--absolute-urls` and `--base-url` apply to `href` and `src` fields.
+- `--row` cannot be combined with `-e`, `-t`, `-x`, `-a`, `-b`, `-j` or `--json-array`.
+
+### JSON Lines
+
+Use `--jsonl` to get one JSON value per line: the objects of `--row`, or otherwise the items of `--json-array`. With no results the output is empty. It cannot be combined with `-t`, `-x`, `-a`, `-b` or `-j`.
+
+```bash
+scrape --jsonl --row "article" --field "title=h2" --field "date=time/@datetime" page.html
+# {"title": "First story", "date": "2026-01-01"}
+# {"title": "Second story", "date": null}
+
+# Articles with no date, with jq
+scrape --jsonl --row "article" --field "title=h2" --field "date=time/@datetime" page.html \
+  | jq -r 'select(.date == null) | .title'
+
+# Query with DuckDB
+scrape --jsonl --row "article" --field "title=h2" --field "date=time/@datetime" page.html \
+  | duckdb -c "select title, date from read_json('/dev/stdin', format='newline_delimited')"
+```
+
+The whole page is read and the output built in memory: `--jsonl` changes the format, not the memory use.
+
+### Absolute URLs
+
+Links and images often have relative values. Use `--absolute-urls` to get absolute `href` and `src` values, in every output mode except `-t`:
+
+```bash
+scrape --absolute-urls -e "//a/@href" https://www.iana.org/help/example-domains
+```
+
+With stdin or a local file there is no page URL, so pass it with `--base-url`, which implies `--absolute-urls`:
+
+```bash
+curl -s https://www.iana.org/help/example-domains \
+  | scrape --base-url https://www.iana.org/help/example-domains -e "//a/@href"
+```
+
+- The base is the first `<base href>` of the page, resolved against the page URL, as in browsers. With a URL as input, the page URL is the final one after redirects.
+- With no absolute base, the command exits with code 1.
+- Selection uses the original values, so `a[href^="/"]` still matches.
+- Other attributes, such as `srcset`, and XPath functions, such as `string(//a/@href)`, are not changed. Use `//a/@href`.
+- `--absolute-urls` and `--base-url` cannot be combined with `-r`, and are ignored with `-t` and `-x`.
+
 ## License
 
 [MIT](LICENSE)

{scrape_cli-1.3.5 → scrape_cli-1.3.7}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "scrape_cli"
-version = "1.3.5"
+version = "1.3.7"
 description = "It's a command-line tool to extract HTML elements using an XPath query or CSS3 selector."
 readme = "README.md"
 authors = [

{scrape_cli-1.3.5 → scrape_cli-1.3.7}/scrape_cli/__init__.py

@@ -4,7 +4,7 @@ scrape-cli - A command-line tool to extract HTML elements using XPath or CSS3 se
 
 from scrape_cli.scrape import main
 
-__version__ = "1.3.5"
+__version__ = "1.3.7"
 __author__ = "Andrea Borruso"
 __author_email__ = "aborruso@gmail.com"
 

{scrape_cli-1.3.5 → scrape_cli-1.3.7}/scrape_cli/scrape.py

@@ -15,10 +15,12 @@ import sys
 import re
 import json
 import argparse
+import copy
 import requests
 import xmltodict
 from lxml import etree
 from cssselect import GenericTranslator
+from urllib.parse import urljoin, urlsplit
 from xml.parsers.expat import ExpatError
 from xml.sax.saxutils import escape
 
@@ -67,6 +69,92 @@ def xpath_result_items(result):
     return [result] if result else []
 
 
+def is_absolute_url(url):
+    try:
+        parts = urlsplit(url)
+    except ValueError:
+        return False
+    return bool(parts.scheme and parts.netloc)
+
+
+def resolve_url(base, value):
+    """Return value as an absolute URL, or unchanged if it cannot be parsed."""
+    try:
+        return urljoin(base, value.strip())
+    except ValueError:
+        return value
+
+
+def document_base(document, page_url):
+    """Base as in the HTML standard: the first <base href>, against the page URL."""
+    hrefs = document.xpath('(//base[@href])[1]/@href')
+    if hrefs:
+        base = resolve_url(page_url or '', hrefs[0])
+        if is_absolute_url(base):
+            return base
+    return page_url
+
+
+def with_absolute_urls(el, base):
+    """Copy the element, so that later expressions still see the original values."""
+    el = copy.deepcopy(el)
+    for node in el.xpath('descendant-or-self::*[@href or @src]'):
+        for name in ('href', 'src'):
+            if name in node.attrib:
+                node.set(name, resolve_url(base, node.get(name)))
+    return el
+
+
+def evaluate(context, expression):
+    try:
+        return context.xpath(expression)
+    except etree.XPathError as err:
+        print(f"Error: invalid XPath expression '{expression}': {err}", file=sys.stderr)
+        sys.exit(1)
+
+
+def parse_fields(fields):
+    """Turn NAME=EXPR strings into {name: XPath}, exiting on invalid ones."""
+    parsed = {}
+    for field in fields:
+        name, _, expression = (part.strip() for part in field.partition('='))
+        if not name or not expression:
+            print(f"Error: --field must be NAME=EXPR, got '{field}'.", file=sys.stderr)
+            sys.exit(1)
+        if name in parsed:
+            print(f"Error: repeated --field name '{name}'.", file=sys.stderr)
+            sys.exit(1)
+        if expression.startswith(('/', '(/')):
+            print(f"Error: --field '{field}' reads the whole document on each row: "
+                  "start the expression with .// instead.", file=sys.stderr)
+            sys.exit(1)
+        parsed[name] = expression if is_xpath(expression) else convert_css_to_xpath(expression)
+    return parsed
+
+
+def field_value(row, expression, base):
+    """String form of the first match in the row, or None if there is none."""
+    for item in xpath_result_items(evaluate(row, expression)):
+        if isinstance(item, str):
+            is_attribute = getattr(item, 'is_attribute', False)
+            if base and is_attribute and item.attrname in ('href', 'src'):
+                return resolve_url(base, item)
+            if is_attribute or item.strip():
+                return item.strip()
+        elif isinstance(item.tag, str):  # not a comment or processing instruction
+            return clean_text(''.join(item.itertext()))
+    return None
+
+
+def write_json(value, lines):
+    """Print a JSON value, or with lines one item of the list per line."""
+    if lines:
+        for item in value:
+            sys.stdout.write(json.dumps(item, ensure_ascii=False) + "\n")
+    else:
+        sys.stdout.write(json.dumps(value, indent=2, ensure_ascii=False) + "\n")
+
+
 def wrap_in_body(results):
     return "<!DOCTYPE html>\n<html>\n<body>\n" + "".join(r + "\n" for r in results) + "</body>\n</html>\n"
 
@@ -84,7 +172,7 @@ def is_xpath(expression):
     expr = expression.strip()
 
     # Direct XPath patterns
-    if expr.startswith('/') or expr.startswith('//'):
+    if expr.startswith('/') or expr.startswith('//') or expr.startswith('./'):
         return True
     if '::' in expr:
         return True
@@ -133,6 +221,15 @@ examples:
   scrape -t file.html                   all body text, no script or style
   scrape -te "//article" file.html      text of the matching elements
   scrape -je "a" file.html              JSON
+  scrape --json-array -e "a" file.html  JSON, always a flat list
+  scrape --absolute-urls -e "//a/@href" https://example.org/
+                                        absolute href and src values
+  curl -s https://example.org/ | scrape --base-url https://example.org/ -e "a"
+                                        same, for stdin or a local file
+  scrape --row "article" --field "title=h2" --field "url=.//a/@href" file.html
+                                        one JSON object per article
+  scrape --jsonl -e "//a/@href" file.html
+                                        one JSON value per line
   scrape -be "//article" file.html      wrap output in <html><body>
   scrape -x -e "//h1" file.html         test only: exit code, no output
   scrape -e "//h1" https://example.com  fetch from URL
@@ -147,6 +244,15 @@ output:
                        is an object, several are a list
              strings   one is a string, several are a list
              nothing   null
+  --json-array  always a flat list, one item per result, in expression order:
+             [] for nothing, strings as strings, elements as {"tag": ...},
+             function values as strings: count() gives "2"
+  --row    a list with one object per row, one key per --field in the order
+           given, each value a string or null (no match); "" is a match with
+           no text. Elements give their cleaned text. Rows with no match in
+           any field stay in the list.
+  --jsonl  one JSON value per line: the objects of --row, otherwise the items
+           of --json-array. Nothing at all when there are no results.
 
 exit codes:
   0  success, also when nothing matches: empty output, or null body with -j
@@ -155,13 +261,24 @@ exit codes:
 
 notes:
   - No match is not an error: use -x to test for it.
+  - HTML with no elements, such as only spaces or comments, exits 1.
   - With short flags together, -e goes last: -be, -je, -te. As an argument on
     its own, -eb or -ej is rejected, because -e would take "b" or "j" as value.
   - -x evaluates only the first -e.
   - -t with -e includes the text of any script or style inside the matches.
   - -j fails with exit 1 on attribute names that are not valid XML, such as
     @click or :bind. Use -e without -j on those pages.
+  - --field is relative to the row and gives only the first match. Write
+    .//a/@href: a/@href looks among direct children only, and //a is refused
+    because it would read the whole document. In functions, write
+    string(.//h2): string(//h2) is not refused and reads the whole document.
+  - --row is not allowed with -e, -t, -x, -a, -b, -j, --json-array.
+    --jsonl is not allowed with -t, -x, -a, -b, -j.
   - URLs: 30 s timeout, HTTP errors (4xx, 5xx) exit 1.
+  - --absolute-urls changes href and src in the output, not in selection:
+    a[href^="/"] still matches. Base: the first <base href>, against the page
+    URL or --base-url. No absolute base: exit 1. XPath functions such as
+    string(//a/@href) are not changed. Not with -r, ignored with -t and -x.
   - Charset: from the meta tag, otherwise UTF-8. If the bytes are not valid
     UTF-8: Windows-1252, then ISO-8859-1. Other encodings need the meta tag.
 '''
@@ -192,6 +309,18 @@ notes:
     # Option to output structured JSON (built-in, no external xq needed)
     parser.add_argument('-j', '--json', action='store_true', default=False,
                         help="Output structured JSON, see below. Not with -t, -x, -a")
+    parser.add_argument('--json-array', action='store_true', default=False,
+                        help="Output JSON as a flat list, one item per result, see below. Same limits as -j")
+    parser.add_argument('--absolute-urls', action='store_true', default=False,
+                        help="Output href and src values as absolute URLs, see notes")
+    parser.add_argument('--base-url', default=None, metavar="URL",
+                        help="Page URL for stdin and local files. Implies --absolute-urls")
+    parser.add_argument('--row', default=None, metavar="EXPR",
+                        help="Select the elements that give one JSON object each. Needs --field")
+    parser.add_argument('--field', default=[], action='append', metavar="NAME=EXPR",
+                        help="Value of a key in each --row object, relative to the row. Repeatable")
+    parser.add_argument('--jsonl', action='store_true', default=False,
+                        help="Output one JSON value per line, see below. Implies --json-array without --row")
     # Allows to specify one or more XPath or CSS3 selector expressions
     parser.add_argument('-e', '--expression', default=[], action='append',
                         help="XPath query or CSS3 selector, detected automatically. Repeatable")
@@ -206,19 +335,46 @@ notes:
                         help="User-Agent for URL requests")
     args = parser.parse_args()
 
+    if args.jsonl and (args.text or args.check_existence or args.argument or args.body or args.json):
+        print("Error: --jsonl cannot be combined with -t, -x, -a, -b or -j.", file=sys.stderr)
+        sys.exit(1)
+    if bool(args.row) != bool(args.field):
+        print("Error: --row and --field must be used together.", file=sys.stderr)
+        sys.exit(1)
+    fields = {}
+    if args.row:
+        if (args.expression or args.text or args.check_existence or args.argument
+                or args.body or args.json or args.json_array):
+            print("Error: --row cannot be combined with -e, -t, -x, -a, -b, -j or --json-array.", file=sys.stderr)
+            sys.exit(1)
+        fields = parse_fields(args.field)
+    elif args.jsonl:
+        args.json_array = True
+
     # JSON flag is mutually exclusive with -t, -x, -a
+    json_flag = '--json-array' if args.json_array else '--json'
+    if args.json_array:
+        args.json = True
     if args.json and args.text:
-        print("Error: --json and --text are mutually exclusive.", file=sys.stderr)
+        print(f"Error: {json_flag} and --text are mutually exclusive.", file=sys.stderr)
         sys.exit(1)
     if args.json and args.check_existence:
-        print("Error: --json and --check-existence are mutually exclusive.", file=sys.stderr)
+        print(f"Error: {json_flag} and --check-existence are mutually exclusive.", file=sys.stderr)
         sys.exit(1)
     if args.json and args.argument:
-        print("Error: --json cannot be combined with -a/--argument.", file=sys.stderr)
+        print(f"Error: {json_flag} cannot be combined with -a/--argument.", file=sys.stderr)
+        sys.exit(1)
+
+    if args.base_url and not is_absolute_url(args.base_url):
+        print(f"Error: --base-url must be an absolute URL, got '{args.base_url}'.", file=sys.stderr)
+        sys.exit(1)
+    absolute_urls = args.absolute_urls or bool(args.base_url)
+    if absolute_urls and args.rawinput:
+        print("Error: --absolute-urls and --base-url cannot be combined with -r/--rawinput.", file=sys.stderr)
         sys.exit(1)
 
     # Check that at least one expression is provided by the user (unless using -t option)
-    if not args.expression and not args.text:
+    if not args.expression and not args.text and not args.row:
         print(
             "Error: no expression specified. Use -e \"//selector\" or -t.\n"
             "  scrape -e \"//h1\" file.html\n"
@@ -227,6 +383,7 @@ notes:
         )
         sys.exit(1)
 
+    page_url = args.base_url
     # Determine the source of the input: URL, file, or stdin
     if args.html:
         if args.html.startswith('http://') or args.html.startswith('https://'):
@@ -237,6 +394,7 @@ notes:
                 response = requests.get(args.html, headers=headers, timeout=30)
                 response.raise_for_status()
                 inp = response.content
+                page_url = page_url or response.url
             except requests.RequestException as e:
                 print(f"Error downloading HTML: {e}", file=sys.stderr)
                 sys.exit(1)
@@ -314,19 +472,40 @@ notes:
                     inp = inp.decode('iso-8859-1').encode('utf-8')
             document = etree.fromstring(inp, html_parser)
     except (etree.XMLSyntaxError, UnicodeDecodeError) as e:
-        # Print an error in case of syntax issues in the HTML
-        print(f"Error parsing HTML: {e}", file=sys.stderr)
+        # Print an error in case of syntax issues in the input
+        print(f"Error parsing {'XML' if args.rawinput else 'HTML'}: {e}", file=sys.stderr)
+        sys.exit(1)
+
+    if document is None:
+        print("Error parsing HTML: no elements found in the input", file=sys.stderr)
         sys.exit(1)
 
+    # -t and -x output no attribute values
+    base = None
+    if absolute_urls and not args.text and not args.check_existence:
+        base = document_base(document, page_url)
+        if not base:
+            print("Error: no base for --absolute-urls: use --base-url URL, "
+                  "or a page with an absolute <base href>.", file=sys.stderr)
+            sys.exit(1)
+
+    if args.row:
+        row_expression = args.row if is_xpath(args.row) else convert_css_to_xpath(args.row)
+        rows = xpath_result_items(evaluate(document, row_expression))
+        if any(isinstance(row, str) or not isinstance(row.tag, str) for row in rows):
+            print(f"Error: --row must select elements, not values: '{args.row}'.", file=sys.stderr)
+            sys.exit(1)
+        write_json([{name: field_value(row, expression, base) for name, expression in fields.items()}
+                    for row in rows], args.jsonl)
+        return
+
     results = []
     string_results = []
+    # --json-array items: str for strings, bytes for element markup
+    array_items = []
     # For each expression, perform the search in the parsed HTML
     for e in expression:
-        try:
-            result = document.xpath(e)
-        except etree.XPathError as err:
-            print(f"Error: invalid XPath expression '{e}': {err}", file=sys.stderr)
-            sys.exit(1)
+        result = evaluate(document, e)
 
         # If check-existence is enabled, return 0 or 1 depending on the existence of elements
         if args.check_existence:
@@ -336,6 +515,19 @@ notes:
 
         # Extract the text or content of the found elements
         for el in els:
+            if base:
+                if isinstance(el, str):
+                    if getattr(el, 'is_attribute', False) and el.attrname in ('href', 'src'):
+                        el = resolve_url(base, el)
+                elif isinstance(el.tag, str):
+                    el = with_absolute_urls(el, base)
+            if args.json_array:
+                if isinstance(el, str):
+                    if el.strip():
+                        array_items.append(el.strip())
+                elif isinstance(el.tag, str):  # not a comment or processing instruction
+                    array_items.append(etree.tostring(el, with_tail=False))
+                continue
             if isinstance(el, str):
                 # If the element is a string, use the text directly
                 text = el
@@ -356,7 +548,7 @@ notes:
                 text = clean_text(text)
             elif not args.argument:
                 # If no attribute is specified, return the element as HTML string
-                text = etree.tostring(el, pretty_print=True).decode('utf-8')
+                text = etree.tostring(el, pretty_print=True, with_tail=False).decode('utf-8')
             else:
                 # Otherwise, extract the specified attribute
                 text = el.get(args.argument)
@@ -372,17 +564,19 @@ notes:
     # Output handling
     if args.json:
         values = [s for s in string_results if s]
-        if values and len(string_results) == len(results):
-            # Only strings (@attr, text()): XML conversion would merge them into one
-            parsed = {"html": {"body": values[0] if len(values) == 1 else values}}
-        else:
-            try:
+        try:
+            if args.json_array:
+                parsed = [xmltodict.parse(i) if isinstance(i, bytes) else i for i in array_items]
+            elif values and len(string_results) == len(results):
+                # Only strings (@attr, text()): XML conversion would merge them into one
+                parsed = {"html": {"body": values[0] if len(values) == 1 else values}}
+            else:
                 parsed = xmltodict.parse(wrap_in_body(results))
-            except ExpatError as e:
-                print(f"Error: cannot convert result to JSON, it is not valid XML ({e}). "
-                      "Attribute names like @click or :bind are not supported by -j.", file=sys.stderr)
-                sys.exit(1)
-        sys.stdout.write(json.dumps(parsed, indent=2, ensure_ascii=False) + "\n")
+        except ExpatError as e:
+            print(f"Error: cannot convert result to JSON, it is not valid XML ({e}). "
+                  f"Attribute names like @click or :bind are not supported by {'--json-array' if args.json_array else '-j'}.", file=sys.stderr)
+            sys.exit(1)
+        write_json(parsed, args.jsonl)
     elif args.body and not args.text:
         sys.stdout.write(wrap_in_body(results))
     else: