wmlinksfromhell 1.0.0__tar.gz

wmlinksfromhell-1.0.0/API_REFERENCE.md

@@ -0,0 +1,898 @@
+# wmlinksfromhell API reference
+
+This is the reference for `wmlinksfromhell 1.0.0`. For a quick start, see [README.md](README.md).
+
+```python
+import wmlinksfromhell
+```
+
+The package-level functions are the easiest place to start. The lower-level classes are useful when you want to keep a resolver or metadata store around.
+
+## Package functions
+
+### `parse(text, source=None, metadata=None) -> Code`
+
+Parse MediaWiki wikitext and return a `Code` object.
+
+`source` can be a dbname, hostname, Wikimedia URL, `Wiki`/`WikiInfo`, Pywikibot `Site`, or Pywikibot `Page`.
+
+```python
+code = wmlinksfromhell.parse(
+    '[[w:en:Apple]] and https://de.wikipedia.org/wiki/Berlin',
+    source='metawiki',
+)
+```
+
+### `parse_page(page, metadata=None) -> Code`
+
+Pywikibot convenience wrapper. It uses `page.site` as the wiki source and keeps the page itself as context.
+
+```python
+code = wmlinksfromhell.parse_page(page)
+```
+
+Keeping the page context matters for anchors and relative links.
+
+### `resolve(value, source=None, metadata=None) -> Result`
+
+Resolve a value without first deciding whether it is a URL, interwiki target, or local link.
+
+```python
+result = wmlinksfromhell.resolve('https://en.wikipedia.org/wiki/Apple')
+```
+
+### `resolve_url(value, source=None, metadata=None) -> Result`
+
+Resolve a URL. Wikimedia article paths, `index.php`, revisions, diffs, actions, fragments, page IDs, and protocol-relative URLs are supported.
+
+### `resolve_interwiki(value, source=None, metadata=None) -> Result`
+
+Resolve an interwiki or local target. A leading `:` is supported.
+
+```python
+result = wmlinksfromhell.resolve_interwiki(
+    ':w:en:Apple',
+    source='metawiki',
+)
+```
+
+### `to_url(destination, metadata=None) -> str`
+
+Render a `Destination` as a URL.
+
+### `to_interwiki(destination, source=None, metadata=None) -> str`
+
+Render a `Destination` as an interwiki target. Exact revision and diff targets use `Special:` forms when possible. If an operation cannot be represented exactly but the page title is known, the normal page interwiki target is used rather than returning an empty value.
+
+### `to_local(destination, source=None, metadata=None) -> str`
+
+Render a destination as a local target on the source wiki. The destination must belong to that wiki.
+
+### `wiki(dbname, metadata=None) -> WikiInfo`
+
+Return a wiki by database name.
+
+```python
+wiki = wmlinksfromhell.wiki('enwiki')
+print(wiki.language)
+print(wiki.hostname)
+```
+
+### `interwiki(dbname, metadata=None, source=None) -> InterwikiInfo`
+
+Return the canonical interwiki identity for a wiki.
+
+```python
+info = wmlinksfromhell.interwiki('dewiktionary')
+print(info.prefix)      # wikt:de
+print(info.project)     # wikt
+print(info.language)    # de
+```
+
+### `url(dbname, title, metadata=None, source=None) -> str`
+
+Build a page URL directly from a dbname and title.
+
+```python
+wmlinksfromhell.url('enwiki', 'Apple')
+```
+
+### `link(dbname, title, label=None, metadata=None, source=None) -> str`
+
+Build an interwiki wikilink directly.
+
+```python
+wmlinksfromhell.link('enwiki', 'Apple')
+wmlinksfromhell.link('enwiki', 'Apple', 'the fruit')
+```
+
+## Code
+
+`Code` is the preferred short name for `WMCode`.
+
+```python
+code = wmlinksfromhell.parse(text)
+```
+
+### `code.links`
+
+The normal link selection, equivalent to `code.filter_links()`.
+
+Links inside HTML comments are not included unless requested.
+
+### `code.filter_links(recursive=True, predicate=None, comment_links=False, **criteria) -> list[Link]`
+
+Select links using semantic criteria.
+
+```python
+code.filter_links()
+code.filter_links(family='wikipedia')
+code.filter_links(project='wikipedia')
+code.filter_links(language='en')
+code.filter_links(dbname='enwiki')
+code.filter_links(namespace='MediaWiki')
+code.filter_links(title='Xtools.js')
+code.filter_links(hostname='en.wikipedia.org')
+```
+
+Boolean filters:
+
+```python
+code.filter_links(wikimedia_only=True)
+code.filter_links(wiki_only=True)
+code.filter_links(external_only=True)
+code.filter_links(interwiki_only=True)
+code.filter_links(local_only=True)
+```
+
+`predicate` receives each `Link` and can be used for custom selection:
+
+```python
+code.filter_links(
+    predicate=lambda link: link.is_wiki and link.namespace == 'MediaWiki'
+)
+```
+
+`comment_links=True` includes links inside HTML comments. `in_comment=True` can then be used to select comments only.
+
+### `code.filter_wikilinks(...) -> list[Link]`
+
+Like `filter_links()`, but limited to wikilink nodes.
+
+### `code.filter_external_links(...) -> list[Link]`
+
+Like `filter_links()`, but limited to external-link nodes.
+
+### `code.convert(to, recursive=True, predicate=None, comment_links=False, **criteria) -> list[Link]`
+
+Convert selected links in place. `to` is one of `url`, `interwiki`, or `local`.
+
+```python
+code.convert('url', family='wikipedia', language='en')
+```
+
+Only selected links are changed. Labels and surrounding wikitext are kept.
+
+### `code.get_sections(*args, **kwargs) -> list[Code]`
+
+Return section wrappers using the same resolver and source context.
+
+### `code.wikicode`
+
+The underlying `mwparserfromhell.wikicode.Wikicode` object. This is mainly useful when you need something that `wmlinksfromhell` does not expose directly.
+
+## Link
+
+`Link` is the preferred short name for `WMLink`.
+
+A `Link` wraps one parsed wikilink or external link.
+
+### Source and syntax
+
+```python
+link.original
+link.raw
+link.label
+link.node
+link.index
+link.in_comment
+link.is_wikilink
+link.is_external_link
+link.syntax_type
+link.input_type
+link.link_type
+```
+
+`syntax_type` values:
+
+```text
+local_wikilink
+interwiki_wikilink
+external_link
+bare_url
+```
+
+`link_type` distinguishes comment links too:
+
+```text
+wikilink
+external_link
+comment_wikilink
+comment_external_link
+```
+
+### Resolution
+
+```python
+link.result
+link.destination
+link.status
+link.reason
+link.confidence
+```
+
+### Prefix information
+
+```python
+link.prefix
+link.canonical_prefix
+link.prefix_kind
+link.is_project_prefix
+link.is_language_prefix
+link.is_language_project
+```
+
+### Destination information
+
+```python
+link.destination_type
+link.is_wikimedia
+link.is_self_link
+link.family
+link.project
+link.language
+link.dbname
+link.wikiid
+link.hostname
+link.title
+link.namespace
+link.fragment
+link.page_id
+link.revision
+link.diff
+link.action
+link.special_page
+link.organization_name
+link.service_name
+link.is_standard_project
+```
+
+### Rendered forms
+
+```python
+link.interwiki
+link.url
+link.canonical_interwiki
+link.canonical_url
+link.canonical_wikilink
+```
+
+### `link.matches(**criteria) -> bool`
+
+Compare a link with semantic fields or another representation.
+
+```python
+link.matches(dbname='enwiki')
+link.matches(family='wikipedia', language='en')
+link.matches(namespace='MediaWiki')
+link.matches(url='https://en.wikipedia.org/wiki/Apple')
+link.matches(interwiki='w:en:Apple')
+```
+
+For a `Destination`:
+
+```python
+link.matches(destination=destination)
+link.matches(same_page_as=destination)
+```
+
+`destination=` compares the complete destination. `same_page_as=` compares page identity without requiring fragments and other non-page state to match.
+
+### `link.set_url() -> None`
+
+Replace the link with its URL form. The link must have a resolved destination.
+
+### `link.set_interwiki(source=None) -> None`
+
+Replace the link with an interwiki form. Raises `ConversionError` when a safe representation is not possible.
+
+### `link.set_local(source=None) -> None`
+
+Replace the link with a local form on the source wiki.
+
+### `link.convert(to, source=None) -> Link`
+
+Convert one link to `url`, `interwiki`, or `local`.
+
+### `link.refresh() -> None`
+
+Clear the cached result for the link so it can be resolved again.
+
+### `link.as_dict() -> dict`
+
+Return a JSON-friendly dictionary.
+
+## Result
+
+`Result` is the preferred short name for `ResolutionResult`.
+
+Useful fields:
+
+```python
+result.raw
+result.syntax_type
+result.input_type
+result.status
+result.reason
+result.confidence
+result.destination
+result.source_wiki
+
+result.prefix
+result.canonical_prefix
+result.prefix_kind
+
+result.family
+result.project
+result.language
+result.dbname
+result.wikiid
+result.hostname
+result.title
+result.namespace
+result.fragment
+result.page_id
+result.revision
+result.diff
+result.action
+result.special_page
+
+result.interwiki
+result.url
+```
+
+There are also convenience flags such as `is_wikimedia`, `is_project_prefix`, `is_language_prefix`, `is_language_project`, `is_revision`, `is_diff`, and `is_action`.
+
+### `result.as_dict() -> dict`
+
+Return a structured dictionary. The resolved `Destination` is included as nested data.
+
+## Destination
+
+`Destination` is the representation used after parsing a link. It does not depend on how the link was written.
+
+```python
+result = wmlinksfromhell.resolve('w:en:Apple')
+destination = result.destination
+```
+
+Fields:
+
+```python
+destination.destination_type
+destination.family
+destination.project
+destination.language
+destination.dbname
+destination.wikiid
+destination.hostname
+destination.title
+destination.namespace
+destination.fragment
+destination.page_id
+destination.revision
+destination.diff
+destination.action
+destination.special_page
+destination.organization_name
+destination.service_name
+destination.query_parameters
+destination.canonical_url
+destination.metadata_source
+destination.is_multilingual
+destination.is_standard_project
+destination.is_wikimedia_project
+```
+
+Methods:
+
+```python
+destination.full_title
+destination.is_revision
+destination.is_diff
+destination.is_action
+destination.page_identity_key()
+destination.identity_key()
+destination.same_page_as(other)
+destination.as_dict()
+destination.as_json()
+```
+
+`page_identity_key()` leaves out fragments and other state that is not part of page identity. `identity_key()` includes the full semantic state.
+
+## Metadata
+
+`Metadata` is the preferred short name for `MetadataStore`.
+
+```python
+metadata = wmlinksfromhell.Metadata()
+```
+
+The store starts with a small bootstrap data set. Network refreshes are explicit.
+
+### Lookups
+
+```python
+metadata.wiki(dbname)
+metadata.wiki_by_dbname(dbname)
+metadata.wiki_by_hostname(hostname)
+metadata.wiki_for_family_language(family, language)
+metadata.known_wiki_hosts()
+metadata.resolve_wikimedia_hostname(hostname)
+metadata.resolve_source(source)
+```
+
+Language helpers:
+
+```python
+metadata.language_name(code)
+metadata.is_known_language(code)
+```
+
+Namespace helpers:
+
+```python
+metadata.namespace_names(dbname)
+metadata.namespace_aliases(dbname)
+metadata.is_namespace(text, source)
+```
+
+Interwiki helpers:
+
+```python
+metadata.interwiki(dbname, source=None)
+metadata.interwiki_map(source=None)
+metadata.global_interwiki_entry(prefix)
+metadata.interwiki_language_for_wiki(wiki)
+metadata.interwiki_prefix_for_destination(destination, source=None)
+```
+
+### Load already-fetched data
+
+```python
+metadata.load_sitematrix(data)
+metadata.load_interwiki_config(text)
+```
+
+These methods do not fetch anything themselves.
+
+### Refresh data
+
+```python
+metadata.update()
+metadata.refresh()
+metadata.update_sitematrix()
+metadata.update_interwiki_config()
+metadata.update_wiki_metadata('enwiki')
+```
+
+`update()` refreshes the metadata normally needed by the resolver and stores it in the local cache. The other methods let you refresh individual pieces.
+
+`update_from_network()` and `refresh_all()` are older names for compatibility.
+
+### `MetadataCache`
+
+`MetadataCache` stores the JSON metadata cache.
+
+```python
+cache = wmlinksfromhell.MetadataCache()
+```
+
+Useful members:
+
+```python
+cache.path
+cache.default_path()
+cache.load()
+cache.save(data)
+cache.is_expired(max_age_seconds)
+cache.clear()
+cache.empty()
+```
+
+A corrupt or incompatible cache is discarded rather than being used as if it were valid metadata.
+
+## Wiki
+
+`Wiki` is the preferred short name for `WikiInfo`.
+
+Fields:
+
+```python
+wiki.dbname
+wiki.family
+wiki.project
+wiki.language
+wiki.language_name
+wiki.hostname
+wiki.article_path
+wiki.script_path
+wiki.wikiid
+wiki.sitename
+wiki.api_url
+wiki.metadata_source
+wiki.site_code
+wiki.is_multilingual
+wiki.is_standard_project
+```
+
+Helpers:
+
+```python
+wiki.url
+wiki.page_url(title)
+wiki.interwiki(title)
+wiki.link(title)
+wiki.main_page_url()
+wiki.interwiki_info(metadata=None, source=None)
+wiki.as_dict()
+```
+
+## InterwikiInfo
+
+`InterwikiInfo` describes how a wiki is represented in an interwiki target.
+
+Common fields:
+
+```python
+info.prefix
+info.project
+info.language
+info.dbname
+info.wiki
+```
+
+Helpers:
+
+```python
+info.target(title)
+info.url(title, metadata=None, source=None)
+info.link(title, metadata=None, source=None)
+info.as_dict()
+```
+
+For example:
+
+```python
+info = wmlinksfromhell.interwiki('dewiktionary')
+info.target('Haus')
+# wikt:de:Haus
+```
+
+## InterwikiMap
+
+`InterwikiMap` contains the interwiki prefixes available to a source wiki.
+
+```python
+imap = metadata.interwiki_map('metawiki')
+```
+
+Members include:
+
+```python
+imap.entries
+imap.family_prefixes
+imap.single_wiki_prefixes
+imap.service_prefixes
+imap.chapter_prefixes
+imap.organization_entries
+```
+
+Lookup:
+
+```python
+imap.get(prefix)
+prefix in imap
+```
+
+## InterwikiEntry
+
+An `InterwikiEntry` is one record from the interwiki configuration.
+
+```python
+entry.prefix
+entry.url
+entry.local
+entry.trans
+entry.language
+entry.localinterwiki
+entry.extralanglink
+entry.linktext
+entry.sitename
+entry.wikiid
+entry.api
+entry.destination_dbname
+entry.destination_type
+entry.as_dict()
+```
+
+## Resolver
+
+`Resolver` is the lower-level interface used by the package functions.
+
+```python
+resolver = wmlinksfromhell.Resolver()
+```
+
+A shared metadata store can be supplied:
+
+```python
+metadata = wmlinksfromhell.Metadata()
+resolver = wmlinksfromhell.Resolver(metadata)
+```
+
+Methods:
+
+```python
+resolver.resolve(value, source=None, strict=False)
+resolver.resolve_url(value, source=None, strict=False)
+resolver.resolve_interwiki(value, source=None, strict=False)
+resolver.to_url(destination)
+resolver.to_interwiki(destination, source=None)
+resolver.to_local(destination, source=None)
+```
+
+With `strict=True`, unresolved input raises `ResolutionError` instead of returning a result with an unresolved status.
+
+## Enums
+
+### `DestinationType`
+
+```text
+WIKI
+ORGANIZATION
+SERVICE
+TOOL
+SPECIAL
+EXTERNAL
+UNKNOWN
+```
+
+### `ResolutionStatus`
+
+```text
+RESOLVED
+AMBIGUOUS
+CONTEXT_REQUIRED
+UNKNOWN_PREFIX
+UNKNOWN_DESTINATION
+METADATA_MISSING
+EXTERNAL
+MALFORMED
+UNSUPPORTED
+```
+
+### `SyntaxType`
+
+```text
+LOCAL_WIKILINK
+INTERWIKI_WIKILINK
+EXTERNAL_LINK
+BARE_URL
+```
+
+## Wikimedia prefix behaviour
+
+The resolver understands the usual language and project shortcuts.
+
+Examples:
+
+```text
+w:en:Apple
+wikt:simple:uppercase
+b:de:Main Page
+q:fr:Main Page
+c:File:Example.jpg
+d:Q42
+m:Main Page
+```
+
+Short project shortcuts can inherit the source language when switching between language-based project families. On a multilingual project, or when the shortcut names the source project family, English is used. Long project names such as `wikipedia:` and `wikibooks:` are normal interwiki entries and normally point to the English project.
+
+Prefixes can be chained from left to right. Examples include:
+
+```text
+:de:q:Hauptseite
+:m:en:About
+:pl:w:2006
+:w:it:b:Wiskunde
+:ja:ja:2006
+```
+
+A leading colon also changes ordinary MediaWiki link parsing. For example, on `dewiki`:
+
+```text
+[[:en:Apple]]
+```
+
+is a local link whose title is `en:Apple`, while:
+
+```text
+[[en:Apple]]
+```
+
+is an interlanguage link to English Wikipedia.
+
+## URL behaviour
+
+The URL resolver handles article URLs, `index.php` forms, encoded titles, fragments, revisions, diffs, actions, page IDs, and protocol-relative URLs.
+
+Examples:
+
+```text
+https://en.wikipedia.org/wiki/Apple
+https://en.wikipedia.org/wiki/Foo%20bar
+https://en.wikipedia.org/wiki/Apple#History
+https://en.wikipedia.org/w/index.php?title=Apple
+https://en.wikipedia.org/w/index.php?title=Apple&oldid=123456
+https://en.wikipedia.org/w/index.php?title=Apple&diff=123456
+https://en.wikipedia.org/w/index.php?title=Apple&action=edit
+https://en.wikipedia.org/w/index.php?curid=9906
+//en.wikipedia.org/wiki/Apple
+```
+
+Fragments are kept separately from titles. `curid` is kept as a page ID. Revision and diff information is kept in the destination rather than thrown away.
+
+## Revisions and diffs
+
+Where an interwiki representation exists, the resolver uses special-page targets:
+
+```text
+oldid=123456
+    → w:en:Special:PermanentLink/123456
+
+diff=123456
+    → w:en:Special:Diff/123456
+
+title=Apple&oldid=123&diff=124
+    → w:en:Special:Diff/123/124
+
+action=edit&title=Apple
+    → w:en:Special:Edit/Apple
+```
+
+Symbolic diffs such as `prev`, `next`, and `cur` stay attached to the destination. If an operation or query parameter cannot be represented by an interwiki target but the page title is known, `to_interwiki()` uses the normal page target rather than silently losing the page.
+
+## Local and relative links
+
+These forms depend on source context:
+
+```text
+#History
+/example
+../example2
+```
+
+Without enough source information, resolution returns `CONTEXT_REQUIRED` instead of guessing.
+
+A Pywikibot `Page` is useful here because it supplies both the wiki and page title.
+
+## Comments
+
+Links inside HTML comments are ignored by default:
+
+```python
+code.filter_links()
+```
+
+Include them with:
+
+```python
+code.filter_links(comment_links=True)
+```
+
+Select only comment links with:
+
+```python
+code.filter_links(
+    comment_links=True,
+    in_comment=True,
+)
+```
+
+Comment links use the same `Link` and conversion APIs as other links.
+
+## Conversion safety
+
+Conversions do not intentionally throw away meaningful destination information.
+
+For example, a URL can contain a revision, diff, action, fragment, or extra query parameters that have no exact interwiki spelling. When the page title is known, `to_interwiki()` can fall back to the normal page target; when even that would lose the destination itself, conversion raises `ConversionError`.
+
+Category and file links are also treated carefully. A normal `[[Category:...]]` or `[[File:...]]` has MediaWiki behaviour beyond a clickable link, so it is not silently rewritten as a URL. A leading-colon form such as `[[:Category:Help]]` is an ordinary clickable link and can be converted.
+
+## Exceptions
+
+```python
+wmlinksfromhell.ConversionError
+wmlinksfromhell.MetadataMissingError
+wmlinksfromhell.SourceResolutionError
+wmlinksfromhell.ResolutionError
+wmlinksfromhell.CacheError
+wmlinksfromhell.WMLinksFromHellError
+```
+
+`WMLinkFromHellError` and `Error` remain as compatibility aliases for `WMLinksFromHellError`.
+
+Normal non-strict resolution returns a `Result` with a status. Use strict mode when you want unresolved input to raise:
+
+```python
+result = wmlinksfromhell.resolve(value, strict=True)
+```
+
+## Compatibility names
+
+The shorter names are preferred for new code:
+
+```python
+Code
+Link
+Wiki
+Metadata
+Result
+```
+
+The older/public names remain available:
+
+```python
+WMCode
+WMLink
+WikiInfo
+MetadataStore
+ResolutionResult
+```
+
+## Pywikibot
+
+Pywikibot is optional.
+
+```python
+import pywikibot
+import wmlinksfromhell
+
+site = pywikibot.Site('meta', 'meta')
+page = pywikibot.Page(site, 'Global sysops/Requests')
+
+code = wmlinksfromhell.parse_page(page)
+```
+
+You can also pass the page to `parse()`:
+
+```python
+code = wmlinksfromhell.parse(
+    page.text,
+    source=page,
+)
+```
+
+## Examples
+
+The `examples/` directory contains small scripts for:
+
+- the simple API
+- selecting links
+- converting links
+- comment links
+- SiteMatrix metadata
+- Pywikibot
+
+For installation and a quick introduction, see [README.md](README.md).