wikinator 0.6.2__tar.gz → 0.7.0__tar.gz

{wikinator-0.6.2 → wikinator-0.7.0}/PKG-INFO

@@ -1,12 +1,16 @@
 Metadata-Version: 2.3
 Name: wikinator
-Version: 0.6.2
+Version: 0.7.0
 Summary: Convert Google docs to markdown
+Requires-Dist: confuse>=2.2.0
+Requires-Dist: cryptography>=46.0.5
 Requires-Dist: dotenv>=0.9.9
 Requires-Dist: google-api-python-client>=2.175.0
 Requires-Dist: google-auth-httplib2>=0.2.0
 Requires-Dist: google-auth-oauthlib>=1.2.2
 Requires-Dist: gql[all]>=3.5.3
+Requires-Dist: humanize>=4.13.0
+Requires-Dist: pillow>=11.3.0
 Requires-Dist: pytest>=8.4.1
 Requires-Dist: python-docx>=1.2.0
 Requires-Dist: typer-slim>=0.16.0
@@ -23,24 +27,22 @@ Convert a Google drive download into a markdown-based wiki.
 **Note**: This is a work in progress, and not all features will be supported or working properly.
 
 ## tl;dr
+[Install `uv`](https://docs.astral.sh/uv/getting-started/installation/) and then:
 ```
-uvx wikinator upload --help
-Usage: wikinator upload [OPTIONS] SOURCE WIKIROOT
-
-  Convert and upload a file hierarchy to a GraphQL wiki. Given a source
-  directrory, walk the directory tree and for each file: - if MD or image
-  file, upload at the same path relative to the wikiroot path - If DOCX,
-  convert to MD and upload at the same path... - Unknown files are skipped.
-  For example, with source=/src and wikiroot=/wiki/root, A DOCX file at
-  /src/dir/some_file.docx will be uploaded to /wiki/root/dir/some_file on the
-  wiki.
-
-Arguments:
-  SOURCE    [required]
-  WIKIROOT  defaults to root of wiki, /
+uvx wikinator --help
+Usage: wikinator [OPTIONS] COMMAND [ARGS]...
 
 Options:
-  --help  Show this message and exit.
+  --version  Display version and exit.
+  -v         Show verbose logging.
+  -vv        Show debug logging.
+  -vvv       Show full trace logging.
+  --help     Show this message and exit.
+
+Commands:
+  upload   Convert and upload a file hierarchy to a GraphQL wiki.
+  convert  Given the URL of a specific gdoc:
+  config   View or set configuration settings.
 ```
 
 Given a directory, convert supported file types into markdown-based files while maintaining names and directory structure. This can then be uploaded into various wiki systems.
@@ -61,44 +63,50 @@ The development log will be kept here until the 1.0 release.
 
 ## Usage
 ```
+uxv wikinator convert https://gdoc/full/url path=test
+
 uvx wikinator upload target_dir
 uvx wikinator upload target_file.md new/path
 ```
 
+`convert` will take a single URL to a google doc, convert it to markdown, and
+upload that to the configured GraphQL server. An optional `path` option is provided
+to specify to path in the wiki to upload the document to.
+
 `upload` loads a full directory into the wiki. In the above examples:
 - Upload the directoy tree at `target_dir` into the wikipath `target_dir`
-- Upload the file `target_file.md` into the wiki path `new/path/target_file`
+- Upload the file `target_file.docx` into the wiki path `new/path/target_file`
 
 Assuming the `en` locale, the final paths in the wiki will be:
 - $GRAPH_DB/en/target_dir/...
 - $GRAPH_DB/en/new/path/target_file
 
-## Install & Configure
+## Configuration
 There is nothing to install, the `wikinator` command can be run from anywhere [`uvx` is installed](https://docs.astral.sh/uv/getting-started/installation/).
 
-To upload to your wiki, you must have the URL and an authorization token.
-
-Create a file in your home directory: `~/.config/wikinator.env`
+To upload to your wiki, you must have:
+1. the URL
+2. an authentication token
 
-In that file, add your wiki GraphQL URL and the access token:
+Configuring the values in wikinator:
 ```
-GRAPH_DB=https://wiki.example.org/graphql
-AUTH_TOKEN=your-authorization-token
+uvx wikinator config db_url https://db.example.com/graphql
+uvx wikinator config db_token <authentication-token-for-your-graphdb>
 ```
 
+When accessing Google docs, `wikinator` will confirm access to the requested files with a browser-based user authentication. These details will be stored in the configuration directory (`uvx wikinator config config_dir`) in `token.json` for future use.
+
 ### wiki.js
 
-TODO: Details on setting up creds for wikijs.
+This section is specific to the getting configuration values for a wiki.js server.
 
-Once you have the URL and authorization token from wiki,js, add them to an `.env` file:
-```
-GRAPH_DB=https://wiki.example.com/graphql
-AUTH_TOKEN=kfhgdkfjhgkdjfhgkdjhfgk...
-```
+You'll need the URL of the server, and the [authentication token](https://docs.requarks.io/dev/api#authentication) for you account.
 
-Once this file is set up correctly, `wikinator` is run with:
+With those values, configure the `db_url` and `db_token` with the `wikinator config` command, as above.
+
+Once this file is set up correctly, confirm with with:
 ```
-uvx wikinator upload --help
+uvx wikinator config
 ```
 
 
@@ -119,6 +127,11 @@ uvx wikinator upload --help
 
 ## Development Log
 
+### 2026-03-08
+Adding `config` and `convert` commands.
+- `config` helps manage config
+- `convert` will read, convert and upload a google doc
+
 ### 2025-08-07
 Refactored and disabled (for now) the convert, extract and teleport commands. The code remains in place, but the
 current focus is on convert and upload to graphql, and I wan to disable any code that's not in that path while testing.

{wikinator-0.6.2 → wikinator-0.7.0}/README.md

@@ -5,24 +5,22 @@ Convert a Google drive download into a markdown-based wiki.
 **Note**: This is a work in progress, and not all features will be supported or working properly.
 
 ## tl;dr
+[Install `uv`](https://docs.astral.sh/uv/getting-started/installation/) and then:
 ```
-uvx wikinator upload --help
-Usage: wikinator upload [OPTIONS] SOURCE WIKIROOT
-
-  Convert and upload a file hierarchy to a GraphQL wiki. Given a source
-  directrory, walk the directory tree and for each file: - if MD or image
-  file, upload at the same path relative to the wikiroot path - If DOCX,
-  convert to MD and upload at the same path... - Unknown files are skipped.
-  For example, with source=/src and wikiroot=/wiki/root, A DOCX file at
-  /src/dir/some_file.docx will be uploaded to /wiki/root/dir/some_file on the
-  wiki.
-
-Arguments:
-  SOURCE    [required]
-  WIKIROOT  defaults to root of wiki, /
+uvx wikinator --help
+Usage: wikinator [OPTIONS] COMMAND [ARGS]...
 
 Options:
-  --help  Show this message and exit.
+  --version  Display version and exit.
+  -v         Show verbose logging.
+  -vv        Show debug logging.
+  -vvv       Show full trace logging.
+  --help     Show this message and exit.
+
+Commands:
+  upload   Convert and upload a file hierarchy to a GraphQL wiki.
+  convert  Given the URL of a specific gdoc:
+  config   View or set configuration settings.
 ```
 
 Given a directory, convert supported file types into markdown-based files while maintaining names and directory structure. This can then be uploaded into various wiki systems.
@@ -43,44 +41,50 @@ The development log will be kept here until the 1.0 release.
 
 ## Usage
 ```
+uxv wikinator convert https://gdoc/full/url path=test
+
 uvx wikinator upload target_dir
 uvx wikinator upload target_file.md new/path
 ```
 
+`convert` will take a single URL to a google doc, convert it to markdown, and
+upload that to the configured GraphQL server. An optional `path` option is provided
+to specify to path in the wiki to upload the document to.
+
 `upload` loads a full directory into the wiki. In the above examples:
 - Upload the directoy tree at `target_dir` into the wikipath `target_dir`
-- Upload the file `target_file.md` into the wiki path `new/path/target_file`
+- Upload the file `target_file.docx` into the wiki path `new/path/target_file`
 
 Assuming the `en` locale, the final paths in the wiki will be:
 - $GRAPH_DB/en/target_dir/...
 - $GRAPH_DB/en/new/path/target_file
 
-## Install & Configure
+## Configuration
 There is nothing to install, the `wikinator` command can be run from anywhere [`uvx` is installed](https://docs.astral.sh/uv/getting-started/installation/).
 
-To upload to your wiki, you must have the URL and an authorization token.
-
-Create a file in your home directory: `~/.config/wikinator.env`
+To upload to your wiki, you must have:
+1. the URL
+2. an authentication token
 
-In that file, add your wiki GraphQL URL and the access token:
+Configuring the values in wikinator:
 ```
-GRAPH_DB=https://wiki.example.org/graphql
-AUTH_TOKEN=your-authorization-token
+uvx wikinator config db_url https://db.example.com/graphql
+uvx wikinator config db_token <authentication-token-for-your-graphdb>
 ```
 
+When accessing Google docs, `wikinator` will confirm access to the requested files with a browser-based user authentication. These details will be stored in the configuration directory (`uvx wikinator config config_dir`) in `token.json` for future use.
+
 ### wiki.js
 
-TODO: Details on setting up creds for wikijs.
+This section is specific to the getting configuration values for a wiki.js server.
 
-Once you have the URL and authorization token from wiki,js, add them to an `.env` file:
-```
-GRAPH_DB=https://wiki.example.com/graphql
-AUTH_TOKEN=kfhgdkfjhgkdjfhgkdjhfgk...
-```
+You'll need the URL of the server, and the [authentication token](https://docs.requarks.io/dev/api#authentication) for you account.
 
-Once this file is set up correctly, `wikinator` is run with:
+With those values, configure the `db_url` and `db_token` with the `wikinator config` command, as above.
+
+Once this file is set up correctly, confirm with with:
 ```
-uvx wikinator upload --help
+uvx wikinator config
 ```
 
 
@@ -101,6 +105,11 @@ uvx wikinator upload --help
 
 ## Development Log
 
+### 2026-03-08
+Adding `config` and `convert` commands.
+- `config` helps manage config
+- `convert` will read, convert and upload a google doc
+
 ### 2025-08-07
 Refactored and disabled (for now) the convert, extract and teleport commands. The code remains in place, but the
 current focus is on convert and upload to graphql, and I wan to disable any code that's not in that path while testing.

{wikinator-0.6.2 → wikinator-0.7.0}/pyproject.toml

@@ -1,15 +1,19 @@
 [project]
 name = "wikinator"
-version = "0.6.2"
+version = "0.7.0"
 description = "Convert Google docs to markdown"
 readme = "README.md"
 requires-python = ">=3.13"
 dependencies = [
+    "confuse>=2.2.0",
+    "cryptography>=46.0.5",
     "dotenv>=0.9.9",
     "google-api-python-client>=2.175.0",
     "google-auth-httplib2>=0.2.0",
     "google-auth-oauthlib>=1.2.2",
     "gql[all]>=3.5.3",
+    "humanize>=4.13.0",
+    "pillow>=11.3.0",
     "pytest>=8.4.1",
     "python-docx>=1.2.0",
     "typer-slim>=0.16.0",

{wikinator-0.6.2 → wikinator-0.7.0}/src/wikinator/__init__.py

@@ -1,22 +1,19 @@
 import importlib.metadata
 import logging
-import json
+
 import typer
-from typing import Optional
 from typing_extensions import Annotated
-import os
-from dotenv import load_dotenv
-
 
-from .wiki import GraphIngester, GraphDB
+from wikinator.config import AppConfig, __app_name__
+from wikinator.gdrive import GoogleDrive
+from wikinator.wiki import GraphDB as GraphDB
+from wikinator.wiki import GraphIngester
 
-
-__app_name__ = "wikinator"
-__app_version__ = importlib.metadata.version(__app_name__)
+#__app_version__ = importlib.metadata.version(__app_name__)
 
 
+app_config = AppConfig()
 log = logging.getLogger(__name__)
-
 app = typer.Typer(add_completion=False)
 
 
@@ -45,13 +42,17 @@ def init_logging(level:int) -> None:
     logging.basicConfig(level=logging.WARNING, format=fmt, style='{', handlers=(typer_handler,))
     # gql.transport.aiohttp = INFO for full transport logging
     log.setLevel(level)
-    log.info(f"starting {__app_name__} v{__app_version__}")
-    log.debug("debug logging enabled.")
+    log.info(f"starting {app_config.value('app_info')}")
+
+
+# initialize logging from config
+init_logging(logging.getLevelName(app_config.get('log_level').upper()))
+log.debug("debug logging enabled")
 
 
 def version_callback(value: bool) -> None:
     if value:
-        typer.echo(f"{__app_name__} v{__app_version__}")
+        print(f"{app_config.value('app_info')}")
         raise typer.Exit()
 
 
@@ -77,8 +78,8 @@ def trace_callback(value: bool) -> None:
 def upload(
     source: str,
     wikiroot: str = "/",
-    #url: Annotated[str, typer.Option("--db", help="URL of the GraphQL database")] = os.getenv("GRAPH_DB"),
-    #token: Annotated[str, typer.Option("--token", help="URL of the GraphQL database")] = os.getenv("AUTH_TOKEN"),
+    db_url: Annotated[str, typer.Option("--db", help="URL of the GraphQL database")] = app_config.get('db_url'),
+    db_token: Annotated[str, typer.Option("--token", help="URL of the GraphQL database")] = app_config.get('db_token'),
     output: Annotated[bool, typer.Option("-o", help="Make a local copy of the converted file")] = False,
 ) -> None:
     """
@@ -91,12 +92,9 @@ def upload(
     For example, with source=/src and wikiroot=/wiki/root,
     a DOCX file at /src/dir/some_file.docx will be uploaded to /wiki/root/dir/some_file on the wiki.
     """
-    load_dotenv(dotenv_path=os.path.expanduser("~/.config/wikinator.env"))
-    url = os.getenv("GRAPH_DB")
-    token = os.getenv("AUTH_TOKEN")
-    log.debug(f"Loaded env, db={url}")
+    GraphIngester(url=db_url, token=db_token, output=output).convert_directory(source, wikiroot)
+    raise typer.Exit()
 
-    GraphIngester(url=url, token=token, output=output).convert_directory(source, wikiroot)
 
 
 # Setup global options using callbacks
@@ -115,16 +113,71 @@ def common(
     pass
 
 
+@app.command()
+def convert(
+    doc_url: Annotated[str, typer.Argument(help="URL of the google-doc")],
+    db_url: Annotated[str, typer.Option("--db", help="URL of the GraphQL database.")] = app_config.get('db_url'),
+    path: Annotated[str, typer.Option("--path", help="Path to upload to. Defaults to '/' (root)")] = None,
+    name: Annotated[str, typer.Option("--name", help="Name of the uploaded file. Defaults to '<document-name>.md'")] = None,
+    token: Annotated[str, typer.Option("--token", help="Secure token for GraphQL database.")] = app_config.get('db_token'),
+) -> None:
+    """
+    Given the URL of a google doc, and options upload path: download the document, convert it to markdown, and upload
+    the converted file to a GraphQL wiki with /path/name.
+
+    Example:
+
+        uvx wikinator https://example.com/docker_tutorials.docx --path edu/tutorials
+
+    will create a new wiki entry '/edu/tutorials/docker_tutorials.md' from https://example.com/docker_tutorials.docx
 
-# TODO
+    GraphQL credentials can be set on the command line or configured with `wikinator config`. Loading a Googledoc will
+    invoke an authorization flow in the browser to confirm access to document. See README.md#Configuration for details.
+    """
+    log.debug(f"downloading {doc_url}")
+    page = GoogleDrive(*app_config.config_dir()).get_doc_url(doc_url)
 
+    if name is not None:
+        page.path = name
 
-# @app.command()
-# def convert(
-#     source: str,
-#     destination: Annotated[str, typer.Argument()] = "."
-# ) -> None:
-#     DocxitConverter().convert_directory(source, destination)
+    if path is not None:
+        page.update_path(path)
+
+    log.debug(f"uploading to {db_url}/{page.path}")
+    db = GraphDB(db_url, token)
+    db.update(page)
+
+    raise typer.Exit()
+
+
+@app.command()
+def config(
+    name: Annotated[str, typer.Argument(help="Display or set the name of a config value")] = None,
+    value: Annotated[str, typer.Argument(help="If provided, set the supplied name to this value")] = None,
+) -> None:
+    """
+    View or set configuration settings.
+
+    Example:
+
+        uvx wikinator config db_url https://mydb.example.com/graphql
+
+        uvx wikinator config db_token <authorization-token-for-graphdb>
+
+        uvx wikinator config
+    """
+    if name and value:
+        app_config.set(name, value)
+        app_config.write()
+        log.info(f"Value set: {name} = {value}")
+        raise typer.Exit()
+
+    if name:
+        print(app_config.value(name))
+        raise typer.Exit()
+
+    for k in app_config.keys():
+        print(f"{k}: {app_config.value(k)}")
 
 
 #- extract : from googledocs -> file system
@@ -151,4 +204,4 @@ def common(
 # ) -> None:
 #     wiki = GraphDB()
 #     for page in GoogleDrive().list_files("application/vnd.google-apps.document"):
-#         wiki.upload(page, wikiroot)
+#         wiki.upload(page, wikiroot)

wikinator-0.7.0/src/wikinator/config.py

@@ -0,0 +1,71 @@
+import importlib
+
+import confuse
+import os
+
+
+__app_name__ = "wikinator"
+__app_version__ = importlib.metadata.version(__app_name__)
+
+__config__ = confuse.Configuration(__app_name__, __name__)
+
+
+class AppConfig:
+    def __init__(self):
+        self._config = self._load()
+
+
+    def config_dir(self):
+        from .gdrive import config_link
+        return __config__.config_dir(), config_link
+
+
+    def _load(self):
+        cdir, _ = self.config_dir()
+        template = {
+            "db_url": f"Configure db_url in ${cdir}",
+            "db_token": f"Configure db_token in ${cdir}",
+            "log_level": "warning",
+        }
+        __config__.set_env()
+        config = __config__.get(template)
+        config['app_info'] = f"{__app_name__} v{__app_version__}"
+        config['config_dir'] = __config__.config_dir()
+        config['config_file'] = os.path.join(__config__.config_dir(), confuse.CONFIG_FILENAME)
+
+        # config["key"].redact = True
+        #config.dump(redact=True)
+        return config
+
+
+    def keys(self):
+        return self._config.keys()
+
+
+    def get(self, name:str, default:str = "") -> str:
+        return self._config.get(name, default)
+
+
+    def set(self, name:str, value) -> None:
+        self._config[name] = value
+
+
+    def value(self, name:str, default:str = "") -> str:
+        val = self._config.get(name, default)
+        if "secret" in name or "token" in name:
+            return f"{val[:6]}...{val[-6:]}"
+        else:
+            return val
+
+
+    _SKIP_NAMES = ["app_info", "config_dir", "config_file"]
+    def write(self):
+        # FIXME copy and remove values:   app_info: config_dir: config_file:
+
+        config_filename = os.path.join(__config__.config_dir(), confuse.CONFIG_FILENAME)
+        with open(config_filename, "w") as f:
+            #yaml.safe_dump(self._config, f) #, default_flow_style=False)
+            #f.write(self._config.dump())
+            for k, v in self._config.items():
+                if k not in self._SKIP_NAMES:
+                    f.write(f"{k}: {v}\n")

wikinator-0.7.0/src/wikinator/config_default.yaml

@@ -0,0 +1,5 @@
+
+db_url: http://example.com/graphdb
+db_token: FIXME-WITH-A-REAL-TOKEN
+
+log_level: warning