convoviz 0.2.3__tar.gz → 0.2.5__tar.gz

{convoviz-0.2.3 → convoviz-0.2.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: convoviz
-Version: 0.2.3
+Version: 0.2.5
 Summary: Get analytics and visualizations on your ChatGPT data!
 Keywords: markdown,chatgpt,openai,visualization,analytics,json,export,data-analysis,obsidian
 Author: Mohamed Cheikh Sidiya
@@ -24,7 +24,7 @@ Requires-Python: >=3.12
 Project-URL: Repository, https://github.com/mohamed-chs/chatgpt-history-export-to-md
 Description-Content-Type: text/markdown
 
-# Convoviz 📊: Visualize your entire ChatGPT data !
+# Convoviz 📊: Visualize your entire ChatGPT data
 
 Convert your ChatGPT history into well-formatted Markdown files. Additionally, visualize your data with word clouds 🔡☁️, view your prompt history graphs 📈, and access all your custom instructions 🤖 in a single location.
 
@@ -68,7 +68,7 @@ or pipx:
 pipx install convoviz
 ```
 
-### 3. Run the Script 🏃‍♂️
+### 3. Run the tool 🏃‍♂️
 
 Simply run the command and follow the prompts:
 
@@ -81,9 +81,18 @@ convoviz
 You can provide arguments directly to skip the prompts:
 
 ```bash
-convoviz --zip path/to/your/export.zip --output path/to/output/folder
+convoviz --input path/to/your/export.zip --output path/to/output/folder
 ```
 
+Inputs can be any of:
+- A ChatGPT export ZIP (downloaded from OpenAI)
+- An extracted export directory containing `conversations.json`
+- A `conversations.json` file directly
+
+Notes:
+- `--zip` / `-z` is kept as an alias for `--input` for convenience.
+- You can force non-interactive mode with `--no-interactive`.
+
 For more options, run:
 
 ```bash
@@ -118,4 +127,20 @@ It was also a great opportunity to learn more about Python and type annotations.
 
 It should(?) also work as library, so you can import and use the models and functions. I need to add more documentation for that tho. Feel free to reach out if you need help.
 
-I'm working on automating it to add new conversations and updating old ones. Had some luck with a JavaScript bookmarklet, still ironing it out tho.
+### Offline / reproducible runs
+
+Convoviz uses NLTK stopwords for word clouds. If you’re offline and NLTK data isn’t already installed, pre-download it once:
+
+```bash
+python -c "import nltk; nltk.download('stopwords')"
+```
+
+If you’re using `uv` without a global install, you can run:
+
+```bash
+uv run python -c "import nltk; nltk.download('stopwords')"
+```
+
+### Bookmarklet
+
+There’s also a JavaScript bookmarklet flow under `js/` (experimental) for exporting additional conversation data outside the official ZIP export.

{convoviz-0.2.3 → convoviz-0.2.5}/README.md

@@ -1,4 +1,4 @@
-# Convoviz 📊: Visualize your entire ChatGPT data !
+# Convoviz 📊: Visualize your entire ChatGPT data
 
 Convert your ChatGPT history into well-formatted Markdown files. Additionally, visualize your data with word clouds 🔡☁️, view your prompt history graphs 📈, and access all your custom instructions 🤖 in a single location.
 
@@ -42,7 +42,7 @@ or pipx:
 pipx install convoviz
 ```
 
-### 3. Run the Script 🏃‍♂️
+### 3. Run the tool 🏃‍♂️
 
 Simply run the command and follow the prompts:
 
@@ -55,9 +55,18 @@ convoviz
 You can provide arguments directly to skip the prompts:
 
 ```bash
-convoviz --zip path/to/your/export.zip --output path/to/output/folder
+convoviz --input path/to/your/export.zip --output path/to/output/folder
 ```
 
+Inputs can be any of:
+- A ChatGPT export ZIP (downloaded from OpenAI)
+- An extracted export directory containing `conversations.json`
+- A `conversations.json` file directly
+
+Notes:
+- `--zip` / `-z` is kept as an alias for `--input` for convenience.
+- You can force non-interactive mode with `--no-interactive`.
+
 For more options, run:
 
 ```bash
@@ -92,4 +101,20 @@ It was also a great opportunity to learn more about Python and type annotations.
 
 It should(?) also work as library, so you can import and use the models and functions. I need to add more documentation for that tho. Feel free to reach out if you need help.
 
-I'm working on automating it to add new conversations and updating old ones. Had some luck with a JavaScript bookmarklet, still ironing it out tho.
+### Offline / reproducible runs
+
+Convoviz uses NLTK stopwords for word clouds. If you’re offline and NLTK data isn’t already installed, pre-download it once:
+
+```bash
+python -c "import nltk; nltk.download('stopwords')"
+```
+
+If you’re using `uv` without a global install, you can run:
+
+```bash
+uv run python -c "import nltk; nltk.download('stopwords')"
+```
+
+### Bookmarklet
+
+There’s also a JavaScript bookmarklet flow under `js/` (experimental) for exporting additional conversation data outside the official ZIP export.

{convoviz-0.2.3 → convoviz-0.2.5}/convoviz/analysis/graphs.py

@@ -4,7 +4,9 @@ from collections import defaultdict
 from datetime import UTC, datetime
 from pathlib import Path
 
+import matplotlib.dates as mdates
 import matplotlib.font_manager as fm
+from matplotlib.axes import Axes
 from matplotlib.figure import Figure
 from tqdm import tqdm
 
@@ -23,10 +25,10 @@ WEEKDAYS = [
 ]
 
 
-def _setup_figure(config: GraphConfig) -> tuple[Figure, fm.FontProperties]:
+def _setup_figure(config: GraphConfig) -> tuple[Figure, Axes, fm.FontProperties]:
     """Internal helper to setup a figure with common styling."""
-    fig = Figure(figsize=config.figsize, dpi=300)
-    ax = fig.add_subplot()
+    fig = Figure(figsize=config.figsize, dpi=config.dpi)
+    ax: Axes = fig.add_subplot()
 
     # Load custom font if possible
     font_path = get_asset_path(f"fonts/{config.font_name}")
@@ -35,12 +37,27 @@ def _setup_figure(config: GraphConfig) -> tuple[Figure, fm.FontProperties]:
     )
 
     # Styling
+    fig.set_facecolor("white")
+    ax.set_facecolor("white")
     ax.spines["top"].set_visible(False)
     ax.spines["right"].set_visible(False)
     if config.grid:
         ax.grid(axis="y", linestyle="--", alpha=0.7)
+    ax.set_axisbelow(True)
 
-    return fig, font_prop
+    return fig, ax, font_prop
+
+
+def _ts_to_dt(ts: float, config: GraphConfig) -> datetime:
+    """Convert epoch timestamps into aware datetimes based on config."""
+    dt_utc = datetime.fromtimestamp(ts, UTC)
+    if config.timezone == "utc":
+        return dt_utc
+    return dt_utc.astimezone()
+
+
+def _tz_label(config: GraphConfig) -> str:
+    return "UTC" if config.timezone == "utc" else "Local"
 
 
 def generate_week_barplot(
@@ -59,37 +76,37 @@ def generate_week_barplot(
         Matplotlib Figure object
     """
     cfg = config or get_default_config().graph
-    dates = [datetime.fromtimestamp(ts, UTC) for ts in timestamps]
+    dates = [_ts_to_dt(ts, cfg) for ts in timestamps]
 
     weekday_counts: defaultdict[str, int] = defaultdict(int)
     for date in dates:
         weekday_counts[WEEKDAYS[date.weekday()]] += 1
 
-    x = WEEKDAYS
+    x = list(range(len(WEEKDAYS)))
     y = [weekday_counts[day] for day in WEEKDAYS]
 
-    fig, font_prop = _setup_figure(cfg)
-    ax = fig.gca()
+    fig, ax, font_prop = _setup_figure(cfg)
 
-    bars = ax.bar(x, y, color=cfg.color, alpha=0.8)
+    bars = ax.bar(x, y, color=cfg.color, alpha=0.85)
 
     if cfg.show_counts:
         for bar in bars:
             height = bar.get_height()
-            ax.text(
-                bar.get_x() + bar.get_width() / 2.0,
-                height,
-                f"{int(height)}",
-                ha="center",
-                va="bottom",
-                fontproperties=font_prop,
-            )
+            if height > 0:
+                ax.text(
+                    bar.get_x() + bar.get_width() / 2.0,
+                    height,
+                    f"{int(height)}",
+                    ha="center",
+                    va="bottom",
+                    fontproperties=font_prop,
+                )
 
     ax.set_xlabel("Weekday", fontproperties=font_prop)
-    ax.set_ylabel("Prompt Count", fontproperties=font_prop)
+    ax.set_ylabel("User Prompt Count", fontproperties=font_prop)
     ax.set_title(title, fontproperties=font_prop, fontsize=16, pad=20)
-    ax.set_xticks(range(len(x)))
-    ax.set_xticklabels(x, rotation=45, fontproperties=font_prop)
+    ax.set_xticks(x)
+    ax.set_xticklabels(WEEKDAYS, rotation=45, fontproperties=font_prop)
 
     for label in ax.get_yticklabels():
         label.set_fontproperties(font_prop)
@@ -114,7 +131,7 @@ def generate_hour_barplot(
         Matplotlib Figure object
     """
     cfg = config or get_default_config().graph
-    dates = [datetime.fromtimestamp(ts, UTC) for ts in timestamps]
+    dates = [_ts_to_dt(ts, cfg) for ts in timestamps]
 
     hour_counts: dict[int, int] = dict.fromkeys(range(24), 0)
     for date in dates:
@@ -123,8 +140,7 @@ def generate_hour_barplot(
     x = [f"{i:02d}:00" for i in range(24)]
     y = [hour_counts[i] for i in range(24)]
 
-    fig, font_prop = _setup_figure(cfg)
-    ax = fig.gca()
+    fig, ax, font_prop = _setup_figure(cfg)
 
     bars = ax.bar(range(24), y, color=cfg.color, alpha=0.8)
 
@@ -142,8 +158,8 @@ def generate_hour_barplot(
                     fontsize=8,
                 )
 
-    ax.set_xlabel("Hour of Day (UTC)", fontproperties=font_prop)
-    ax.set_ylabel("Prompt Count", fontproperties=font_prop)
+    ax.set_xlabel(f"Hour of Day ({_tz_label(cfg)})", fontproperties=font_prop)
+    ax.set_ylabel("User Prompt Count", fontproperties=font_prop)
     ax.set_title(f"{title} - Hourly Distribution", fontproperties=font_prop, fontsize=16, pad=20)
     ax.set_xticks(range(24))
     ax.set_xticklabels(x, rotation=90, fontproperties=font_prop)
@@ -180,8 +196,7 @@ def generate_model_piechart(
     total = sum(model_counts.values())
     if total == 0:
         # Return empty figure or figure with "No Data"
-        fig, font_prop = _setup_figure(cfg)
-        ax = fig.gca()
+        fig, ax, font_prop = _setup_figure(cfg)
         ax.text(0.5, 0.5, "No Data", ha="center", va="center", fontproperties=font_prop)
         return fig
 
@@ -204,8 +219,7 @@ def generate_model_piechart(
     labels = [item[0] for item in sorted_items]
     sizes = [item[1] for item in sorted_items]
 
-    fig, font_prop = _setup_figure(cfg)
-    ax = fig.gca()
+    fig, ax, font_prop = _setup_figure(cfg)
 
     colors = [
         "#4A90E2",
@@ -250,17 +264,16 @@ def generate_length_histogram(
     cfg = config or get_default_config().graph
     lengths = [conv.message_count("user") for conv in collection.conversations]
 
-    fig, font_prop = _setup_figure(cfg)
-    ax = fig.gca()
+    fig, ax, font_prop = _setup_figure(cfg)
 
     if not lengths:
         ax.text(0.5, 0.5, "No Data", ha="center", va="center", fontproperties=font_prop)
         return fig
 
-    import numpy as np
-
     # Cap at 95th percentile to focus on most conversations
-    cap = int(np.percentile(lengths, 95))
+    sorted_lengths = sorted(lengths)
+    idx = int(0.95 * (len(sorted_lengths) - 1))
+    cap = int(sorted_lengths[idx])
     cap = max(cap, 5)  # Ensure at least some range
 
     # Filter lengths for the histogram plot, but keep the data correct
@@ -306,10 +319,10 @@ def generate_monthly_activity_barplot(
     x = [m.strftime("%b '%y") for m in sorted_months]
     y = [len(month_groups[m].timestamps("user")) for m in sorted_months]
 
-    fig, font_prop = _setup_figure(cfg)
-    ax = fig.gca()
+    fig, ax, font_prop = _setup_figure(cfg)
 
-    bars = ax.bar(x, y, color=cfg.color, alpha=0.8)
+    positions = list(range(len(x)))
+    bars = ax.bar(positions, y, color=cfg.color, alpha=0.85)
 
     if cfg.show_counts:
         for bar in bars:
@@ -326,10 +339,12 @@ def generate_monthly_activity_barplot(
                 )
 
     ax.set_xlabel("Month", fontproperties=font_prop)
-    ax.set_ylabel("Total Prompt Count", fontproperties=font_prop)
+    ax.set_ylabel("User Prompt Count", fontproperties=font_prop)
     ax.set_title("Monthly Activity History", fontproperties=font_prop, fontsize=16, pad=20)
-    ax.set_xticks(range(len(x)))
-    ax.set_xticklabels(x, rotation=45, fontproperties=font_prop)
+    tick_step = max(1, len(positions) // 12)  # show ~12 labels max
+    shown = positions[::tick_step] if positions else []
+    ax.set_xticks(shown)
+    ax.set_xticklabels([x[i] for i in shown], rotation=45, fontproperties=font_prop)
 
     for label in ax.get_yticklabels():
         label.set_fontproperties(font_prop)
@@ -338,6 +353,45 @@ def generate_monthly_activity_barplot(
     return fig
 
 
+def generate_daily_activity_lineplot(
+    collection: ConversationCollection,
+    config: GraphConfig | None = None,
+) -> Figure:
+    """Create a line chart showing user prompt count per day."""
+    cfg = config or get_default_config().graph
+    timestamps = collection.timestamps("user")
+
+    fig, ax, font_prop = _setup_figure(cfg)
+    if not timestamps:
+        ax.text(0.5, 0.5, "No Data", ha="center", va="center", fontproperties=font_prop)
+        return fig
+
+    counts: defaultdict[datetime, int] = defaultdict(int)
+    for ts in timestamps:
+        dt = _ts_to_dt(ts, cfg)
+        day = dt.replace(hour=0, minute=0, second=0, microsecond=0)
+        counts[day] += 1
+
+    days = sorted(counts.keys())
+    values = [counts[d] for d in days]
+
+    x = mdates.date2num(days)
+    ax.plot(x, values, color=cfg.color, linewidth=2.0)
+    ax.fill_between(x, values, color=cfg.color, alpha=0.15)
+    locator = mdates.AutoDateLocator()
+    ax.xaxis.set_major_locator(locator)
+    ax.xaxis.set_major_formatter(mdates.ConciseDateFormatter(locator))
+    ax.set_title("Daily Activity History", fontproperties=font_prop, fontsize=16, pad=20)
+    ax.set_xlabel(f"Day ({_tz_label(cfg)})", fontproperties=font_prop)
+    ax.set_ylabel("User Prompt Count", fontproperties=font_prop)
+
+    for label in ax.get_xticklabels() + ax.get_yticklabels():
+        label.set_fontproperties(font_prop)
+
+    fig.tight_layout()
+    return fig
+
+
 def generate_summary_graphs(
     collection: ConversationCollection,
     output_dir: Path,
@@ -368,6 +422,10 @@ def generate_summary_graphs(
     fig_activity = generate_monthly_activity_barplot(collection, config)
     fig_activity.savefig(summary_dir / "monthly_activity.png")
 
+    # Daily activity
+    fig_daily = generate_daily_activity_lineplot(collection, config)
+    fig_daily.savefig(summary_dir / "daily_activity.png")
+
 
 def generate_graphs(
     collection: ConversationCollection,

{convoviz-0.2.3 → convoviz-0.2.5}/convoviz/analysis/wordcloud.py

@@ -62,7 +62,7 @@ def load_nltk_stopwords() -> frozenset[str]:
     return frozenset(words)
 
 
-def parse_custom_stopwords(stopwords_str: str) -> set[str]:
+def parse_custom_stopwords(stopwords_str: str | None) -> set[str]:
     """Parse a comma-separated string of custom stopwords.
 
     Args:

{convoviz-0.2.3 → convoviz-0.2.5}/convoviz/config.py

@@ -19,7 +19,7 @@ class MarkdownConfig(BaseModel):
     """Configuration for markdown output."""
 
     latex_delimiters: Literal["default", "dollars"] = "default"
-    flavor: Literal["obsidian", "standard"] = "obsidian"
+    flavor: Literal["obsidian", "standard"] = "standard"
 
 
 class YAMLConfig(BaseModel):
@@ -72,6 +72,8 @@ class GraphConfig(BaseModel):
     show_counts: bool = True
     font_name: str = "Montserrat-Regular.ttf"
     figsize: tuple[int, int] = (10, 6)
+    dpi: int = 300
+    timezone: Literal["utc", "local"] = "local"
 
 
 class ConvovizConfig(BaseModel):

convoviz-0.2.5/convoviz/interactive.py

@@ -0,0 +1,247 @@
+"""Interactive configuration prompts using questionary."""
+
+from pathlib import Path
+from typing import Literal, Protocol, cast
+
+from questionary import Choice, Style, checkbox, select
+from questionary import path as qst_path
+from questionary import text as qst_text
+
+from convoviz.config import ConvovizConfig, get_default_config
+from convoviz.io.loaders import find_latest_zip, validate_zip
+from convoviz.utils import colormaps, default_font_path, font_names, font_path, validate_header
+
+CUSTOM_STYLE = Style(
+    [
+        ("qmark", "fg:#34eb9b bold"),
+        ("question", "bold fg:#e0e0e0"),
+        ("answer", "fg:#34ebeb bold"),
+        ("pointer", "fg:#e834eb bold"),
+        ("highlighted", "fg:#349ceb bold"),
+        ("selected", "fg:#34ebeb"),
+        ("separator", "fg:#eb3434"),
+        ("instruction", "fg:#eb9434"),
+        ("text", "fg:#b2eb34"),
+        ("disabled", "fg:#858585 italic"),
+    ]
+)
+
+class _QuestionaryPrompt[T](Protocol):
+    def ask(self) -> T | None: ...
+
+
+def _ask_or_cancel[T](prompt: _QuestionaryPrompt[T]) -> T:
+    """Ask a questionary prompt; treat Ctrl+C/Ctrl+D as cancelling the run.
+
+    questionary's `.ask()` returns `None` on cancellation (Ctrl+C / Ctrl+D). We
+    convert that to `KeyboardInterrupt` so callers can abort the whole
+    interactive session with a single Ctrl+C.
+    """
+
+    result = prompt.ask()
+    if result is None:
+        raise KeyboardInterrupt
+    return result
+
+
+def _validate_input_path(raw: str) -> bool | str:
+    path = Path(raw)
+    if not path.exists():
+        return "Path must exist"
+
+    if path.is_dir():
+        if (path / "conversations.json").exists():
+            return True
+        return "Directory must contain conversations.json"
+
+    if path.suffix.lower() == ".json":
+        return True
+
+    if path.suffix.lower() == ".zip":
+        return True if validate_zip(path) else "ZIP must contain conversations.json"
+
+    return "Input must be a .zip, a .json, or a directory containing conversations.json"
+
+
+def run_interactive_config(initial_config: ConvovizConfig | None = None) -> ConvovizConfig:
+    """Run interactive prompts to configure convoviz.
+
+    Args:
+        initial_config: Optional starting configuration (uses defaults if None)
+
+    Returns:
+        Updated configuration based on user input
+    """
+    config = initial_config or get_default_config()
+
+    # Set sensible defaults if not already set
+    if not config.input_path:
+        latest = find_latest_zip()
+        if latest:
+            config.input_path = latest
+
+    if not config.wordcloud.font_path:
+        config.wordcloud.font_path = default_font_path()
+
+    # Prompt for input path
+    input_default = str(config.input_path) if config.input_path else ""
+    input_result: str = _ask_or_cancel(
+        qst_path(
+            "Enter the path to the export ZIP, conversations JSON, or extracted directory:",
+            default=input_default,
+            validate=_validate_input_path,
+            style=CUSTOM_STYLE,
+        )
+    )
+
+    if input_result:
+        config.input_path = Path(input_result)
+
+    # Prompt for output folder
+    output_result: str = _ask_or_cancel(
+        qst_path(
+            "Enter the path to the output folder:",
+            default=str(config.output_folder),
+            style=CUSTOM_STYLE,
+        )
+    )
+
+    if output_result:
+        config.output_folder = Path(output_result)
+
+    # Prompt for author headers
+    headers = config.message.author_headers
+    for role in ["system", "user", "assistant", "tool"]:
+        current = getattr(headers, role)
+        result: str = _ask_or_cancel(
+            qst_text(
+                f"Enter the message header for '{role}':",
+                default=current,
+                validate=lambda t: validate_header(t)
+                or "Must be a valid markdown header (e.g., # Title)",
+                style=CUSTOM_STYLE,
+            )
+        )
+        if result:
+            setattr(headers, role, result)
+
+    # Prompt for LaTeX delimiters
+    latex_result = cast(
+        Literal["default", "dollars"],
+        _ask_or_cancel(
+            select(
+                "Select the LaTeX math delimiters:",
+                choices=["default", "dollars"],
+                default=config.conversation.markdown.latex_delimiters,
+                style=CUSTOM_STYLE,
+            )
+        ),
+    )
+
+    if latex_result:
+        config.conversation.markdown.latex_delimiters = latex_result
+
+    # Prompt for markdown flavor
+    flavor_result = cast(
+        Literal["obsidian", "standard"],
+        _ask_or_cancel(
+            select(
+                "Select the markdown flavor:",
+                choices=["obsidian", "standard"],
+                default=config.conversation.markdown.flavor,
+                style=CUSTOM_STYLE,
+            )
+        ),
+    )
+
+    if flavor_result:
+        config.conversation.markdown.flavor = flavor_result
+
+    # Prompt for YAML headers
+    yaml_config = config.conversation.yaml
+    yaml_choices = [
+        Choice(title=field, checked=getattr(yaml_config, field))
+        for field in [
+            "title",
+            "tags",
+            "chat_link",
+            "create_time",
+            "update_time",
+            "model",
+            "used_plugins",
+            "message_count",
+            "content_types",
+            "custom_instructions",
+        ]
+    ]
+
+    selected: list[str] = _ask_or_cancel(
+        checkbox(
+            "Select YAML metadata headers to include:",
+            choices=yaml_choices,
+            style=CUSTOM_STYLE,
+        )
+    )
+
+    selected_set = set(selected)
+    for field_name in [
+        "title",
+        "tags",
+        "chat_link",
+        "create_time",
+        "update_time",
+        "model",
+        "used_plugins",
+        "message_count",
+        "content_types",
+        "custom_instructions",
+    ]:
+        setattr(yaml_config, field_name, field_name in selected_set)
+
+    # Prompt for font
+    available_fonts = font_names()
+    if available_fonts:
+        current_font = (
+            config.wordcloud.font_path.stem if config.wordcloud.font_path else available_fonts[0]
+        )
+        font_result: str = _ask_or_cancel(
+            select(
+                "Select the font for word clouds:",
+                choices=available_fonts,
+                default=current_font if current_font in available_fonts else available_fonts[0],
+                style=CUSTOM_STYLE,
+            )
+        )
+
+        if font_result:
+            config.wordcloud.font_path = font_path(font_result)
+
+    # Prompt for colormap
+    available_colormaps = colormaps()
+    if available_colormaps:
+        colormap_result: str = _ask_or_cancel(
+            select(
+                "Select the color theme for word clouds:",
+                choices=available_colormaps,
+                default=config.wordcloud.colormap
+                if config.wordcloud.colormap in available_colormaps
+                else available_colormaps[0],
+                style=CUSTOM_STYLE,
+            )
+        )
+
+        if colormap_result:
+            config.wordcloud.colormap = colormap_result
+
+    # Prompt for custom stopwords
+    stopwords_result: str = _ask_or_cancel(
+        qst_text(
+            "Enter custom stopwords (comma-separated):",
+            default=config.wordcloud.custom_stopwords,
+            style=CUSTOM_STYLE,
+        )
+    )
+
+    config.wordcloud.custom_stopwords = stopwords_result
+
+    return config