gemini-chat-exporter 0.1.3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (29) hide show
  1. gemini_chat_exporter-0.1.3/LICENSE +21 -0
  2. gemini_chat_exporter-0.1.3/PKG-INFO +504 -0
  3. gemini_chat_exporter-0.1.3/README.md +472 -0
  4. gemini_chat_exporter-0.1.3/gemini_chat_exporter.egg-info/PKG-INFO +504 -0
  5. gemini_chat_exporter-0.1.3/gemini_chat_exporter.egg-info/SOURCES.txt +27 -0
  6. gemini_chat_exporter-0.1.3/gemini_chat_exporter.egg-info/dependency_links.txt +1 -0
  7. gemini_chat_exporter-0.1.3/gemini_chat_exporter.egg-info/entry_points.txt +2 -0
  8. gemini_chat_exporter-0.1.3/gemini_chat_exporter.egg-info/requires.txt +5 -0
  9. gemini_chat_exporter-0.1.3/gemini_chat_exporter.egg-info/top_level.txt +1 -0
  10. gemini_chat_exporter-0.1.3/gemini_export/__init__.py +11 -0
  11. gemini_chat_exporter-0.1.3/gemini_export/cli.py +178 -0
  12. gemini_chat_exporter-0.1.3/gemini_export/config.py +93 -0
  13. gemini_chat_exporter-0.1.3/gemini_export/convert.py +139 -0
  14. gemini_chat_exporter-0.1.3/gemini_export/menu.py +106 -0
  15. gemini_chat_exporter-0.1.3/gemini_export/models.py +138 -0
  16. gemini_chat_exporter-0.1.3/gemini_export/parsers.py +109 -0
  17. gemini_chat_exporter-0.1.3/gemini_export/reconcile.py +80 -0
  18. gemini_chat_exporter-0.1.3/gemini_export/takeout.py +196 -0
  19. gemini_chat_exporter-0.1.3/gemini_export/verify.py +244 -0
  20. gemini_chat_exporter-0.1.3/pyproject.toml +74 -0
  21. gemini_chat_exporter-0.1.3/setup.cfg +4 -0
  22. gemini_chat_exporter-0.1.3/tests/test_cli.py +65 -0
  23. gemini_chat_exporter-0.1.3/tests/test_config.py +48 -0
  24. gemini_chat_exporter-0.1.3/tests/test_convert.py +78 -0
  25. gemini_chat_exporter-0.1.3/tests/test_models.py +63 -0
  26. gemini_chat_exporter-0.1.3/tests/test_parsers.py +50 -0
  27. gemini_chat_exporter-0.1.3/tests/test_reconcile.py +44 -0
  28. gemini_chat_exporter-0.1.3/tests/test_takeout.py +41 -0
  29. gemini_chat_exporter-0.1.3/tests/test_verify.py +90 -0
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 David Malko
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,504 @@
1
+ Metadata-Version: 2.4
2
+ Name: gemini-chat-exporter
3
+ Version: 0.1.3
4
+ Summary: Export & back up Google Gemini chat history to JSON + Markdown, with a corruption verifier
5
+ Author-email: David Malko <davidmalko87@gmail.com>
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/davidmalko87/gemini-chat-exporter
8
+ Project-URL: Repository, https://github.com/davidmalko87/gemini-chat-exporter
9
+ Project-URL: Changelog, https://github.com/davidmalko87/gemini-chat-exporter/blob/master/CHANGELOG.md
10
+ Project-URL: Bug Tracker, https://github.com/davidmalko87/gemini-chat-exporter/issues
11
+ Keywords: gemini,google-gemini,gemini-ai,chat-export,chat-history,conversation-export,backup,archiving,data-export,data-portability,google-takeout,markdown,json,userscript,tampermonkey,web-scraping,cli
12
+ Classifier: Development Status :: 4 - Beta
13
+ Classifier: Environment :: Console
14
+ Classifier: Intended Audience :: End Users/Desktop
15
+ Classifier: Operating System :: OS Independent
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3.10
18
+ Classifier: Programming Language :: Python :: 3.11
19
+ Classifier: Programming Language :: Python :: 3.12
20
+ Classifier: Programming Language :: Python :: 3.13
21
+ Classifier: Topic :: Communications :: Chat
22
+ Classifier: Topic :: System :: Archiving :: Backup
23
+ Classifier: Topic :: Utilities
24
+ Requires-Python: >=3.10
25
+ Description-Content-Type: text/markdown
26
+ License-File: LICENSE
27
+ Requires-Dist: python-dotenv>=1.0
28
+ Provides-Extra: test
29
+ Requires-Dist: pytest>=7.0; extra == "test"
30
+ Requires-Dist: ruff>=0.4; extra == "test"
31
+ Dynamic: license-file
32
+
33
+ # Gemini Chat Exporter
34
+
35
+ [![CI](https://github.com/davidmalko87/gemini-chat-exporter/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/davidmalko87/gemini-chat-exporter/actions/workflows/ci.yml)
36
+ [![PyPI version](https://img.shields.io/pypi/v/gemini-chat-exporter.svg)](https://pypi.org/project/gemini-chat-exporter/)
37
+ [![PyPI downloads](https://img.shields.io/pypi/dm/gemini-chat-exporter.svg)](https://pypi.org/project/gemini-chat-exporter/)
38
+ [![Python](https://img.shields.io/pypi/pyversions/gemini-chat-exporter.svg?logo=python&logoColor=white)](https://pypi.org/project/gemini-chat-exporter/)
39
+ [![Userscript](https://img.shields.io/badge/userscript-Tampermonkey-00485B.svg?logo=tampermonkey&logoColor=white)](#install)
40
+ [![Sources](https://img.shields.io/badge/sources-userscript%20%7C%20Takeout-success.svg)](#two-ways-to-export--and-why-it-matters)
41
+ [![Output](https://img.shields.io/badge/output-JSON%20%7C%20Markdown-success.svg)](#what-gets-exported)
42
+ [![Verified](https://img.shields.io/badge/corruption-verified-success.svg)](#-verified)
43
+ [![License](https://img.shields.io/badge/license-MIT-green.svg)](https://github.com/davidmalko87/gemini-chat-exporter/blob/master/LICENSE)
44
+ [![Platform](https://img.shields.io/badge/platform-Windows%20%7C%20macOS%20%7C%20Linux-lightgrey.svg)](#)
45
+ [![Last commit](https://img.shields.io/github/last-commit/davidmalko87/gemini-chat-exporter.svg)](https://github.com/davidmalko87/gemini-chat-exporter/commits/master)
46
+ [![GitHub issues](https://img.shields.io/github/issues/davidmalko87/gemini-chat-exporter.svg)](https://github.com/davidmalko87/gemini-chat-exporter/issues)
47
+
48
+ **Export & back up your Google Gemini chat history** to **JSON + Markdown** — from a
49
+ hardened **browser userscript** or from **Google Takeout** — with a built-in
50
+ **verifier** that refuses to let a silently corrupt export pass for a good one.
51
+ Fully local, no API key, no servers, no telemetry.
52
+
53
+ ---
54
+
55
+ ## Why?
56
+
57
+ Getting your own Gemini conversations out, completely and faithfully, is harder than it
58
+ should be:
59
+
60
+ - **Google Takeout is prompt-centric.** My Activity > Gemini Apps includes your
61
+ prompts, but often no model response, and the schema varies by account and is
62
+ under-documented — you have to verify what you actually got.
63
+ - **The page HTML has no conversation content.** Gemini is an Angular SPA; transcripts
64
+ are rendered client-side after load, and `gemini.google.com` blocks iframing and
65
+ background fetches. So a scraper has to drive the real, logged-in UI.
66
+ - **A naive scraper fails silently — and that is worse than failing loudly.** A prior
67
+ run of an earlier exporter scraped 473 conversations but only the first ~190 had real
68
+ bodies. The rest were overwritten with one duplicated stub, while it reported *zero
69
+ errors*. "Zero exceptions" is not "zero wrong content":
70
+
71
+ ```
72
+ [FAIL] 285/473 conversations (60.3%) share a body with another conversation
73
+ - exceeds the 2.0% threshold. This is the duplicate-stub corruption fingerprint.
74
+ [FAIL] contiguous duplicate run: conversations #193-#473 (281 in a row) all share
75
+ one body ("Spheres for Self-Realization") - classic stale-page corruption
76
+ RESULT: FAIL
77
+ ```
78
+
79
+ That report is this tool's verifier running on the actual corrupt file. The whole
80
+ project is built so that failure mode can never go unnoticed again.
81
+
82
+ For backing up other platforms, see the sibling project
83
+ [`jira-project-backup-restore`](https://github.com/davidmalko87/jira-project-backup-restore).
84
+
85
+ ---
86
+
87
+ ## How it works
88
+
89
+ It is **two steps**, and they use two different tools:
90
+
91
+ 1. **Capture** — a browser userscript adds an **"Export all"** button to Gemini.
92
+ It clicks through every conversation and streams them to one file,
93
+ `gemini_chats.ndjson`. *(This part runs in your browser because Gemini only
94
+ renders chats in the logged-in UI.)*
95
+ 2. **Process** — a small Python toolkit reads that file, **verifies** it isn't
96
+ corrupt, and **converts** it to readable per-conversation JSON + Markdown.
97
+
98
+ ```
99
+ Browser (userscript) Your computer (Python)
100
+ ┌──────────────────┐ ┌───────────────────────────────┐
101
+ │ "Export all" btn │ ─────────► │ gemini_chats.ndjson │
102
+ │ scrapes chats │ saves │ → gemini-export --verify │
103
+ └──────────────────┘ │ → gemini-export --convert │
104
+ │ → out/*.json + *.md │
105
+ └───────────────────────────────┘
106
+ ```
107
+
108
+ > Don't want to scrape? Skip step 1 and feed a **Google Takeout** export into
109
+ > step 2 instead (`--import-takeout`). See [Quick Start](#quick-start).
110
+
111
+ ---
112
+
113
+ ## Two ways to export — and why it matters
114
+
115
+ | Route | Fidelity | Cost / risk | Role |
116
+ |---|---|---|---|
117
+ | **Userscript / console scraper** (JS) | Full multi-turn transcripts | Undocumented selectors; fragile; *can* lag the UI | Convenient live capture — hardened |
118
+ | **Google Takeout** (Python importer) | Authoritative — Google generates it | Manual export; schema varies; My Activity is prompt-centric | Trustworthy / recovery route |
119
+ | **Verifier** (Python) | — | — | The safety net: catches duplicate-body / stale-page corruption and duplicate ids before you trust an export |
120
+
121
+ Use the scraper for convenience, Takeout for ground truth, and **always run the
122
+ verifier** on whatever you get (`--import-takeout` runs it for you on Takeout files).
123
+ `reconcile` diffs the two and lists which corrupt scraper conversations Takeout can
124
+ replace.
125
+
126
+ ---
127
+
128
+ ## Features
129
+
130
+ | Feature | Description |
131
+ |---|---|
132
+ | **Content-identity validation** | The scraper won't save a conversation until the URL id matches the target and the first prompt is present **and differs from the previous one** — the guard the old tool lacked |
133
+ | **Duplicate-body guard** | Each body is SHA-256 hashed; a body identical to the previous one is retried once, then marked `failed` instead of saved |
134
+ | **Streaming writes + resume** | Each conversation is written as one NDJSON line to the file you picked (or buffered and downloaded at the end when there is no save dialog); the file is committed when the run finishes. Already-saved ids are skipped on re-run |
135
+ | **Fail loudly** | Zero sidebar links aborts with a clear error instead of emitting empty output; a chat whose content never renders is marked `failed`, not saved |
136
+ | **Independent verifier** | `--verify` audits any export for duplicate bodies, ratio, contiguous runs and duplicate ids; exits non-zero for cron/CI |
137
+ | **Schema-tolerant Takeout import** | Handles both the My Activity array form and the conversations object form |
138
+ | **JSON + Markdown + manifest** | Per-conversation files plus `manifest.json` with checksums and a `complete` flag tied to verification |
139
+ | **Reconcile** | Diffs a scraper export against Takeout and flags which corrupt conversations are recoverable |
140
+
141
+ ---
142
+
143
+ ## Install
144
+
145
+ You install **two pieces** — the browser userscript (captures chats) and the
146
+ Python toolkit (verifies + converts).
147
+
148
+ ### 1. Browser userscript — the capture step
149
+
150
+ 1. Install a userscript manager — **[Tampermonkey](https://www.tampermonkey.net/)**
151
+ (Chrome, Edge, Firefox, Safari) is the easiest.
152
+ 2. Open this raw file and click **Install** in the dialog Tampermonkey shows:
153
+ **[gemini-chat-exporter.user.js](https://raw.githubusercontent.com/davidmalko87/gemini-chat-exporter/master/userscript/gemini-chat-exporter.user.js)**
154
+ 3. In the Tampermonkey dashboard, confirm the script header reads
155
+ **`@version 0.1.3`** (or newer). To update later: Tampermonkey ▸ *Check for
156
+ userscript updates*.
157
+
158
+ > **No userscript manager?** You don't need one. Copy all of
159
+ > [`userscript/console-snippet.js`](https://raw.githubusercontent.com/davidmalko87/gemini-chat-exporter/master/userscript/console-snippet.js), open
160
+ > <https://gemini.google.com/>, press **F12 ▸ Console**, paste it, and press Enter.
161
+
162
+ ### 2. Python toolkit — the verify / convert step
163
+
164
+ Requires **Python 3.10+**.
165
+
166
+ **Via PyPI (recommended)** — provides the `gemini-export` command:
167
+
168
+ ```bash
169
+ pip install gemini-chat-exporter
170
+ ```
171
+
172
+ **Or as an isolated app with [pipx](https://pipx.pypa.io/)** — keeps its dependencies out of your global Python:
173
+
174
+ ```bash
175
+ pipx install gemini-chat-exporter
176
+ ```
177
+
178
+ **Upgrade:**
179
+
180
+ ```bash
181
+ pip install -U gemini-chat-exporter # pipx: pipx upgrade gemini-chat-exporter
182
+ ```
183
+
184
+ **Or clone and run from source** (development):
185
+
186
+ ```bash
187
+ git clone https://github.com/davidmalko87/gemini-chat-exporter.git
188
+ cd gemini-chat-exporter
189
+ pip install -r requirements.txt
190
+ python main.py --help # same flags as gemini-export
191
+ ```
192
+
193
+ > **Where files live:** everything is relative to the folder you run `gemini-export`
194
+ > from. An optional `.env` is read only from that folder (variables already set in
195
+ > your shell win), `--verify` / `--reconcile` write `verify_report.json` /
196
+ > `reconcile_report.json` into it (or into `--out DIR`, which must already exist),
197
+ > and converted output goes to `./out` (change it with `--out DIR` or `OUTPUT_ROOT`).
198
+ > There is no log file and nothing is written to your home or install folder; a
199
+ > clone behaves the same way.
200
+
201
+ Check it works:
202
+
203
+ ```bash
204
+ gemini-export --help # all flags
205
+ gemini-export --version # installed version
206
+ ```
207
+
208
+ From a clone, use `python main.py` in place of `gemini-export` — the flags are identical.
209
+
210
+ ---
211
+
212
+ ## Quick Start
213
+
214
+ > Install both pieces first (see [Install](#install)).
215
+
216
+ ### Step 1 — Capture your chats from the browser
217
+
218
+ 1. Open **<https://gemini.google.com/>** and make sure you are **signed in**.
219
+ 2. Click the blue **"GCE: Export all"** button in the **bottom-right** of the page.
220
+ 3. Watch the button — it walks through these states:
221
+ - **`GCE: loading list... N`** — it opens the sidebar and scrolls your whole
222
+ chat list (can take a minute for hundreds of chats).
223
+ - A **save dialog** may appear — choose where to put **`gemini_chats.ndjson`**.
224
+ *(If it doesn't — your browser lacks the picker, or loading a long list took
225
+ too long — the file just downloads at the end instead.)*
226
+ - **`GCE: 12/542`** — it exports each conversation in turn. **Keep the tab open
227
+ and in front.** A full history takes roughly **30–45 minutes**.
228
+ 4. When it's done you get an alert: *"Done. N saved, M skipped, K failed..."*.
229
+
230
+ It is built to be safe and unattended-friendly:
231
+
232
+ - opens the sidebar if collapsed, and **re-opens it if it collapses mid-run**;
233
+ - loads the **full** lazy-paginated list (not just the first page);
234
+ - **skips chats already saved** when you run it again (resume) — but the file is
235
+ only written when a run finishes, so **keep the tab open until the "Done"
236
+ alert** (see [Troubleshooting](#troubleshooting) if a run was interrupted);
237
+ - **refuses stale or duplicate pages it detects** — those are marked `failed` and
238
+ retried on the next run, instead of silently corrupting your backup.
239
+
240
+ > **Prefer not to scrape?** Use Google Takeout as the source instead — it's the
241
+ > authoritative route. At [takeout.google.com](https://takeout.google.com/):
242
+ > *Deselect all* ▸ select **"My Activity"** ▸ set format to **JSON** ▸ click
243
+ > *"All activity data included"* and keep only **"Gemini Apps"** ▸ export.
244
+ > (The standalone **"Gemini"** product exports *Gems*, not your chats.) Then go
245
+ > to Step 3 and use `--import-takeout`.
246
+
247
+ ### Step 2 — Verify the capture (always do this)
248
+
249
+ Run this in the folder that holds `gemini_chats.ndjson`, or pass the file's full
250
+ path (e.g. `~/Downloads/gemini_chats.ndjson`):
251
+
252
+ ```bash
253
+ gemini-export --verify gemini_chats.ndjson
254
+ ```
255
+
256
+ You want **`RESULT: PASS`**. If it fails, the report names exactly what's wrong
257
+ (duplicate bodies, a stale-page run, duplicate ids) — **don't trust the export
258
+ until it passes**. (`--verify` exits non-zero on failure, so it fits cron/CI. It
259
+ also saves the report as `verify_report.json` in the current folder.)
260
+
261
+ A PASS means no corruption was found, not that every chat was captured: chats the
262
+ userscript marked `failed` only show up as a "no model text" warning, so also check
263
+ the *K failed* count in the final alert.
264
+
265
+ ### Step 3 — Convert to JSON + Markdown
266
+
267
+ ```bash
268
+ # from a scraper capture:
269
+ gemini-export --convert gemini_chats.ndjson --out ./out
270
+
271
+ # ...or straight from a Google Takeout export:
272
+ gemini-export --import-takeout "Takeout/My Activity/Gemini Apps/MyActivity.json" --out ./out
273
+ ```
274
+
275
+ You get one `.json` and one `.md` per conversation in `./out/`, plus a
276
+ `manifest.json` (checksums + a `complete` flag). To see which corrupt chats
277
+ Takeout can replace, diff the capture against Takeout:
278
+
279
+ ```bash
280
+ gemini-export --reconcile gemini_chats.ndjson MyActivity.json
281
+ ```
282
+
283
+ The console shows counts; the ids of the recoverable conversations are listed in
284
+ `reconcile_report.json` (written to the current folder). Reconcile only reports —
285
+ it does not change or merge either file.
286
+
287
+ ### More examples
288
+
289
+ **Verify:**
290
+
291
+ ```bash
292
+ gemini-export --validate gemini_chats.ndjson # same as --verify
293
+ gemini-export --verify gemini_chats.ndjson --dup-ratio 0.05 # custom duplicate threshold (0..1)
294
+ ```
295
+
296
+ **Convert / import:**
297
+
298
+ ```bash
299
+ gemini-export --convert gemini_chats.ndjson --out ./out --md-only # Markdown only (--json-only: JSON only)
300
+ gemini-export --convert gemini_chats.ndjson --dry-run # verify and count, write no files
301
+ gemini-export --import-takeout takeout-export.zip --out ./out # a Takeout .zip or extracted folder
302
+ ```
303
+
304
+ **Accepted inputs:** `--verify` / `--convert` read the scraper's `.ndjson`, a JSON
305
+ array (or `{"conversations": [...]}`), or the legacy delimited `.txt` export.
306
+ `--import-takeout` takes `MyActivity.json`, a Takeout `.zip`, or the extracted
307
+ folder — use the `.zip` / folder only when the export contains just *My Activity ▸
308
+ Gemini Apps* (as in the recipe above); otherwise point it at `MyActivity.json`.
309
+ Don't run `--verify` on a Takeout file; `--import-takeout` verifies it for you.
310
+
311
+ ### Don't like flags? Use the menu
312
+
313
+ From a clone, run `python main.py` with **no arguments** for an interactive menu
314
+ (the installed `gemini-export` command has no menu — with no arguments it prints
315
+ its help):
316
+
317
+ ```
318
+ ==================================================
319
+ Gemini Chat Exporter v0.1.3
320
+ ==================================================
321
+ --- Verify ---
322
+ 1) Verify / audit an export for corruption
323
+ --- Import & convert ---
324
+ 2) Import a Google Takeout export
325
+ 3) Convert an export to JSON + Markdown
326
+ --- Reconcile ---
327
+ 4) Reconcile a scraper export against Takeout
328
+ 0) Exit
329
+ ```
330
+
331
+ ---
332
+
333
+ ## Troubleshooting
334
+
335
+ | What you see | Why, and what to do |
336
+ |---|---|
337
+ | **"No conversation links found"** | The left sidebar was collapsed (only an issue on v0.1.1 and earlier). **Update to v0.1.3+** — it opens the sidebar automatically. One-off workaround: open the left sidebar so your chats are visible, then click Export. |
338
+ | **Total looks too low** (e.g. `X/333` when you have more chats) | An old version didn't load the lazy-paginated list. **Update to v0.1.2+.** |
339
+ | **Looks stuck; `FAIL` keeps climbing** | On a long run the sidebar collapsed and an old version couldn't recover. **Update to v0.1.3+** (it re-opens the sidebar and continues). |
340
+ | **A few chats marked `failed`** | Normal for very large conversations that don't finish rendering in time. Click **"Export all"** again — it retries only the failures (resume skips the rest). The re-run's file holds **only** the retried chats: keep it next to the first one (saving over `gemini_chats.ndjson` replaces your first capture), then verify and convert it separately (e.g. `--out ./out-retry`). |
341
+ | **A run was interrupted** (tab closed, browser crashed) | The file is only written when a run finishes, but the chats it reached are already remembered as saved, so a re-run would skip them. Before running again, clear the resume state: **F12 ▸ Application ▸ Local Storage ▸ `https://gemini.google.com`** ▸ delete `gce_saved_ids_v1`. |
342
+ | **Where did the file go?** | Wherever you chose in the save dialog. If no save dialog appeared, it's in your **Downloads** as `gemini_chats.ndjson`. |
343
+ | **`python: command not found`** | Try `python3`, or install **Python 3.10+**. |
344
+ | **`gemini-export: command not found`** | pip put the command in a scripts folder that isn't on your `PATH`. Use `pipx install gemini-chat-exporter` (then `pipx ensurepath` and open a new terminal), or activate the virtual environment you installed it into. From a clone, use `python main.py` instead. |
345
+ | **`error: externally-managed-environment`** | Your system Python (e.g. Debian/Ubuntu, Homebrew) blocks global `pip install` (PEP 668). Use `pipx install gemini-chat-exporter`, or create a virtual environment first: `python3 -m venv .venv`, activate it, then `pip install gemini-chat-exporter`. |
346
+ | **Verify says `FAIL`** | That's the safety net doing its job. Read the report, then re-capture or recover the flagged chats from Takeout before trusting the backup. |
347
+ | **Selectors broke after a Gemini update** | If a capture suddenly returns nothing, Gemini changed its DOM. Update the `CONFIG` selectors at the top of [`userscript/gemini-chat-exporter.user.js`](https://github.com/davidmalko87/gemini-chat-exporter/blob/master/userscript/gemini-chat-exporter.user.js) (or open an issue). |
348
+
349
+ ---
350
+
351
+ ## What Gets Exported
352
+
353
+ | File | Contents |
354
+ |---|---|
355
+ | `gemini_chats.ndjson` | Scraper output: one JSON conversation per line (`id`, `title`, `turns`, `source`, `flags`) — streaming-friendly and resumable |
356
+ | `NNNN_<slug>.json` | One structured JSON file per conversation (ASCII filename slugged from the title, or the conversation id when the title has no Latin letters or digits; UTF-8 content) |
357
+ | `NNNN_<slug>.md` | One readable Markdown transcript per conversation (David / Gemini headers) |
358
+ | `manifest.json` | Per-file SHA-256 checksums, body hashes, counts, and a `complete` flag that is only `true` when verification passed |
359
+ | `verify_report.json` | Full audit from `--verify` / `--validate`, written to the current folder (or `--out DIR`) |
360
+ | `reconcile_report.json` | Counts and the recoverable / corrupt conversation ids from `--reconcile`, written to the current folder (or `--out DIR`) |
361
+
362
+ ---
363
+
364
+ ## ✅ Verified
365
+
366
+ This tool's verifier is **proven against the real corrupt export** that prompted the
367
+ project: it correctly flags 285/473 conversations as sharing a duplicated body, names
368
+ the dominant stub (281x), and pinpoints the contiguous corruption run (#193-#473) — and
369
+ exits non-zero. That file is personal data and is not in the repo, so CI runs a
370
+ synthetic regression fixture with the same shape (a duplicated stub in a contiguous run)
371
+ on every push to `master` and every pull request, plus a clean synthetic export that
372
+ must pass. Both fixtures use the
373
+ legacy `.txt` format; there is no NDJSON (scrape-format) fixture yet.
374
+
375
+ Honest framing: the **scraper is best-effort** and depends on undocumented Gemini
376
+ internals; **Google Takeout is the authoritative route** for guaranteed-complete
377
+ history. The verifier exists precisely so you never have to take either source on faith.
378
+ An export is only "verified" once the verifier passes it — structural validity alone is
379
+ not enough.
380
+
381
+ ---
382
+
383
+ ## Known Limitations
384
+
385
+ These are platform/source constraints, stated honestly — not all of them are fixable:
386
+
387
+ | Area | Status | Notes |
388
+ |---|---|---|
389
+ | Gemini DOM selectors | Will break over time | Undocumented Angular internals; update `CONFIG` in the userscript when scraping returns nothing. If the sidebar links stop matching, the export aborts with an error; if the conversation content stops matching, chats are marked `failed` (watch the failed count). |
390
+ | Takeout "My Activity" | Prompt-centric | Often one record per prompt and frequently **no model response**; counted and reported (`without_response`). Use the scraper for full transcripts, and `reconcile` to cross-check. |
391
+ | Images / Canvas / Gems | Not captured | Image-only replies are flagged (empty model text); generated images, Canvas docs, and Gems are out of scope. |
392
+ | Already-corrupted dumps | Not recoverable from themselves | Once a body was overwritten by a stub it is gone; `reconcile` lists which of those ids Takeout still has. |
393
+ | Re-importing into Gemini | Not possible | There is no write API; this is export/backup only (no "restore"). |
394
+
395
+ ---
396
+
397
+ ## Project Structure
398
+
399
+ ```
400
+ gemini-chat-exporter/
401
+ ├── main.py # From a clone: interactive menu (no args) + CLI flags
402
+ ├── pyproject.toml # Package metadata; installs the gemini-export command
403
+ ├── requirements.txt # Runtime dependency for a clone (python-dotenv)
404
+ ├── .env.example # Optional settings template
405
+ ├── userscript/
406
+ │ ├── gemini-chat-exporter.user.js # Hardened Tampermonkey scraper
407
+ │ ├── console-snippet.js # DevTools-paste fallback
408
+ │ └── README.md # Browser-capture install + usage
409
+ ├── gemini_export/
410
+ │ ├── __init__.py # Package version
411
+ │ ├── models.py # Conversation/Turn model + body hashing (dup detection)
412
+ │ ├── config.py # Optional .env loader and validation
413
+ │ ├── parsers.py # NDJSON / legacy-txt / JSON readers
414
+ │ ├── verify.py # The safety net: corruption + duplicate-id audit
415
+ │ ├── takeout.py # Schema-tolerant Google Takeout importer
416
+ │ ├── convert.py # Per-conversation JSON + Markdown + manifest (SHA-256)
417
+ │ ├── reconcile.py # Diff scraper export vs Takeout
418
+ │ ├── cli.py # Argparse CLI behind gemini-export (UTF-8-safe console)
419
+ │ └── menu.py # Interactive menu
420
+ ├── tests/ # 40 offline pytest cases (synthetic fixtures only)
421
+ ├── .github/
422
+ │ ├── workflows/ci.yml # ruff + pytest (3.10-3.13) + node --check userscript
423
+ │ ├── workflows/publish.yml # Publishes to PyPI when a GitHub Release is published
424
+ │ └── dependabot.yml # Monthly dependency update PRs
425
+ ├── CHANGELOG.md # Version history
426
+ ├── CONTRIBUTING.md # Ground rules + release steps
427
+ ├── LICENSE # MIT
428
+ └── PROMPTS.md # Agent-Coding Kit (the workflow that built this)
429
+ ```
430
+
431
+ ---
432
+
433
+ ## Configuration Reference
434
+
435
+ The tool needs no credentials. Optional tunables can be set as environment
436
+ variables, in a `.env` file in the folder you run `gemini-export` from, or as CLI
437
+ flags (flags win over both; variables already set in your shell win over `.env`).
438
+ `.env.example` is not included in the PyPI package, so create `.env` yourself:
439
+
440
+ ```ini
441
+ OUTPUT_ROOT=./out
442
+ DUP_RATIO_FAIL=0.02
443
+ MIN_RUN=5
444
+ ```
445
+
446
+ …or download the template (from a clone: `cp .env.example .env`):
447
+
448
+ ```bash
449
+ curl -o .env https://raw.githubusercontent.com/davidmalko87/gemini-chat-exporter/master/.env.example
450
+ ```
451
+
452
+ | Setting | CLI flag | Default | Description |
453
+ |---|---|---|---|
454
+ | `OUTPUT_ROOT` | `--out DIR` | `./out` | Where converted output is written (`--out` also moves the report files) |
455
+ | `DUP_RATIO_FAIL` | `--dup-ratio F` | `0.02` | Fail if more than this fraction (0..1) of conversations share a body |
456
+ | `MIN_RUN` | — | `5` | Min consecutive same-body run length to flag |
457
+ | — | `--json-only` | off | Write only the per-conversation JSON (skip Markdown) |
458
+ | — | `--md-only` | off | Write only the Markdown transcripts (skip JSON) |
459
+ | — | `--dry-run` | off | For `--convert` / `--import-takeout`: verify and count, but write no files (`--verify` / `--reconcile` still write their report) |
460
+
461
+ `WRITE_JSON` / `WRITE_MARKDOWN` from older `.env` files no longer change the output
462
+ (and setting both to `false` is rejected as invalid); remove them and use
463
+ `--json-only` / `--md-only`.
464
+
465
+ ---
466
+
467
+ ## Requirements
468
+
469
+ - Python **3.10+**
470
+ - [`python-dotenv`](https://pypi.org/project/python-dotenv/) >= 1.0 (the only runtime dependency)
471
+ - A browser + userscript manager (for live scraping); the toolkit itself is offline
472
+
473
+ ---
474
+
475
+ ## Changelog
476
+
477
+ See [CHANGELOG.md](https://github.com/davidmalko87/gemini-chat-exporter/blob/master/CHANGELOG.md) for the full version history.
478
+
479
+ ## Contributing
480
+
481
+ See [CONTRIBUTING.md](https://github.com/davidmalko87/gemini-chat-exporter/blob/master/CONTRIBUTING.md). In short: no personal data in the repo,
482
+ synthetic test fixtures only, ASCII console output, fail loudly, prove it with the
483
+ verifier.
484
+
485
+ ## License
486
+
487
+ [MIT](https://github.com/davidmalko87/gemini-chat-exporter/blob/master/LICENSE)
488
+
489
+ ## Related
490
+
491
+ - [`jira-project-backup-restore`](https://github.com/davidmalko87/jira-project-backup-restore)
492
+ — backup & restore individual Jira Cloud projects via REST API.
493
+
494
+ <!--
495
+ GitHub repo "About" settings (the SEO layer, set in the repo's About panel):
496
+ DESCRIPTION:
497
+ Export & back up your Google Gemini chat history to JSON + Markdown - via browser
498
+ userscript or Google Takeout - with a built-in corruption (duplicate/stale-page)
499
+ verifier. Local & private, no API key.
500
+ TOPICS:
501
+ google-gemini, gemini, gemini-ai, chat-export, conversation-export, chat-history,
502
+ backup, data-export, google-takeout, data-portability, userscript, tampermonkey,
503
+ markdown, json, web-scraping, cli, python, archiving, data-integrity
504
+ -->