@intentsolutionsio/jeremy-firestore 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +26 -0
- package/LICENSE +21 -0
- package/README.md +615 -0
- package/agents/firebase-operations-agent.md +411 -0
- package/agents/firestore-security-agent.md +478 -0
- package/commands/firestore-setup.md +543 -0
- package/package.json +48 -0
- package/skills/firestore-operations-manager/ARD.md +215 -0
- package/skills/firestore-operations-manager/PRD.md +106 -0
- package/skills/firestore-operations-manager/SKILL.md +67 -0
- package/skills/firestore-operations-manager/references/errors.md +85 -0
- package/skills/firestore-operations-manager/references/examples.md +211 -0
- package/skills/firestore-operations-manager/references/implementation.md +214 -0
- package/skills/firestore-operations-manager/scripts/setup-firestore.sh +63 -0
|
@@ -0,0 +1,215 @@
|
|
|
1
|
+
# ARD: Firestore Operations Manager Skill
|
|
2
|
+
|
|
3
|
+
> Part of [Tons of Skills](https://tonsofskills.com) by [Intent Solutions](https://intentsolutions.io) | [jeremylongshore.com](https://jeremylongshore.com)
|
|
4
|
+
|
|
5
|
+
## System Context
|
|
6
|
+
|
|
7
|
+
This skill operates within the Firestore ecosystem on Google Cloud Platform. The primary components are:
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
┌──────────────┐ ┌───────────────────┐ ┌──────────────────┐
|
|
11
|
+
│ Admin SDK │────▶│ Cloud Firestore │────▶│ Composite Indexes│
|
|
12
|
+
│ (Node.js) │ │ (Document DB) │ │ (auto + manual) │
|
|
13
|
+
└──────────────┘ └───────┬───────────┘ └──────────────────┘
|
|
14
|
+
│
|
|
15
|
+
┌─────────┴──────────┐
|
|
16
|
+
│ │
|
|
17
|
+
┌─────▼──────┐ ┌───────▼────────┐
|
|
18
|
+
│ Security │ │ Firestore │
|
|
19
|
+
│ Rules │ │ Emulator │
|
|
20
|
+
│ (deploy) │ │ (localhost:8080) │
|
|
21
|
+
└────────────┘ └────────────────┘
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
### External Systems
|
|
25
|
+
|
|
26
|
+
| System | Role | Interface |
|
|
27
|
+
|--------|------|-----------|
|
|
28
|
+
| Cloud Firestore | Document database, query engine | Admin SDK `firestore()`, REST API |
|
|
29
|
+
| Security Rules | Access control layer evaluated on every read/write | `firestore.rules` file deployed via CLI |
|
|
30
|
+
| Composite Indexes | Required for multi-field queries | `firestore.indexes.json` deployed via CLI |
|
|
31
|
+
| Firestore Emulator | Local Firestore replica for testing | `localhost:8080`, reset between test runs |
|
|
32
|
+
| Firebase Auth | Identity provider (rules reference `request.auth`) | Auth emulator at `localhost:9099` for testing |
|
|
33
|
+
|
|
34
|
+
## Data Flow
|
|
35
|
+
|
|
36
|
+
### Standard CRUD Flow
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
1. Identify operation type (create/read/update/delete)
|
|
40
|
+
2. Validate schema: read sample doc to understand existing fields
|
|
41
|
+
3. Check security rules: will the operation be allowed?
|
|
42
|
+
4. Check indexes: does the query need a composite index?
|
|
43
|
+
5. Execute operation (single doc or batch)
|
|
44
|
+
6. Verify result (read-after-write or emulator assertion)
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
### Batch Migration Flow
|
|
48
|
+
|
|
49
|
+
```
|
|
50
|
+
1. Read migration spec: source collection, target field(s), transform function
|
|
51
|
+
2. Check for existing checkpoint in _migrations/{migrationId}
|
|
52
|
+
3. Query next batch of documents (500, starting after checkpoint cursor)
|
|
53
|
+
4. Apply transform to each document
|
|
54
|
+
5. Commit batch (500 writes max per commit)
|
|
55
|
+
6. Write checkpoint: { lastDocId, processedCount, status: "in_progress" }
|
|
56
|
+
7. Repeat steps 3-6 until no more documents
|
|
57
|
+
8. Write final checkpoint: { processedCount, skippedCount, status: "completed" }
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
### Security Rules Testing Flow
|
|
61
|
+
|
|
62
|
+
```
|
|
63
|
+
1. Write firestore.rules with helper functions
|
|
64
|
+
2. Start Firestore + Auth emulators
|
|
65
|
+
3. Create test contexts (authenticated, unauthenticated, admin)
|
|
66
|
+
4. Assert each access pattern (read/write per collection per role)
|
|
67
|
+
5. Fix failing assertions by adjusting rules
|
|
68
|
+
6. Deploy: firebase deploy --only firestore:rules
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
## Design Decisions
|
|
72
|
+
|
|
73
|
+
### DD-1: Batch Over Individual Writes
|
|
74
|
+
|
|
75
|
+
**Decision**: All multi-document operations use `WriteBatch` (up to 500 operations) rather than individual `doc.set()` or `doc.update()` calls.
|
|
76
|
+
|
|
77
|
+
**Rationale**: Individual writes incur one round trip each. A batch of 500 writes completes in a single round trip, reducing latency by ~500x and cost by reducing billable operations. Firestore enforces a hard limit of 500 operations per batch commit.
|
|
78
|
+
|
|
79
|
+
**Implementation**: Chunk document arrays into groups of 500, commit each chunk, and log progress:
|
|
80
|
+
|
|
81
|
+
```typescript
|
|
82
|
+
const BATCH_SIZE = 500;
|
|
83
|
+
for (let i = 0; i < docs.length; i += BATCH_SIZE) {
|
|
84
|
+
const batch = db.batch();
|
|
85
|
+
const chunk = docs.slice(i, i + BATCH_SIZE);
|
|
86
|
+
chunk.forEach(doc => batch.update(doc.ref, transform(doc.data())));
|
|
87
|
+
await batch.commit();
|
|
88
|
+
console.log(`Committed ${Math.min(i + BATCH_SIZE, docs.length)} / ${docs.length}`);
|
|
89
|
+
}
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
### DD-2: Cursor-Based Pagination for Large Reads
|
|
93
|
+
|
|
94
|
+
**Decision**: Queries that may return more than 100 documents use `startAfter(lastDoc)` cursor pagination, never `offset()`.
|
|
95
|
+
|
|
96
|
+
**Rationale**: Firestore's `offset(N)` still reads and bills for the skipped N documents. Cursor-based pagination with `startAfter()` reads only the next page, making it O(pageSize) per page instead of O(offset + pageSize).
|
|
97
|
+
|
|
98
|
+
**Trade-off**: Requires storing the last document snapshot or a deterministic sort field. All paginated queries must include an `orderBy()` clause.
|
|
99
|
+
|
|
100
|
+
### DD-3: Emulator-First Testing
|
|
101
|
+
|
|
102
|
+
**Decision**: All security rules and query patterns are tested against the Firestore emulator before any production deployment.
|
|
103
|
+
|
|
104
|
+
**Rationale**: Security rules errors in production silently block operations with `PERMISSION_DENIED`. The emulator provides instant feedback and supports `@firebase/rules-unit-testing` for programmatic assertions. Emulator tests run in < 5 seconds versus deploying rules to production (30-60 seconds).
|
|
105
|
+
|
|
106
|
+
**Constraint**: The emulator does not enforce billing or quotas, so cost estimation must be done separately.
|
|
107
|
+
|
|
108
|
+
### DD-4: Checkpoint-Based Migration Resumption
|
|
109
|
+
|
|
110
|
+
**Decision**: Long-running migrations write a checkpoint document after each batch to `_migrations/{migrationId}`.
|
|
111
|
+
|
|
112
|
+
**Rationale**: A migration processing 100,000 documents takes 200 batch commits. If the script crashes at batch 150, without checkpoints it must restart from document 0 (re-reading 75,000 already-processed documents). With checkpoints, it reads the last checkpoint and resumes from document 75,001.
|
|
113
|
+
|
|
114
|
+
**Checkpoint document schema**:
|
|
115
|
+
```typescript
|
|
116
|
+
interface MigrationCheckpoint {
|
|
117
|
+
migrationId: string;
|
|
118
|
+
collection: string;
|
|
119
|
+
lastDocumentId: string; // cursor for startAfter()
|
|
120
|
+
processedCount: number;
|
|
121
|
+
skippedCount: number;
|
|
122
|
+
failedCount: number;
|
|
123
|
+
status: "in_progress" | "completed" | "failed";
|
|
124
|
+
startedAt: Timestamp;
|
|
125
|
+
updatedAt: Timestamp;
|
|
126
|
+
}
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
### DD-5: Distributed Counters for Hot Documents
|
|
130
|
+
|
|
131
|
+
**Decision**: Documents expected to receive > 1 write per second use a sharded counter pattern instead of `FieldValue.increment()` on a single document.
|
|
132
|
+
|
|
133
|
+
**Rationale**: Firestore supports a sustained write rate of 1 write per second per document. Higher rates cause `ABORTED` errors due to contention. Distributing writes across N shard documents and summing on read provides Nx throughput.
|
|
134
|
+
|
|
135
|
+
**When to use**: Page view counters, like counts, real-time vote tallies. Not needed for user profile updates or low-frequency writes.
|
|
136
|
+
|
|
137
|
+
### DD-6: Index-Aware Query Generation
|
|
138
|
+
|
|
139
|
+
**Decision**: When generating a query with multiple `where()` filters or `where()` + `orderBy()` on different fields, the skill must also produce the composite index definition.
|
|
140
|
+
|
|
141
|
+
**Rationale**: Firestore requires composite indexes for these queries. Without the index, the query fails at runtime with `FAILED_PRECONDITION`. Generating the index alongside the query prevents this failure mode entirely.
|
|
142
|
+
|
|
143
|
+
## Component Design
|
|
144
|
+
|
|
145
|
+
### Migration Engine
|
|
146
|
+
|
|
147
|
+
```
|
|
148
|
+
MigrationRunner
|
|
149
|
+
├── readCheckpoint(migrationId) → MigrationCheckpoint | null
|
|
150
|
+
├── runBatch(query, transform) → { processed, skipped, failed }
|
|
151
|
+
├── writeCheckpoint(checkpoint) → void
|
|
152
|
+
├── run(config) → MigrationResult
|
|
153
|
+
│ ├── Resume from checkpoint if exists
|
|
154
|
+
│ ├── Loop: query batch → transform → commit → checkpoint
|
|
155
|
+
│ └── Write final status
|
|
156
|
+
└── dryRun(config) → MigrationResult (reads only, no writes)
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
### Rules Generator
|
|
160
|
+
|
|
161
|
+
```
|
|
162
|
+
RulesGenerator
|
|
163
|
+
├── addCollection(name, accessPatterns)
|
|
164
|
+
├── addHelper(name, body)
|
|
165
|
+
├── addFieldValidation(collection, fieldRules)
|
|
166
|
+
├── generate() → string (firestore.rules content)
|
|
167
|
+
└── generateTests() → string (rules-unit-testing code)
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
### Index Manager
|
|
171
|
+
|
|
172
|
+
```
|
|
173
|
+
IndexManager
|
|
174
|
+
├── analyzeQuery(query) → IndexDefinition | null
|
|
175
|
+
├── readExistingIndexes(path) → IndexDefinition[]
|
|
176
|
+
├── mergeIndexes(existing, new) → IndexDefinition[]
|
|
177
|
+
└── writeIndexFile(path, indexes) → void
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
## Failure Modes and Recovery
|
|
181
|
+
|
|
182
|
+
| Failure | Detection | Recovery |
|
|
183
|
+
|---------|-----------|----------|
|
|
184
|
+
| PERMISSION_DENIED on write | Error code from Admin SDK | Check security rules; test with emulator; verify auth context |
|
|
185
|
+
| FAILED_PRECONDITION (missing index) | Error message contains index creation URL | Add index to `firestore.indexes.json`; deploy with `firebase deploy --only firestore:indexes` |
|
|
186
|
+
| ABORTED (write contention) | Transaction retry count > 0 | Admin SDK auto-retries 5 times; if persists, redesign to reduce writes to that document |
|
|
187
|
+
| Batch commit partially fails | Exception mid-batch (network, timeout) | Resume from last checkpoint; the failed batch is atomic (all or nothing) |
|
|
188
|
+
| DEADLINE_EXCEEDED on read | Query took > 60 seconds | Add indexes; reduce query scope with tighter filters; paginate |
|
|
189
|
+
| RESOURCE_EXHAUSTED | 429 from Firestore API | Back off; check if sustained write rate > 10k/sec database limit |
|
|
190
|
+
|
|
191
|
+
## Observability
|
|
192
|
+
|
|
193
|
+
### Migration Logging
|
|
194
|
+
|
|
195
|
+
Every batch commit logs:
|
|
196
|
+
```
|
|
197
|
+
[migration:backfill-status-field] Batch 150/200 committed. Processed: 75000, Skipped: 12, Failed: 0. Elapsed: 4m32s.
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
### Cost Estimation
|
|
201
|
+
|
|
202
|
+
Before executing large operations, estimate cost:
|
|
203
|
+
```
|
|
204
|
+
Operation: backfill 100,000 documents
|
|
205
|
+
Reads: 100,000 × $0.06/100k = $0.06
|
|
206
|
+
Writes: 100,000 × $0.18/100k = $0.18
|
|
207
|
+
Total estimated cost: $0.24
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
### Key Metrics
|
|
211
|
+
|
|
212
|
+
- Batch commit latency (p50/p95 per commit)
|
|
213
|
+
- Documents processed per minute
|
|
214
|
+
- Contention retry count (should be near zero)
|
|
215
|
+
- Security rules evaluation latency (visible in Firebase Console > Firestore > Rules)
|
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
# PRD: Firestore Operations Manager Skill
|
|
2
|
+
|
|
3
|
+
## Problem Statement
|
|
4
|
+
|
|
5
|
+
Firestore operations at scale require deep knowledge of batch write mechanics, composite index design, security rules authoring, and migration strategies. Developers routinely hit production issues -- missing composite indexes that crash queries, write contention on hot documents, security rules that silently block legitimate operations, and data migrations that lose documents or corrupt fields.
|
|
6
|
+
|
|
7
|
+
These problems share a root cause: Firestore's document model and operational constraints (500-write batch limit, 1-write-per-second-per-document sustained, mandatory composite indexes for multi-field queries) are well documented but poorly surfaced at development time. Errors appear only at runtime, often in production.
|
|
8
|
+
|
|
9
|
+
## Target Users
|
|
10
|
+
|
|
11
|
+
| Persona | Description | Key Need |
|
|
12
|
+
|---------|-------------|----------|
|
|
13
|
+
| Firebase developer | Building apps on Firestore, intermediate skill | Correct CRUD patterns, query optimization, index guidance |
|
|
14
|
+
| Backend engineer | Migrating data or building ingestion pipelines | Safe batch operations with checkpoints and rollback |
|
|
15
|
+
| Data team | Managing production Firestore data at scale | Migration strategies, schema evolution, cost control |
|
|
16
|
+
| Security-conscious developer | Writing or auditing Firestore security rules | Rules that pass emulator tests and enforce least privilege |
|
|
17
|
+
|
|
18
|
+
## Success Criteria
|
|
19
|
+
|
|
20
|
+
1. **Zero data loss on migrations**: Batch operations include checkpointing so that a failure at document 50,000 of 100,000 resumes from 50,000, not from 0.
|
|
21
|
+
2. **Correct indexes on first deploy**: Composite indexes are identified before deployment, not discovered via runtime FAILED_PRECONDITION errors.
|
|
22
|
+
3. **Rules pass emulator tests**: Security rules generated by this skill pass `@firebase/rules-unit-testing` validation against expected access patterns.
|
|
23
|
+
4. **Batch operations under limits**: All batch writes stay within the 500-operation limit per commit; all transactions complete within the 270-second server-side deadline.
|
|
24
|
+
5. **Paginated reads by default**: Queries returning potentially large result sets use cursor-based pagination, not unbounded `.get()`.
|
|
25
|
+
|
|
26
|
+
## Scope
|
|
27
|
+
|
|
28
|
+
### In Scope
|
|
29
|
+
|
|
30
|
+
- Document CRUD (create, read, update, delete) with proper error handling
|
|
31
|
+
- Batch writes (up to 500 operations per commit) with retry logic
|
|
32
|
+
- Transactions for multi-document atomic operations
|
|
33
|
+
- Composite index detection and `firestore.indexes.json` generation
|
|
34
|
+
- Security rules authoring with helper functions and field-level validation
|
|
35
|
+
- Emulator-first testing workflow for rules and queries
|
|
36
|
+
- Data migrations: backfill new fields, transform existing fields, move between collections
|
|
37
|
+
- Cursor-based pagination for large result sets
|
|
38
|
+
- Write contention mitigation (distributed counters, sharding)
|
|
39
|
+
- Cost estimation for read/write patterns
|
|
40
|
+
|
|
41
|
+
### Out of Scope
|
|
42
|
+
|
|
43
|
+
- Firestore-to-BigQuery export streaming (use Firestore Extensions)
|
|
44
|
+
- Real-time listener architecture design
|
|
45
|
+
- Firestore in Datastore mode
|
|
46
|
+
- Cross-database replication or multi-region configuration
|
|
47
|
+
- Vertex AI vector search in Firestore (covered by firebase-vertex-ai skill)
|
|
48
|
+
|
|
49
|
+
## Functional Requirements
|
|
50
|
+
|
|
51
|
+
### FR-1: Schema-Aware CRUD
|
|
52
|
+
The skill must detect the existing collection schema (by reading sample documents) before proposing writes. New fields added via `update()` must not overwrite existing data unless explicitly requested.
|
|
53
|
+
|
|
54
|
+
### FR-2: Batch Write Management
|
|
55
|
+
Batch writes must:
|
|
56
|
+
- Chunk operations into groups of 500 (Firestore limit per `batch.commit()`)
|
|
57
|
+
- Log progress every N documents (configurable, default 1,000)
|
|
58
|
+
- Write a checkpoint document after each successful batch so the operation can resume
|
|
59
|
+
- Retry failed batches with exponential backoff (3 attempts, 1s/2s/4s delays)
|
|
60
|
+
|
|
61
|
+
### FR-3: Composite Index Generation
|
|
62
|
+
When the skill generates a query with multiple `where()` clauses or a `where()` + `orderBy()` combination, it must also produce the corresponding composite index entry for `firestore.indexes.json`.
|
|
63
|
+
|
|
64
|
+
### FR-4: Security Rules
|
|
65
|
+
Generated rules must:
|
|
66
|
+
- Deny all access by default (no `match /{document=**} { allow read, write: if true }`)
|
|
67
|
+
- Use helper functions for authentication and ownership checks
|
|
68
|
+
- Include field-level validation for create and update operations
|
|
69
|
+
- Be testable with `@firebase/rules-unit-testing`
|
|
70
|
+
|
|
71
|
+
### FR-5: Migration Operations
|
|
72
|
+
Migrations must:
|
|
73
|
+
- Support backfill (add field to existing documents) and transform (modify field values)
|
|
74
|
+
- Write a checkpoint after each batch to a `_migrations/{migrationId}` document
|
|
75
|
+
- Support dry-run mode that reads but does not write
|
|
76
|
+
- Produce a summary (documents processed, skipped, failed)
|
|
77
|
+
|
|
78
|
+
### FR-6: Pagination
|
|
79
|
+
Queries that may return more than 100 documents must use `startAfter()` cursor-based pagination. The skill must produce both the query code and the pagination cursor management logic.
|
|
80
|
+
|
|
81
|
+
## Non-Functional Requirements
|
|
82
|
+
|
|
83
|
+
- **Throughput**: Batch operations must sustain 500 writes per second against Firestore without triggering rate limits (using staggered batch commits).
|
|
84
|
+
- **Idempotency**: Migrations must be idempotent -- running the same migration twice produces the same result without duplicating data.
|
|
85
|
+
- **Cost transparency**: For operations affecting > 1,000 documents, the skill must estimate Firestore read/write costs using current pricing ($0.06/100k reads, $0.18/100k writes).
|
|
86
|
+
- **Emulator compatibility**: All generated code must work against the Firestore emulator (`localhost:8080`) with no modification.
|
|
87
|
+
|
|
88
|
+
## Dependencies
|
|
89
|
+
|
|
90
|
+
| Dependency | Version | Purpose |
|
|
91
|
+
|------------|---------|---------|
|
|
92
|
+
| firebase-admin | >= 12.0 | Server-side Firestore access |
|
|
93
|
+
| @firebase/rules-unit-testing | >= 3.0 | Security rules testing |
|
|
94
|
+
| Firebase CLI | >= 13.0 | Emulator, rules deploy, index deploy |
|
|
95
|
+
| Node.js | >= 18 LTS | Runtime for Admin SDK scripts |
|
|
96
|
+
|
|
97
|
+
## Risks and Mitigations
|
|
98
|
+
|
|
99
|
+
| Risk | Impact | Mitigation |
|
|
100
|
+
|------|--------|------------|
|
|
101
|
+
| Hot document write contention | Writes to same doc > 1/sec cause ABORTED errors | Use distributed counters or sharded writes for high-throughput paths |
|
|
102
|
+
| Composite index build time | New indexes take minutes to hours for large collections | Deploy indexes before code that depends on them; use `firebase firestore:indexes` to check status |
|
|
103
|
+
| Security rules too restrictive | Legitimate operations blocked in production | Test all access patterns against emulator before deploy |
|
|
104
|
+
| Migration halfway failure | Partial data state if script crashes mid-batch | Checkpoint after each batch; resume from last checkpoint |
|
|
105
|
+
| Unbounded reads | Forgetting `.limit()` on a million-doc collection | Enforce pagination for any query without a known small result set |
|
|
106
|
+
| Stale reads in transactions | Reading outside transaction sees older data | Always read within `runTransaction()` when consistency matters |
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: firestore-operations-manager
|
|
3
|
+
description: |
|
|
4
|
+
Manage Firebase/Firestore operations including CRUD, queries, batch processing, and index/rule guidance.
|
|
5
|
+
Use when you need to create/update/query Firestore documents, run batch writes, troubleshoot missing indexes, or plan migrations.
|
|
6
|
+
Trigger with phrases like "firestore operations", "create firestore document", "batch write", "missing index", or "fix firestore query".
|
|
7
|
+
allowed-tools: Read, Write, Edit, Grep, Glob, Bash(cmd:*)
|
|
8
|
+
version: 1.0.0
|
|
9
|
+
author: Jeremy Longshore <jeremy@intentsolutions.io>
|
|
10
|
+
license: MIT
|
|
11
|
+
compatible-with: claude-code, codex, openclaw
|
|
12
|
+
tags: [community, migration, firestore-operations]
|
|
13
|
+
---
|
|
14
|
+
# Firestore Operations Manager
|
|
15
|
+
|
|
16
|
+
Operate Firestore safely in production: schema-aware CRUD, query/index tuning, batch processing, and guardrails for permissions and cost.
|
|
17
|
+
|
|
18
|
+
## Overview
|
|
19
|
+
|
|
20
|
+
Use this skill to design Firestore data access patterns and implement changes with the right indexes, security rules, and operational checks (emulator tests, monitoring, and rollback plans).
|
|
21
|
+
|
|
22
|
+
## Prerequisites
|
|
23
|
+
|
|
24
|
+
- A Firebase project with Firestore enabled (or a local emulator setup)
|
|
25
|
+
- A clear collection/document schema (or permission to propose one)
|
|
26
|
+
- Credentials for the target environment (service account / ADC) and a plan for secrets
|
|
27
|
+
|
|
28
|
+
## Instructions
|
|
29
|
+
|
|
30
|
+
1. Identify the operation: create/update/delete/query/batch/migration.
|
|
31
|
+
2. Confirm schema expectations and security rules constraints.
|
|
32
|
+
3. Implement the change (or propose a patch) using safe patterns:
|
|
33
|
+
- prefer batched writes/transactions where consistency matters
|
|
34
|
+
- add pagination for large queries
|
|
35
|
+
4. Check indexes:
|
|
36
|
+
- detect required composite indexes and provide `firestore.indexes.json` updates
|
|
37
|
+
5. Validate:
|
|
38
|
+
- run emulator tests or a minimal smoke query
|
|
39
|
+
- confirm cost/perf implications for the query pattern
|
|
40
|
+
|
|
41
|
+
## Output
|
|
42
|
+
|
|
43
|
+
- Code changes or snippets for the requested Firestore operation
|
|
44
|
+
- Index recommendations (and config updates when needed)
|
|
45
|
+
- A validation checklist (emulator commands and production smoke tests)
|
|
46
|
+
|
|
47
|
+
## Error Handling
|
|
48
|
+
|
|
49
|
+
- Permission denied: identify the rule/role blocking the operation and propose least-privilege changes.
|
|
50
|
+
- Missing index: provide the exact composite index needed for the query.
|
|
51
|
+
- Hotspot/latency issues: propose sharding, pagination, or query redesign.
|
|
52
|
+
|
|
53
|
+
## Examples
|
|
54
|
+
|
|
55
|
+
**Example: Fix a failing query**
|
|
56
|
+
- Request: “This query needs a composite index—what do I add?”
|
|
57
|
+
- Result: the exact index definition and a safer query pattern if needed.
|
|
58
|
+
|
|
59
|
+
**Example: Batch migration**
|
|
60
|
+
- Request: “Backfill a new field across 100k docs.”
|
|
61
|
+
- Result: batched write strategy, checkpoints, and rollback guidance.
|
|
62
|
+
|
|
63
|
+
## Resources
|
|
64
|
+
|
|
65
|
+
- Full detailed guide (kept for reference): `${CLAUDE_SKILL_DIR}/references/SKILL.full.md`
|
|
66
|
+
- Firestore docs: https://firebase.google.com/docs/firestore
|
|
67
|
+
- Firestore indexes: https://firebase.google.com/docs/firestore/query-data/indexing
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Firestore Operations Manager: Error Reference
|
|
2
|
+
|
|
3
|
+
## Security Rules Errors
|
|
4
|
+
|
|
5
|
+
| Error | Cause | Fix |
|
|
6
|
+
|-------|-------|-----|
|
|
7
|
+
| `PERMISSION_DENIED: Missing or insufficient permissions` | Security rules block the read or write | Check `firestore.rules` for the matching collection path; verify `request.auth` is not null and meets rule conditions |
|
|
8
|
+
| `PERMISSION_DENIED` on admin SDK writes | Admin SDK bypasses rules by default; this error means the service account lacks IAM roles | Grant `roles/datastore.user` to the service account in IAM |
|
|
9
|
+
| Rules pass in emulator but fail in production | Rules reference a document that does not exist in production (e.g., `get()` on missing user profile) | Ensure referenced documents exist before deploying; add null checks in rules |
|
|
10
|
+
| `request.resource.data.X` undefined | Write operation does not include field `X` that rules validate | Add the required field to the write payload, or adjust rules to use `request.resource.data.get('X', default)` |
|
|
11
|
+
|
|
12
|
+
## Composite Index Errors
|
|
13
|
+
|
|
14
|
+
| Error | Cause | Fix |
|
|
15
|
+
|-------|-------|-----|
|
|
16
|
+
| `FAILED_PRECONDITION: The query requires an index` | Query uses multiple `where()` clauses or `where()` + `orderBy()` on different fields without a composite index | Click the URL in the error message to auto-create, or add to `firestore.indexes.json` and deploy |
|
|
17
|
+
| Index build stuck at "Building" | Large collection, or conflicting index on same fields | Wait (can take hours for millions of docs); check Firebase Console > Firestore > Indexes for status |
|
|
18
|
+
| `INVALID_ARGUMENT: Too many composite indexes` | Exceeded 200 composite index limit per database | Remove unused indexes; consolidate queries to share indexes |
|
|
19
|
+
| Query works locally but fails in production | Emulator does not enforce index requirements | Always deploy indexes before deploying code that uses new queries |
|
|
20
|
+
|
|
21
|
+
## Write Contention Errors
|
|
22
|
+
|
|
23
|
+
| Error | Cause | Fix |
|
|
24
|
+
|-------|-------|-----|
|
|
25
|
+
| `ABORTED: Too much contention on these documents` | Multiple clients writing to the same document simultaneously (> 1 write/sec sustained) | Use distributed counters or sharded writes for high-frequency update paths |
|
|
26
|
+
| `ABORTED` inside `runTransaction()` | Transaction read-set modified by another write before commit | Admin SDK auto-retries up to 5 times; if still failing, reduce transaction scope or redesign data model |
|
|
27
|
+
| Transaction succeeds on retry but data looks wrong | Read outside transaction sees stale data; write based on stale read | Move all reads that inform writes inside `runTransaction()` |
|
|
28
|
+
|
|
29
|
+
## Batch Operation Errors
|
|
30
|
+
|
|
31
|
+
| Error | Cause | Fix |
|
|
32
|
+
|-------|-------|-----|
|
|
33
|
+
| `INVALID_ARGUMENT: maximum 500 writes allowed per request` | Batch contains > 500 operations | Chunk operations into groups of 500; commit each chunk separately |
|
|
34
|
+
| `DEADLINE_EXCEEDED` on `batch.commit()` | Batch took > 270 seconds server-side | Reduce batch size; check if individual document writes trigger expensive Cloud Functions |
|
|
35
|
+
| Partial failure on batch | Network error after server received but before client got response | Batch commits are atomic: either all 500 succeed or none do; safe to retry the entire batch |
|
|
36
|
+
| `NOT_FOUND: No document to update` inside batch | `batch.update()` called on a document that does not exist | Use `batch.set(ref, data, { merge: true })` instead, or verify document existence before batching |
|
|
37
|
+
|
|
38
|
+
## Transaction Errors
|
|
39
|
+
|
|
40
|
+
| Error | Cause | Fix |
|
|
41
|
+
|-------|-------|-----|
|
|
42
|
+
| `ABORTED: Transaction was aborted due to contention` | Concurrent writes to documents in the transaction's read set | Reduce documents read in transaction; Admin SDK retries automatically |
|
|
43
|
+
| `DEADLINE_EXCEEDED: Transaction has expired` | Transaction exceeded 270-second server-side limit | Break large transactions into smaller ones; avoid long-running async work inside transaction |
|
|
44
|
+
| `INVALID_ARGUMENT: Transaction has already been committed/rolled back` | Calling operations on a transaction object after it resolved | Ensure all operations happen before the transaction callback returns |
|
|
45
|
+
| Writes outside transaction not seeing transaction results | Reads after `runTransaction()` returns may hit cache | Use `{ source: 'server' }` for critical post-transaction reads on client SDK |
|
|
46
|
+
|
|
47
|
+
## Query Errors
|
|
48
|
+
|
|
49
|
+
| Error | Cause | Fix |
|
|
50
|
+
|-------|-------|-----|
|
|
51
|
+
| `INVALID_ARGUMENT: Cannot have inequality filters on multiple properties` | Query has `where('a', '>', x)` and `where('b', '<', y)` | Restructure query: inequality filter on one field only; use composite index for second field with `==` |
|
|
52
|
+
| `INVALID_ARGUMENT: Order by must match the first inequality field` | `orderBy('name')` when inequality filter is on `createdAt` | Add `orderBy('createdAt')` before any other `orderBy()` |
|
|
53
|
+
| Empty results when documents exist | Query field name has typo, or field stored as different type (string vs number) | Verify field name casing; check document in Firebase Console; Firestore is case-sensitive |
|
|
54
|
+
| `RESOURCE_EXHAUSTED: Quota exceeded` | Exceeded read quota (50k reads/minute on free tier) | Upgrade to Blaze plan; optimize queries with tighter filters and limits |
|
|
55
|
+
|
|
56
|
+
## Emulator Errors
|
|
57
|
+
|
|
58
|
+
| Error | Cause | Fix |
|
|
59
|
+
|-------|-------|-----|
|
|
60
|
+
| `EADDRINUSE: address already in use :::8080` | Another process (or previous emulator) using port 8080 | Kill the process: `lsof -ti:8080 \| xargs kill`; or change port in `firebase.json` |
|
|
61
|
+
| `Error: Could not start Firestore Emulator, port taken` | Same as above but reported by Firebase CLI | Stop other emulator instances; check for zombie Java processes |
|
|
62
|
+
| Emulator data disappears between restarts | Emulator does not persist by default | Use `--export-on-exit=./emulator-data` and `--import=./emulator-data` flags |
|
|
63
|
+
| Rules changes not reflected in emulator | Emulator loaded rules at startup | Restart emulator after rules changes; or use hot-reload (CLI v13+) |
|
|
64
|
+
| `connect ECONNREFUSED 127.0.0.1:8080` | Emulator not running or wrong port | Start emulator: `firebase emulators:start --only firestore`; verify port matches code |
|
|
65
|
+
|
|
66
|
+
## Data Migration Errors
|
|
67
|
+
|
|
68
|
+
| Error | Cause | Fix |
|
|
69
|
+
|-------|-------|-----|
|
|
70
|
+
| Migration script re-processes already-migrated documents | No checkpoint mechanism; script restarted from beginning | Implement checkpoint pattern: write `_migrations/{id}` doc after each batch with `lastDocumentId` |
|
|
71
|
+
| Documents have mixed old/new schema | Migration interrupted midway | Resume from checkpoint; run validation query to find documents missing the new field |
|
|
72
|
+
| `INVALID_ARGUMENT: Value for argument "data" is not a valid Firestore document` | Transform function returned undefined or invalid type | Validate transform output before writing; skip documents that produce invalid results |
|
|
73
|
+
| Migration too slow (hours for 100k docs) | Processing documents one at a time instead of in batches | Use batch writes (500 per commit); parallelize with multiple query cursors on sharded key |
|
|
74
|
+
|
|
75
|
+
## Cost and Billing Errors
|
|
76
|
+
|
|
77
|
+
| Error | Cause | Fix |
|
|
78
|
+
|-------|-------|-----|
|
|
79
|
+
| Unexpected high Firestore bill | Runaway query without `.limit()`; or real-time listener on large collection | Add `.limit()` to all queries; audit listeners; check Firebase Console > Usage |
|
|
80
|
+
| `RESOURCE_EXHAUSTED` on free tier | Exceeded 50k reads/day or 20k writes/day (Spark plan) | Upgrade to Blaze plan; optimize query patterns; cache frequently read documents |
|
|
81
|
+
| Reads cost more than expected | `get()` on a collection with 10,000 docs counts as 10,000 reads | Always use `.where()` and `.limit()` to narrow results; paginate large reads |
|
|
82
|
+
| Deletes cost more than expected | Deleting a document with subcollections does not delete subcollections | Recursively delete subcollections first; use `firebase firestore:delete --recursive` for CLI deletion |
|
|
83
|
+
|
|
84
|
+
---
|
|
85
|
+
*[Tons of Skills](https://tonsofskills.com) by [Intent Solutions](https://intentsolutions.io) | [jeremylongshore.com](https://jeremylongshore.com)*
|