presudo 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/__init__.py +0 -0
- package/index.js +29 -0
- package/main.py +2 -0
- package/package.json +18 -0
- package/pods/DSA/all.txt +914 -0
- package/pods/DSA/lasttime.txt +66 -0
- package/pods/DSA/tg.txt +99 -0
- package/pods/backend/inventory_management_system.txt +71 -0
- package/pods/backend/simple_inventory_management_system_for_bookstore_using_tuples.txt +59 -0
- package/pods/db/mongo.txt +842 -0
- package/pods/db/mongo_MCQ.txt +216 -0
- package/pods/db/sql.txt +728 -0
- package/pods/react/book_list.txt +51 -0
- package/pods/sudo/Product Management System-ts.txt +274 -0
- package/pods/sudo/backend/managing a delivery service.txt +130 -0
- package/pods/sudo/backend/online shopping cart system.txt +200 -0
- package/pods/sudo/backend/restaurant ordering system.txt +104 -0
- package/pods/sudo/das.txt +635 -0
- package/pods/sudo/react/TravelMap Application.txt +212 -0
- package/pods/sudo/sq.txt +217 -0
- package/pods/ts/Product Management System.txt +135 -0
- package/pods/ts/RetailStore.txt +113 -0
- package/pods/ts/Transaction Entity.txt +41 -0
- package/pods/ts/books_in_library.txt +39 -0
- package/pods/ts/find avg marks.txt +7 -0
- package/pods/ts/managing product inventory.txt +12 -0
- package/pods/ts/ts_examples/examples.txt +341 -0
- package/test.js +3 -0
|
@@ -0,0 +1,842 @@
|
|
|
1
|
+
MongoDB: Comprehensive Notes for Beginners 📚
|
|
2
|
+
1. What is MongoDB?
|
|
3
|
+
|
|
4
|
+
MongoDB is a document-oriented, NoSQL database that stores data in a flexible, JSON-like format called BSON.
|
|
5
|
+
It is designed to scale horizontally across many servers and supports high availability.
|
|
6
|
+
|
|
7
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
8
|
+
2. Advantages of MongoDB
|
|
9
|
+
|
|
10
|
+
Scalability: Horizontally scalable through sharding.
|
|
11
|
+
|
|
12
|
+
Flexibility: Schema-less design; easy to change data structures.
|
|
13
|
+
|
|
14
|
+
High Availability: Achieved through replica sets.
|
|
15
|
+
|
|
16
|
+
Rich Query Language: MongoDB offers powerful querying and aggregation capabilities.
|
|
17
|
+
|
|
18
|
+
High Performance: Fast writes and reads with appropriate indexing.
|
|
19
|
+
|
|
20
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
21
|
+
|
|
22
|
+
3. MongoDB Architecture
|
|
23
|
+
|
|
24
|
+
MongoDB is based on a client-server architecture:
|
|
25
|
+
|
|
26
|
+
Client: MongoDB applications or tools like mongo shell or Compass.
|
|
27
|
+
|
|
28
|
+
Server: MongoDB instance (where data is stored).
|
|
29
|
+
|
|
30
|
+
Database: A collection of data.
|
|
31
|
+
|
|
32
|
+
Collection: A grouping of documents, similar to a table in relational databases.
|
|
33
|
+
|
|
34
|
+
Document: A record in a collection, stored in BSON format (Binary JSON).
|
|
35
|
+
|
|
36
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
37
|
+
1. Auto-Sharding in NoSQL Databases
|
|
38
|
+
|
|
39
|
+
Auto-sharding is a technique used in NoSQL databases to distribute large datasets across multiple servers or nodes.
|
|
40
|
+
The database automatically splits the data and allocates it to different servers to balance the load. This allows the database to scale horizontally by adding more servers as data grows.
|
|
41
|
+
|
|
42
|
+
Example in MongoDB: MongoDB uses shards to store data across multiple machines.
|
|
43
|
+
A shard key determines how data is distributed across shards.
|
|
44
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
45
|
+
|
|
46
|
+
2. Loading the Data and Query Automatically Across Servers
|
|
47
|
+
|
|
48
|
+
In NoSQL databases with sharding (like MongoDB), when data is loaded or queried,
|
|
49
|
+
the system automatically distributes the data to the appropriate shard and routes the query to the correct server.
|
|
50
|
+
This automatic distribution ensures that the database can scale horizontally and handle large amounts of data.
|
|
51
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
52
|
+
|
|
53
|
+
3. Sharing of Complex Data
|
|
54
|
+
|
|
55
|
+
Complex data refers to large datasets that may contain nested structures, arrays,
|
|
56
|
+
or other complex types. NoSQL databases are designed to store and efficiently manage these complex data types.
|
|
57
|
+
Sharing complex data across distributed systems requires strategies like:
|
|
58
|
+
|
|
59
|
+
Sharding: Data is split across multiple nodes based on a shard key.
|
|
60
|
+
|
|
61
|
+
Replication: Copies of the data are maintained on multiple servers for high availability.
|
|
62
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
63
|
+
|
|
64
|
+
4. Dependency of CAP Model on BASE Theorem
|
|
65
|
+
|
|
66
|
+
The CAP Theorem states that in a distributed system, you can only achieve two
|
|
67
|
+
out of three properties: Consistency, Availability, and Partition tolerance.
|
|
68
|
+
NoSQL databases generally focus on Availability and Partition tolerance (i.e., the BASE model).
|
|
69
|
+
|
|
70
|
+
BASE (Basically Available, Soft state, Eventually consistent)
|
|
71
|
+
is a more relaxed consistency model compared to ACID (Atomicity, Consistency, Isolation, Durability).
|
|
72
|
+
|
|
73
|
+
Basically Available: The system guarantees availability.
|
|
74
|
+
|
|
75
|
+
Soft State: The state of the system can change over time.
|
|
76
|
+
|
|
77
|
+
Eventually Consistent: The system will become consistent at some point.
|
|
78
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
79
|
+
|
|
80
|
+
5. Updating a Document
|
|
81
|
+
|
|
82
|
+
In NoSQL databases like MongoDB, updating a document is straightforward.
|
|
83
|
+
MongoDB uses an update operation to modify fields in an existing document.
|
|
84
|
+
|
|
85
|
+
Example:
|
|
86
|
+
|
|
87
|
+
db.collection.updateOne({ _id: 1 }, { $set: { status: "Active" } });
|
|
88
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
89
|
+
|
|
90
|
+
6. Understanding MongoDB Queries
|
|
91
|
+
|
|
92
|
+
MongoDB provides a rich query language that allows you to filter, sort, and project data.
|
|
93
|
+
Key components of MongoDB queries include:
|
|
94
|
+
|
|
95
|
+
find(): Retrieves documents based on a query.
|
|
96
|
+
|
|
97
|
+
sort(): Sorts the result set.
|
|
98
|
+
|
|
99
|
+
limit(): Limits the number of documents returned.
|
|
100
|
+
|
|
101
|
+
Example:
|
|
102
|
+
|
|
103
|
+
db.users.find({ age: { $gt: 20 } }).sort({ name: 1 }).limit(10);
|
|
104
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
105
|
+
|
|
106
|
+
7. Working on a Distributed Transaction
|
|
107
|
+
|
|
108
|
+
A distributed transaction involves multiple operations across different nodes or services.
|
|
109
|
+
NoSQL databases like MongoDB support multi-document ACID transactions to ensure that operations across multiple documents or collections are atomic, consistent, isolated, and durable.
|
|
110
|
+
|
|
111
|
+
Example: You start a transaction with startSession(), and if all operations succeed,
|
|
112
|
+
you commit the transaction; otherwise, you abort it.
|
|
113
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
114
|
+
|
|
115
|
+
8. Selecting Documents from a Collection
|
|
116
|
+
|
|
117
|
+
In MongoDB, selecting documents is done using the find() function.
|
|
118
|
+
You can specify filters, projections, sorting, and limiting to retrieve the data you need.
|
|
119
|
+
|
|
120
|
+
Example:
|
|
121
|
+
|
|
122
|
+
db.collection.find({ age: { $gt: 25 } }).sort({ name: 1 }).limit(5);
|
|
123
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
124
|
+
|
|
125
|
+
9. Deploying Replica Sets
|
|
126
|
+
|
|
127
|
+
A replica set in MongoDB is a group of MongoDB instances that maintain the same data set.
|
|
128
|
+
Replica sets provide high availability and data redundancy.
|
|
129
|
+
|
|
130
|
+
Steps to deploy a replica set:
|
|
131
|
+
|
|
132
|
+
Configure multiple MongoDB instances to form the replica set.
|
|
133
|
+
|
|
134
|
+
Elect a primary node and set up secondary nodes for replication.
|
|
135
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
136
|
+
|
|
137
|
+
10. Using the Master-Slave Replication Method
|
|
138
|
+
|
|
139
|
+
In master-slave replication, one node (master) accepts write operations, and the other nodes (slaves)
|
|
140
|
+
replicate data from the master. This method is generally considered outdated, and MongoDB now uses replica sets instead,
|
|
141
|
+
which provide more redundancy and failover capabilities.
|
|
142
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
143
|
+
|
|
144
|
+
11. Data Replication in NoSQL
|
|
145
|
+
|
|
146
|
+
Data replication involves copying data from one node to another for high availability and fault tolerance.
|
|
147
|
+
In NoSQL databases, replication can be done in several ways:
|
|
148
|
+
|
|
149
|
+
Master-slave replication: Data is replicated from a master node to multiple slave nodes.
|
|
150
|
+
|
|
151
|
+
Peer-to-peer replication: All nodes are equal, and data is replicated between all nodes.
|
|
152
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
153
|
+
|
|
154
|
+
12. Defining Nested Key Named Type
|
|
155
|
+
|
|
156
|
+
In NoSQL databases, particularly document-based databases like MongoDB,
|
|
157
|
+
you can define nested fields within documents. These fields can have their own structure,
|
|
158
|
+
which is different from flat relational data models.
|
|
159
|
+
|
|
160
|
+
Example:
|
|
161
|
+
|
|
162
|
+
{ _id: 1, name: "John", address: { street: "123 Main St", city: "New York" } }
|
|
163
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
164
|
+
|
|
165
|
+
13. Debugging a Piece of Code
|
|
166
|
+
|
|
167
|
+
Debugging in NoSQL systems involves identifying issues with queries, data modeling,
|
|
168
|
+
or network operations. Tools like MongoDB Compass, logs, and profiler can help debug and optimize queries.
|
|
169
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
170
|
+
|
|
171
|
+
14. Using the Replication Scheme
|
|
172
|
+
|
|
173
|
+
Replication schemes in NoSQL databases, especially in MongoDB, ensure that data
|
|
174
|
+
is consistently copied to multiple servers for fault tolerance and high availability.
|
|
175
|
+
The replication setup can include primary and secondary nodes, where the primary node handles all write operations,
|
|
176
|
+
and secondary nodes replicate data.
|
|
177
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
178
|
+
|
|
179
|
+
15. Performing Complex Transactions which Require Multiple Operations
|
|
180
|
+
|
|
181
|
+
NoSQL databases like MongoDB support multi-document transactions,
|
|
182
|
+
which ensure that a group of operations across multiple documents or collections is executed atomically.
|
|
183
|
+
|
|
184
|
+
Example:
|
|
185
|
+
|
|
186
|
+
const session = client.startSession();
|
|
187
|
+
session.startTransaction();
|
|
188
|
+
try {
|
|
189
|
+
db.collection.updateOne({ _id: 1 }, { $set: { status: "Active" } }, { session });
|
|
190
|
+
db.collection.updateOne({ _id: 2 }, { $set: { status: "Inactive" } }, { session });
|
|
191
|
+
session.commitTransaction();
|
|
192
|
+
} catch (error) {
|
|
193
|
+
session.abortTransaction();
|
|
194
|
+
} finally {
|
|
195
|
+
session.endSession();
|
|
196
|
+
}
|
|
197
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
198
|
+
|
|
199
|
+
16. NoSQL Storage Partitions
|
|
200
|
+
|
|
201
|
+
Storage partitions in NoSQL databases refer to splitting data across
|
|
202
|
+
different servers (partitions or shards). This partitioning allows the database to scale horizontally,
|
|
203
|
+
ensuring it can handle larger datasets effectively.
|
|
204
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
205
|
+
|
|
206
|
+
17. NoSQL Scenarios
|
|
207
|
+
|
|
208
|
+
NoSQL databases are particularly suitable for the following scenarios:
|
|
209
|
+
|
|
210
|
+
Handling unstructured or semi-structured data.
|
|
211
|
+
|
|
212
|
+
Managing large volumes of data that need to be scaled horizontally.
|
|
213
|
+
|
|
214
|
+
Applications that require low latency and high availability.
|
|
215
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
216
|
+
|
|
217
|
+
18. NoSQL Operations
|
|
218
|
+
|
|
219
|
+
NoSQL databases like MongoDB offer basic operations such as:
|
|
220
|
+
|
|
221
|
+
Create: Inserting documents into collections.
|
|
222
|
+
|
|
223
|
+
Read: Querying documents.
|
|
224
|
+
|
|
225
|
+
Update: Modifying documents.
|
|
226
|
+
|
|
227
|
+
Delete: Removing documents.
|
|
228
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
229
|
+
|
|
230
|
+
19. XML Retrieval Tools
|
|
231
|
+
|
|
232
|
+
NoSQL databases, particularly those that support document-based models (like MongoDB),
|
|
233
|
+
allow you to store and retrieve data in various formats, including XML. XPath and XQuery
|
|
234
|
+
are tools commonly used for querying XML data.
|
|
235
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
236
|
+
|
|
237
|
+
20. NoSQL Data Model Groupings
|
|
238
|
+
|
|
239
|
+
NoSQL data models are often categorized into:
|
|
240
|
+
|
|
241
|
+
Document store: (e.g., MongoDB) stores data as documents.
|
|
242
|
+
|
|
243
|
+
Key-value store: (e.g., Redis) stores data as key-value pairs.
|
|
244
|
+
|
|
245
|
+
Column-family store: (e.g., Cassandra) stores data in columns.
|
|
246
|
+
|
|
247
|
+
Graph database: (e.g., Neo4j) stores data as nodes and edges.
|
|
248
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
249
|
+
|
|
250
|
+
21. Reasons for NoSQL Popularity
|
|
251
|
+
|
|
252
|
+
NoSQL databases have become popular for reasons such as:
|
|
253
|
+
|
|
254
|
+
Scalability: They scale horizontally, making them suitable for large-scale applications.
|
|
255
|
+
|
|
256
|
+
Flexibility: Schema-less design allows easy changes to data structures.
|
|
257
|
+
|
|
258
|
+
Performance: NoSQL databases offer high performance for certain types of workloads.
|
|
259
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
260
|
+
|
|
261
|
+
22. Graph Store Limitations
|
|
262
|
+
|
|
263
|
+
Graph databases are designed for handling highly interconnected data. However, they may have limitations in terms of:
|
|
264
|
+
|
|
265
|
+
Scalability: Graph databases may not scale as well horizontally as other NoSQL databases.
|
|
266
|
+
|
|
267
|
+
Complexity: Querying complex graphs can be challenging.
|
|
268
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
269
|
+
|
|
270
|
+
23. MapR-DB Compatibility
|
|
271
|
+
|
|
272
|
+
MapR-DB is a NoSQL database that provides support for both HBase and Cassandra APIs. It is optimized for performance and is typically used in big data and analytics workloads. MapR-DB is compatible with Hadoop and other big data technologies.
|
|
273
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
274
|
+
|
|
275
|
+
24. NoSQL Advantages
|
|
276
|
+
|
|
277
|
+
Scalability: NoSQL databases can scale horizontally, making them ideal for large-scale applications.
|
|
278
|
+
|
|
279
|
+
Performance: They are optimized for fast reads and writes.
|
|
280
|
+
|
|
281
|
+
Flexibility: NoSQL allows schema-less and semi-structured data storage.
|
|
282
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
283
|
+
|
|
284
|
+
25. Configurations for Availability
|
|
285
|
+
|
|
286
|
+
To ensure high availability in NoSQL databases, configurations such as replica sets and sharding are used.
|
|
287
|
+
Data is replicated across multiple nodes to ensure availability even if some nodes go down.
|
|
288
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
289
|
+
|
|
290
|
+
26. NoSQL Benefits
|
|
291
|
+
|
|
292
|
+
Distributed Architecture: Facilitates scalability and high availability.
|
|
293
|
+
|
|
294
|
+
Flexible Data Models: Handle structured, semi-structured, and unstructured data.
|
|
295
|
+
|
|
296
|
+
Fast Writes: High throughput for writes.
|
|
297
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
298
|
+
|
|
299
|
+
27. First Step in NoSQL Adoption
|
|
300
|
+
|
|
301
|
+
The first step in adopting a NoSQL database is understanding the data model and selecting the appropriate NoSQL
|
|
302
|
+
database (document, key-value, graph, etc.) based on your application's requirements.
|
|
303
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
304
|
+
|
|
305
|
+
28. NoSQL Data Modeling Techniques
|
|
306
|
+
|
|
307
|
+
NoSQL databases offer different data modeling techniques based on the use case. Some common patterns are:
|
|
308
|
+
|
|
309
|
+
Embedding: Store related data within the same document (good for one-to-one relationships).
|
|
310
|
+
|
|
311
|
+
Referencing: Store related data in separate documents (good for one-to-many or many-to-many relationships).
|
|
312
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
313
|
+
|
|
314
|
+
29. Column Store Databases
|
|
315
|
+
|
|
316
|
+
Column-store databases (e.g., Cassandra, HBase) store data by column rather than by row.
|
|
317
|
+
This approach is particularly beneficial for analytical queries where you often need to read only a few columns from a large dataset.
|
|
318
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
319
|
+
|
|
320
|
+
30. Distributed Transaction Cursors
|
|
321
|
+
|
|
322
|
+
In distributed databases, a transaction cursor allows you to work with large datasets
|
|
323
|
+
in chunks, ensuring that the transaction is properly handled even when data is spread across multiple servers.
|
|
324
|
+
|
|
325
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
326
|
+
|
|
327
|
+
4. MongoDB Data Types
|
|
328
|
+
|
|
329
|
+
MongoDB uses BSON (Binary JSON), which supports the following data types:
|
|
330
|
+
|
|
331
|
+
String: Textual data.
|
|
332
|
+
|
|
333
|
+
Integer: Whole numbers.
|
|
334
|
+
|
|
335
|
+
Boolean: true or false.
|
|
336
|
+
|
|
337
|
+
Double: Decimal values.
|
|
338
|
+
|
|
339
|
+
Array: Ordered list of values.
|
|
340
|
+
|
|
341
|
+
Object: A document within another document.
|
|
342
|
+
|
|
343
|
+
Date: Date and time.
|
|
344
|
+
|
|
345
|
+
Null: Empty value.
|
|
346
|
+
|
|
347
|
+
ObjectId: Unique identifier for documents.
|
|
348
|
+
|
|
349
|
+
Binary Data: For storing binary information.
|
|
350
|
+
|
|
351
|
+
Regular Expression: For storing patterns.
|
|
352
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
353
|
+
|
|
354
|
+
5. CRUD Operations (Create, Read, Update, Delete)
|
|
355
|
+
|
|
356
|
+
Create: Add new documents.
|
|
357
|
+
|
|
358
|
+
insertOne(): Inserts a single document.
|
|
359
|
+
|
|
360
|
+
insertMany(): Inserts multiple documents.
|
|
361
|
+
|
|
362
|
+
Read: Query documents.
|
|
363
|
+
|
|
364
|
+
find(): Retrieves documents that match specified criteria.
|
|
365
|
+
|
|
366
|
+
findOne(): Retrieves a single document.
|
|
367
|
+
|
|
368
|
+
countDocuments(): Counts the number of documents.
|
|
369
|
+
|
|
370
|
+
Update: Modify existing documents.
|
|
371
|
+
|
|
372
|
+
updateOne(): Updates one document.
|
|
373
|
+
|
|
374
|
+
updateMany(): Updates multiple documents.
|
|
375
|
+
|
|
376
|
+
replaceOne(): Replaces a document.
|
|
377
|
+
|
|
378
|
+
Delete: Remove documents.
|
|
379
|
+
|
|
380
|
+
deleteOne(): Deletes one document.
|
|
381
|
+
|
|
382
|
+
deleteMany(): Deletes multiple documents.
|
|
383
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
384
|
+
|
|
385
|
+
6. Indexes in MongoDB
|
|
386
|
+
|
|
387
|
+
Indexes improve query performance by allowing fast searching.
|
|
388
|
+
|
|
389
|
+
Creating an Index:
|
|
390
|
+
|
|
391
|
+
db.collection.createIndex({ field: 1 }); // Ascending index
|
|
392
|
+
|
|
393
|
+
|
|
394
|
+
List Indexes:
|
|
395
|
+
|
|
396
|
+
db.collection.getIndexes();
|
|
397
|
+
|
|
398
|
+
|
|
399
|
+
Drop an Index:
|
|
400
|
+
|
|
401
|
+
db.collection.dropIndex("index_name");
|
|
402
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
403
|
+
|
|
404
|
+
7. Aggregation Framework
|
|
405
|
+
|
|
406
|
+
Aggregation allows you to process data records and return computed results:
|
|
407
|
+
|
|
408
|
+
$match: Filters documents.
|
|
409
|
+
|
|
410
|
+
$group: Groups documents by field.
|
|
411
|
+
|
|
412
|
+
$sort: Sorts the results.
|
|
413
|
+
|
|
414
|
+
$project: Modifies documents.
|
|
415
|
+
|
|
416
|
+
$limit: Limits the number of documents.
|
|
417
|
+
|
|
418
|
+
$skip: Skips a number of documents.
|
|
419
|
+
|
|
420
|
+
$unwind: Deconstructs an array.
|
|
421
|
+
|
|
422
|
+
Example aggregation pipeline:
|
|
423
|
+
|
|
424
|
+
db.collection.aggregate([
|
|
425
|
+
{ $match: { age: { $gt: 20 } } },
|
|
426
|
+
{ $group: { _id: "$age", count: { $sum: 1 } } }
|
|
427
|
+
]);
|
|
428
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
429
|
+
|
|
430
|
+
8. Sharding in MongoDB
|
|
431
|
+
|
|
432
|
+
Sharding distributes data across multiple servers for horizontal scaling.
|
|
433
|
+
|
|
434
|
+
Shard Key: Field used to distribute data.
|
|
435
|
+
|
|
436
|
+
Shards: MongoDB instances where data is stored.
|
|
437
|
+
|
|
438
|
+
Mongos: Query router responsible for routing client requests to the correct shard.
|
|
439
|
+
|
|
440
|
+
Sharding is best suited for large datasets and high-traffic applications.
|
|
441
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
442
|
+
|
|
443
|
+
9. Replica Sets in MongoDB
|
|
444
|
+
|
|
445
|
+
A replica set is a group of MongoDB servers that maintain the same data set.
|
|
446
|
+
|
|
447
|
+
Primary: The main server handling reads and writes.
|
|
448
|
+
|
|
449
|
+
Secondary: Copies of the primary that replicate data.
|
|
450
|
+
|
|
451
|
+
Arbiter: A member that doesn't store data but helps with elections.
|
|
452
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
453
|
+
|
|
454
|
+
10. MongoDB Users and Roles
|
|
455
|
+
|
|
456
|
+
MongoDB uses role-based access control (RBAC) to define permissions.
|
|
457
|
+
|
|
458
|
+
Create User:
|
|
459
|
+
|
|
460
|
+
db.createUser({
|
|
461
|
+
user: "username",
|
|
462
|
+
pwd: "password",
|
|
463
|
+
roles: [{ role: "readWrite", db: "mydb" }]
|
|
464
|
+
});
|
|
465
|
+
|
|
466
|
+
|
|
467
|
+
List Users:
|
|
468
|
+
|
|
469
|
+
db.getUsers();
|
|
470
|
+
|
|
471
|
+
|
|
472
|
+
Drop User:
|
|
473
|
+
|
|
474
|
+
db.dropUser("username");
|
|
475
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
476
|
+
|
|
477
|
+
11. Backup and Restore
|
|
478
|
+
|
|
479
|
+
Backup (mongodump):
|
|
480
|
+
|
|
481
|
+
mongodump --uri="mongodb://localhost:27017" --out=/backup/dir
|
|
482
|
+
|
|
483
|
+
|
|
484
|
+
Restore (mongorestore):
|
|
485
|
+
|
|
486
|
+
mongorestore /backup/dir
|
|
487
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
488
|
+
|
|
489
|
+
12. MongoDB with Python (PyMongo)
|
|
490
|
+
|
|
491
|
+
PyMongo is the official MongoDB driver for Python.
|
|
492
|
+
|
|
493
|
+
Install PyMongo:
|
|
494
|
+
|
|
495
|
+
pip install pymongo
|
|
496
|
+
|
|
497
|
+
|
|
498
|
+
Connect to MongoDB:
|
|
499
|
+
|
|
500
|
+
from pymongo import MongoClient
|
|
501
|
+
client = MongoClient("mongodb://localhost:27017/")
|
|
502
|
+
db = client.mydb
|
|
503
|
+
|
|
504
|
+
|
|
505
|
+
Insert Data:
|
|
506
|
+
|
|
507
|
+
db.collection.insert_one({"name": "Alice", "age": 30})
|
|
508
|
+
|
|
509
|
+
|
|
510
|
+
Find Data:
|
|
511
|
+
|
|
512
|
+
person = db.collection.find_one({"name": "Alice"})
|
|
513
|
+
print(person)
|
|
514
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
515
|
+
|
|
516
|
+
13. MongoDB Query Operators
|
|
517
|
+
|
|
518
|
+
$eq: Matches exact value.
|
|
519
|
+
|
|
520
|
+
db.collection.find({ age: { $eq: 30 } });
|
|
521
|
+
|
|
522
|
+
|
|
523
|
+
$gt, $gte: Greater than, greater than or equal to.
|
|
524
|
+
|
|
525
|
+
db.collection.find({ age: { $gt: 20 } });
|
|
526
|
+
|
|
527
|
+
|
|
528
|
+
$lt, $lte: Less than, less than or equal to.
|
|
529
|
+
|
|
530
|
+
$in: Matches any value in a list.
|
|
531
|
+
|
|
532
|
+
db.collection.find({ age: { $in: [20, 25, 30] } });
|
|
533
|
+
|
|
534
|
+
|
|
535
|
+
$regex: Regular expression matching.
|
|
536
|
+
|
|
537
|
+
db.collection.find({ name: { $regex: "^A" } });
|
|
538
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
539
|
+
|
|
540
|
+
14. Common MongoDB Administrative Commands
|
|
541
|
+
|
|
542
|
+
Show databases:
|
|
543
|
+
|
|
544
|
+
show dbs;
|
|
545
|
+
|
|
546
|
+
|
|
547
|
+
Switch database:
|
|
548
|
+
|
|
549
|
+
use mydb;
|
|
550
|
+
|
|
551
|
+
|
|
552
|
+
Show collections:
|
|
553
|
+
|
|
554
|
+
show collections;
|
|
555
|
+
|
|
556
|
+
|
|
557
|
+
Drop collection:
|
|
558
|
+
|
|
559
|
+
db.collection.drop();
|
|
560
|
+
|
|
561
|
+
|
|
562
|
+
Check server status:
|
|
563
|
+
|
|
564
|
+
db.serverStatus();
|
|
565
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
566
|
+
|
|
567
|
+
15. MongoDB Aggregation Operators
|
|
568
|
+
|
|
569
|
+
$sum: Sums values.
|
|
570
|
+
|
|
571
|
+
db.collection.aggregate([{ $group: { _id: null, total: { $sum: "$age" } } }]);
|
|
572
|
+
|
|
573
|
+
|
|
574
|
+
$avg: Averages values.
|
|
575
|
+
|
|
576
|
+
$min, $max: Minimum and maximum values.
|
|
577
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
578
|
+
|
|
579
|
+
16. Data Modeling Strategies in MongoDB
|
|
580
|
+
|
|
581
|
+
Embedded Model: Storing related data in a single document (1-to-1 relationships).
|
|
582
|
+
|
|
583
|
+
Reference Model: Storing related data in separate documents (1-to-many or many-to-many relationships).
|
|
584
|
+
|
|
585
|
+
Example of embedded model:
|
|
586
|
+
|
|
587
|
+
{ _id: 1, name: "John", address: { city: "NY", zip: "10001" } }
|
|
588
|
+
|
|
589
|
+
|
|
590
|
+
Example of reference model:
|
|
591
|
+
|
|
592
|
+
{ _id: 1, name: "John" }
|
|
593
|
+
{ _id: 1, user_id: 1, post: "Hello MongoDB!" }
|
|
594
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
595
|
+
|
|
596
|
+
17. MongoDB Data Validation
|
|
597
|
+
|
|
598
|
+
Schema Validation: MongoDB allows schema validation on collections.
|
|
599
|
+
|
|
600
|
+
db.createCollection("users", {
|
|
601
|
+
validator: {
|
|
602
|
+
$jsonSchema: {
|
|
603
|
+
bsonType: "object",
|
|
604
|
+
required: ["name", "age"],
|
|
605
|
+
properties: {
|
|
606
|
+
name: { bsonType: "string" },
|
|
607
|
+
age: { bsonType: "int" }
|
|
608
|
+
}
|
|
609
|
+
}
|
|
610
|
+
}
|
|
611
|
+
});
|
|
612
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
613
|
+
|
|
614
|
+
18. MongoDB Best Practices
|
|
615
|
+
|
|
616
|
+
Use indexes to improve query performance.
|
|
617
|
+
|
|
618
|
+
Avoid storing large files in MongoDB. Use GridFS for large files.
|
|
619
|
+
|
|
620
|
+
Limit document size to avoid performance bottlenecks.
|
|
621
|
+
|
|
622
|
+
Use findOne and find efficiently to limit results.
|
|
623
|
+
|
|
624
|
+
Keep data duplication low: Use references or embedded models appropriately.
|
|
625
|
+
|
|
626
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
627
|
+
|
|
628
|
+
Extra MongoDB Topics
|
|
629
|
+
1. MongoDB GridFS
|
|
630
|
+
|
|
631
|
+
GridFS is a specification for storing and retrieving large files, such as images, videos, and other binary data, within MongoDB.
|
|
632
|
+
Instead of storing large files as a single document (which can be inefficient), GridFS splits the file into smaller chunks.
|
|
633
|
+
|
|
634
|
+
Basic Operations:
|
|
635
|
+
|
|
636
|
+
Upload a File:
|
|
637
|
+
|
|
638
|
+
from pymongo import MongoClient
|
|
639
|
+
import gridfs
|
|
640
|
+
|
|
641
|
+
client = MongoClient("mongodb://localhost:27017")
|
|
642
|
+
db = client.mydatabase
|
|
643
|
+
fs = gridfs.GridFS(db)
|
|
644
|
+
|
|
645
|
+
with open("largefile.txt", "rb") as f:
|
|
646
|
+
fs.put(f, filename="largefile.txt")
|
|
647
|
+
|
|
648
|
+
|
|
649
|
+
Download a File:
|
|
650
|
+
|
|
651
|
+
with open("downloaded_file.txt", "wb") as f:
|
|
652
|
+
file = fs.get_last_version(filename="largefile.txt")
|
|
653
|
+
f.write(file.read())
|
|
654
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
655
|
+
|
|
656
|
+
2. MongoDB Aggregation Pipeline Stages
|
|
657
|
+
|
|
658
|
+
The aggregation framework provides a powerful set of stages to transform, filter, and manipulate data.
|
|
659
|
+
|
|
660
|
+
$match: Filters documents that pass certain criteria (similar to find()).
|
|
661
|
+
|
|
662
|
+
$group: Groups documents by a specified identifier.
|
|
663
|
+
|
|
664
|
+
$project: Reshapes documents by adding, removing, or renaming fields.
|
|
665
|
+
|
|
666
|
+
$sort: Sorts the documents.
|
|
667
|
+
|
|
668
|
+
$lookup: Performs a left outer join with another collection.
|
|
669
|
+
|
|
670
|
+
Example using $lookup:
|
|
671
|
+
db.orders.aggregate([
|
|
672
|
+
{
|
|
673
|
+
$lookup: {
|
|
674
|
+
from: "products",
|
|
675
|
+
localField: "product_id",
|
|
676
|
+
foreignField: "_id",
|
|
677
|
+
as: "product_details"
|
|
678
|
+
}
|
|
679
|
+
}
|
|
680
|
+
]);
|
|
681
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
682
|
+
|
|
683
|
+
3. Transactions in MongoDB
|
|
684
|
+
|
|
685
|
+
MongoDB supports multi-document ACID transactions, which ensures that multiple operations
|
|
686
|
+
on multiple documents or collections are executed atomically.
|
|
687
|
+
|
|
688
|
+
Basic Transaction:
|
|
689
|
+
const session = client.startSession();
|
|
690
|
+
|
|
691
|
+
session.startTransaction();
|
|
692
|
+
|
|
693
|
+
try {
|
|
694
|
+
db.collection1.updateOne({ _id: 1 }, { $set: { status: "Processed" } }, { session });
|
|
695
|
+
db.collection2.updateOne({ _id: 1 }, { $set: { status: "Completed" } }, { session });
|
|
696
|
+
|
|
697
|
+
session.commitTransaction();
|
|
698
|
+
} catch (error) {
|
|
699
|
+
session.abortTransaction();
|
|
700
|
+
console.log("Transaction failed:", error);
|
|
701
|
+
} finally {
|
|
702
|
+
session.endSession();
|
|
703
|
+
}
|
|
704
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
705
|
+
|
|
706
|
+
4. Data Migration Tools
|
|
707
|
+
|
|
708
|
+
MongoDB Atlas provides a Migration Tool for moving data between clusters.
|
|
709
|
+
|
|
710
|
+
mongoimport and mongoexport: These tools allow you to import and export JSON or CSV data from MongoDB.
|
|
711
|
+
|
|
712
|
+
Example:
|
|
713
|
+
|
|
714
|
+
Import JSON Data:
|
|
715
|
+
|
|
716
|
+
mongoimport --db mydb --collection mycollection --file data.json --jsonArray
|
|
717
|
+
|
|
718
|
+
|
|
719
|
+
Export Data to JSON:
|
|
720
|
+
|
|
721
|
+
mongoexport --db mydb --collection mycollection --out data.json
|
|
722
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
723
|
+
|
|
724
|
+
5. MongoDB WiredTiger Storage Engine
|
|
725
|
+
|
|
726
|
+
The WiredTiger storage engine is the default storage engine in MongoDB, providing features like:
|
|
727
|
+
|
|
728
|
+
Compression: Reduces disk usage.
|
|
729
|
+
|
|
730
|
+
Concurrency: Better performance with multi-core systems due to its ability to handle concurrent reads and writes.
|
|
731
|
+
|
|
732
|
+
Changing Storage Engine:
|
|
733
|
+
db.createCollection("myCollection", { storageEngine: { wiredTiger: { configString: "block_compressor=zlib" } } });
|
|
734
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
735
|
+
|
|
736
|
+
6. MongoDB Data Modeling Patterns
|
|
737
|
+
|
|
738
|
+
One-to-Many: Store related data in separate documents and use references.
|
|
739
|
+
|
|
740
|
+
Example: Users and their posts.
|
|
741
|
+
|
|
742
|
+
Many-to-Many: Use reference documents to represent relationships between multiple collections.
|
|
743
|
+
|
|
744
|
+
Example: Students and Courses (many students can enroll in many courses).
|
|
745
|
+
|
|
746
|
+
Example of One-to-Many:
|
|
747
|
+
// Users collection
|
|
748
|
+
{ "_id": 1, "name": "John" }
|
|
749
|
+
|
|
750
|
+
// Posts collection
|
|
751
|
+
{ "_id": 101, "user_id": 1, "content": "MongoDB is great!" }
|
|
752
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
753
|
+
|
|
754
|
+
7. MongoDB Performance Tuning
|
|
755
|
+
|
|
756
|
+
To ensure optimal performance, MongoDB offers several tuning mechanisms:
|
|
757
|
+
|
|
758
|
+
Indexing: Always create indexes on frequently queried fields.
|
|
759
|
+
|
|
760
|
+
Profiling: Use MongoDB’s built-in profiler to log slow queries and optimize them.
|
|
761
|
+
|
|
762
|
+
Connection Pooling: Efficiently manage the number of connections to the database.
|
|
763
|
+
|
|
764
|
+
Enable Profiling:
|
|
765
|
+
db.setProfilingLevel(2); // Logs all queries.
|
|
766
|
+
|
|
767
|
+
Example of Indexing:
|
|
768
|
+
db.collection.createIndex({ name: 1, age: -1 });
|
|
769
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
770
|
+
|
|
771
|
+
8. MongoDB Replica Set Elections
|
|
772
|
+
|
|
773
|
+
In a replica set, when the primary node goes down, a new primary must be elected from the secondary nodes.
|
|
774
|
+
MongoDB handles this election process automatically. However, you can influence the election process by:
|
|
775
|
+
|
|
776
|
+
Priority: The node with higher priority is more likely to be elected.
|
|
777
|
+
|
|
778
|
+
Votes: Each member of the replica set has a vote.
|
|
779
|
+
|
|
780
|
+
Example of Setting Priority:
|
|
781
|
+
cfg = rs.conf();
|
|
782
|
+
cfg.members[0].priority = 2; // Set higher priority for the first node
|
|
783
|
+
rs.reconfig(cfg);
|
|
784
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
785
|
+
|
|
786
|
+
9. MongoDB Change Streams
|
|
787
|
+
|
|
788
|
+
Change streams allow you to watch real-time changes in the MongoDB database without the need to continuously poll.
|
|
789
|
+
This is useful for building real-time applications, such as notifications, dashboards, and event-driven systems.
|
|
790
|
+
|
|
791
|
+
Example using Change Streams:
|
|
792
|
+
const changeStream = db.collection.watch();
|
|
793
|
+
|
|
794
|
+
changeStream.on('change', (next) => {
|
|
795
|
+
console.log(next);
|
|
796
|
+
});
|
|
797
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
798
|
+
|
|
799
|
+
10. MongoDB Atlas & Cloud
|
|
800
|
+
|
|
801
|
+
MongoDB Atlas is MongoDB’s managed cloud database solution, offering:
|
|
802
|
+
|
|
803
|
+
Auto-scaling: Automatically scales your database based on load.
|
|
804
|
+
|
|
805
|
+
Performance Monitoring: Real-time performance metrics and diagnostics.
|
|
806
|
+
|
|
807
|
+
Data Security: Built-in encryption and access controls.
|
|
808
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
809
|
+
|
|
810
|
+
11. MongoDB and Full-Text Search
|
|
811
|
+
|
|
812
|
+
MongoDB provides a full-text search feature that can be enabled on specific fields of a collection.
|
|
813
|
+
This allows you to perform complex text search queries, including stemming, text indexing, and more.
|
|
814
|
+
|
|
815
|
+
Creating a Text Index:
|
|
816
|
+
db.collection.createIndex({ title: "text", description: "text" });
|
|
817
|
+
|
|
818
|
+
Text Search Query:
|
|
819
|
+
db.collection.find({ $text: { $search: "MongoDB" } });
|
|
820
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
821
|
+
|
|
822
|
+
12. MongoDB Schema Design Patterns
|
|
823
|
+
|
|
824
|
+
There are several design patterns for organizing data in MongoDB:
|
|
825
|
+
|
|
826
|
+
Normalization: Similar to relational database design, where data is kept in separate collections, and references are used.
|
|
827
|
+
|
|
828
|
+
Denormalization: Data is stored in a single collection, often for performance reasons.
|
|
829
|
+
|
|
830
|
+
CQRS (Command Query Responsibility Segregation): Separate the write and read models for better performance and scalability.
|
|
831
|
+
-------------------------------------------------------------------------------------------------------------------------
|
|
832
|
+
|
|
833
|
+
13. MongoDB MapReduce
|
|
834
|
+
|
|
835
|
+
MapReduce is used for processing large datasets in parallel. MongoDB’s MapReduce function allows you to execute JavaScript functions for mapping and reducing tasks.
|
|
836
|
+
|
|
837
|
+
Example:
|
|
838
|
+
db.collection.mapReduce(
|
|
839
|
+
function() { emit(this.age, 1); },
|
|
840
|
+
function(key, values) { return Array.sum(values); },
|
|
841
|
+
{ out: "age_count" }
|
|
842
|
+
);
|