simpledht 0.1.0__tar.gz → 0.1.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- simpledht-0.1.2/PKG-INFO +27 -0
- simpledht-0.1.0/simpledht.egg-info/PKG-INFO → simpledht-0.1.2/README.md +53 -37
- {simpledht-0.1.0 → simpledht-0.1.2}/setup.py +5 -5
- simpledht-0.1.2/simpledht/__init__.py +28 -0
- {simpledht-0.1.0 → simpledht-0.1.2}/simpledht/cli.py +54 -10
- {simpledht-0.1.0 → simpledht-0.1.2}/simpledht/dht_node.py +170 -5
- simpledht-0.1.2/simpledht.egg-info/PKG-INFO +27 -0
- simpledht-0.1.0/PKG-INFO +0 -193
- simpledht-0.1.0/README.md +0 -167
- simpledht-0.1.0/simpledht/__init__.py +0 -10
- {simpledht-0.1.0 → simpledht-0.1.2}/LICENSE +0 -0
- {simpledht-0.1.0 → simpledht-0.1.2}/pyproject.toml +0 -0
- {simpledht-0.1.0 → simpledht-0.1.2}/setup.cfg +0 -0
- {simpledht-0.1.0 → simpledht-0.1.2}/simpledht/test_dht.py +0 -0
- {simpledht-0.1.0 → simpledht-0.1.2}/simpledht.egg-info/SOURCES.txt +0 -0
- {simpledht-0.1.0 → simpledht-0.1.2}/simpledht.egg-info/dependency_links.txt +0 -0
- {simpledht-0.1.0 → simpledht-0.1.2}/simpledht.egg-info/entry_points.txt +0 -0
- {simpledht-0.1.0 → simpledht-0.1.2}/simpledht.egg-info/requires.txt +0 -0
- {simpledht-0.1.0 → simpledht-0.1.2}/simpledht.egg-info/top_level.txt +0 -0
simpledht-0.1.2/PKG-INFO
ADDED
@@ -0,0 +1,27 @@
|
|
1
|
+
Metadata-Version: 2.4
|
2
|
+
Name: simpledht
|
3
|
+
Version: 0.1.2
|
4
|
+
Summary: A simple distributed hash table implementation
|
5
|
+
Home-page: https://github.com/dhruvldrp9/simpledht
|
6
|
+
Author: Dhruvkumar Patel
|
7
|
+
Author-email: dhruv.ldrp9@gmail.com
|
8
|
+
Classifier: Programming Language :: Python :: 3
|
9
|
+
Classifier: License :: OSI Approved :: MIT License
|
10
|
+
Classifier: Operating System :: OS Independent
|
11
|
+
Requires-Python: >=3.6
|
12
|
+
Description-Content-Type: text/markdown
|
13
|
+
License-File: LICENSE
|
14
|
+
Requires-Dist: requests>=2.25.1
|
15
|
+
Requires-Dist: click>=8.0.1
|
16
|
+
Dynamic: author
|
17
|
+
Dynamic: author-email
|
18
|
+
Dynamic: classifier
|
19
|
+
Dynamic: description
|
20
|
+
Dynamic: description-content-type
|
21
|
+
Dynamic: home-page
|
22
|
+
Dynamic: license-file
|
23
|
+
Dynamic: requires-dist
|
24
|
+
Dynamic: requires-python
|
25
|
+
Dynamic: summary
|
26
|
+
|
27
|
+
A simple distributed hash table implementation
|
@@ -1,29 +1,3 @@
|
|
1
|
-
Metadata-Version: 2.4
|
2
|
-
Name: simpledht
|
3
|
-
Version: 0.1.0
|
4
|
-
Summary: A simple distributed hash table implementation
|
5
|
-
Home-page: https://github.com/yourusername/simpledht
|
6
|
-
Author: Your Name
|
7
|
-
Author-email: your.email@example.com
|
8
|
-
Classifier: Programming Language :: Python :: 3
|
9
|
-
Classifier: License :: OSI Approved :: MIT License
|
10
|
-
Classifier: Operating System :: OS Independent
|
11
|
-
Requires-Python: >=3.6
|
12
|
-
Description-Content-Type: text/markdown
|
13
|
-
License-File: LICENSE
|
14
|
-
Requires-Dist: requests>=2.25.1
|
15
|
-
Requires-Dist: click>=8.0.1
|
16
|
-
Dynamic: author
|
17
|
-
Dynamic: author-email
|
18
|
-
Dynamic: classifier
|
19
|
-
Dynamic: description
|
20
|
-
Dynamic: description-content-type
|
21
|
-
Dynamic: home-page
|
22
|
-
Dynamic: license-file
|
23
|
-
Dynamic: requires-dist
|
24
|
-
Dynamic: requires-python
|
25
|
-
Dynamic: summary
|
26
|
-
|
27
1
|
# Distributed Hash Table (DHT) Implementation
|
28
2
|
|
29
3
|
A Python-based Distributed Hash Table implementation that allows nodes to connect across different networks using IP addresses. This implementation supports key-value storage and retrieval across multiple nodes.
|
@@ -34,9 +8,12 @@ A Python-based Distributed Hash Table implementation that allows nodes to connec
|
|
34
8
|
- Key-value storage and retrieval
|
35
9
|
- Automatic node discovery
|
36
10
|
- Data replication between nodes
|
11
|
+
- Data synchronization when joining the network
|
12
|
+
- Reliable bootstrapping with retry mechanism
|
37
13
|
- Simple CLI interface
|
38
14
|
- Public IP detection
|
39
15
|
- Local network support
|
16
|
+
- Python library interface for programmatic use
|
40
17
|
|
41
18
|
## Installation
|
42
19
|
|
@@ -67,51 +44,81 @@ pip install -e .
|
|
67
44
|
|
68
45
|
## Usage
|
69
46
|
|
70
|
-
###
|
47
|
+
### As a Python Library
|
48
|
+
|
49
|
+
The package can be used programmatically in your Python code:
|
50
|
+
|
51
|
+
```python
|
52
|
+
from simpledht import DHTNode
|
53
|
+
|
54
|
+
# Create and start a node
|
55
|
+
node = DHTNode(host='0.0.0.0', port=5000)
|
56
|
+
node.start()
|
57
|
+
|
58
|
+
# Store data
|
59
|
+
node.put('mykey', 'myvalue')
|
60
|
+
|
61
|
+
# Retrieve data
|
62
|
+
value = node.get('mykey')
|
63
|
+
|
64
|
+
# Connect to another node
|
65
|
+
node.bootstrap('other_node_ip:5000')
|
66
|
+
|
67
|
+
# Stop the node when done
|
68
|
+
node.stop()
|
69
|
+
```
|
70
|
+
|
71
|
+
See the `examples/` directory for more detailed usage examples:
|
72
|
+
- `basic_usage.py`: Simple example of creating and connecting nodes
|
73
|
+
- `distributed_storage.py`: Advanced example showing distributed storage with multiple nodes
|
74
|
+
|
75
|
+
### Command Line Interface
|
76
|
+
|
77
|
+
#### Starting a Node
|
71
78
|
|
72
79
|
To start a new DHT node:
|
73
80
|
```bash
|
74
|
-
|
81
|
+
simpledht start --host 0.0.0.0 --port 5000
|
75
82
|
```
|
76
83
|
|
77
84
|
To start a node and connect to existing nodes:
|
78
85
|
```bash
|
79
|
-
|
86
|
+
simpledht start --host 0.0.0.0 --port 5001 --bootstrap "PUBLIC_IP:5000"
|
80
87
|
```
|
81
88
|
|
82
|
-
|
89
|
+
#### Storing Data
|
83
90
|
|
84
91
|
To store a key-value pair:
|
85
92
|
```bash
|
86
|
-
|
93
|
+
simpledht put --host PUBLIC_IP --port 5000 mykey "my value"
|
87
94
|
```
|
88
95
|
|
89
|
-
|
96
|
+
#### Retrieving Data
|
90
97
|
|
91
98
|
To retrieve a value:
|
92
99
|
```bash
|
93
|
-
|
100
|
+
simpledht get --host PUBLIC_IP --port 5000 mykey
|
94
101
|
```
|
95
102
|
|
96
103
|
### Cross-Network Example
|
97
104
|
|
98
105
|
1. Start Node 1 (First network):
|
99
106
|
```bash
|
100
|
-
|
107
|
+
simpledht start --host 0.0.0.0 --port 5000
|
101
108
|
```
|
102
109
|
|
103
110
|
2. Start Node 2 (Second network):
|
104
111
|
```bash
|
105
|
-
|
112
|
+
simpledht start --host 0.0.0.0 --port 5000 --bootstrap "NODE1_PUBLIC_IP:5000"
|
106
113
|
```
|
107
114
|
|
108
115
|
3. Store and retrieve data:
|
109
116
|
```bash
|
110
117
|
# Store on Node 1
|
111
|
-
|
118
|
+
simpledht put --host NODE1_PUBLIC_IP --port 5000 test_key "test_value"
|
112
119
|
|
113
120
|
# Retrieve from Node 2
|
114
|
-
|
121
|
+
simpledht get --host NODE2_PUBLIC_IP --port 5000 test_key
|
115
122
|
```
|
116
123
|
|
117
124
|
## Network Configuration
|
@@ -135,6 +142,13 @@ If your node is behind a NAT router:
|
|
135
142
|
2. Set up port forwarding for UDP port 5000
|
136
143
|
3. Forward to your node's local IP address
|
137
144
|
|
145
|
+
## New Features in Version 0.1.2
|
146
|
+
|
147
|
+
- **Improved Bootstrap Mechanism**: Added retry logic for more reliable connections across networks
|
148
|
+
- **Data Synchronization**: Nodes automatically sync data when joining the network
|
149
|
+
- **Enhanced Error Handling**: Better handling of network timeouts and connection issues
|
150
|
+
- **Full Data Replication**: All nodes maintain a complete copy of the data for redundancy
|
151
|
+
|
138
152
|
## Troubleshooting
|
139
153
|
|
140
154
|
### Common Issues
|
@@ -161,6 +175,7 @@ If your node is behind a NAT router:
|
|
161
175
|
- `No response received`: Node is not responding
|
162
176
|
- `Address already in use`: Port conflict
|
163
177
|
- `Failed to get public IP`: Network connectivity issue
|
178
|
+
- `Connection attempt X/3 timed out, retrying...`: Network latency or connectivity issues
|
164
179
|
|
165
180
|
## Architecture
|
166
181
|
|
@@ -170,6 +185,7 @@ The DHT implementation uses:
|
|
170
185
|
- Automatic public IP detection
|
171
186
|
- Data replication between nodes
|
172
187
|
- Bootstrap nodes for network discovery
|
188
|
+
- Retry mechanism for reliable connections
|
173
189
|
|
174
190
|
## Security Considerations
|
175
191
|
|
@@ -5,13 +5,13 @@ with open("README.md", "r", encoding="utf-8") as fh:
|
|
5
5
|
|
6
6
|
setup(
|
7
7
|
name="simpledht",
|
8
|
-
version="0.1.
|
9
|
-
author="
|
10
|
-
author_email="
|
8
|
+
version="0.1.2",
|
9
|
+
author="Dhruvkumar Patel",
|
10
|
+
author_email="dhruv.ldrp9@gmail.com",
|
11
11
|
description="A simple distributed hash table implementation",
|
12
|
-
long_description=
|
12
|
+
long_description="A simple distributed hash table implementation",
|
13
13
|
long_description_content_type="text/markdown",
|
14
|
-
url="https://github.com/
|
14
|
+
url="https://github.com/dhruvldrp9/simpledht",
|
15
15
|
packages=find_packages(),
|
16
16
|
classifiers=[
|
17
17
|
"Programming Language :: Python :: 3",
|
@@ -0,0 +1,28 @@
|
|
1
|
+
"""
|
2
|
+
SimpleDHT - A simple distributed hash table implementation
|
3
|
+
|
4
|
+
Example usage:
|
5
|
+
from simpledht import DHTNode
|
6
|
+
|
7
|
+
# Create a new node
|
8
|
+
node = DHTNode(host='0.0.0.0', port=5000)
|
9
|
+
|
10
|
+
# Start the node
|
11
|
+
node.start()
|
12
|
+
|
13
|
+
# Store a value
|
14
|
+
node.put('mykey', 'myvalue')
|
15
|
+
|
16
|
+
# Retrieve a value
|
17
|
+
value = node.get('mykey')
|
18
|
+
|
19
|
+
# Connect to another node
|
20
|
+
node.bootstrap('other_node_ip:5000')
|
21
|
+
"""
|
22
|
+
|
23
|
+
__version__ = "0.1.2"
|
24
|
+
|
25
|
+
from .dht_node import DHTNode
|
26
|
+
from .cli import main as cli_main
|
27
|
+
|
28
|
+
__all__ = ["DHTNode", "cli_main"]
|
@@ -1,5 +1,5 @@
|
|
1
1
|
import click
|
2
|
-
from dht_node import DHTNode
|
2
|
+
from .dht_node import DHTNode
|
3
3
|
import json
|
4
4
|
import time
|
5
5
|
import socket
|
@@ -12,19 +12,35 @@ def cli():
|
|
12
12
|
|
13
13
|
@cli.command()
|
14
14
|
@click.option('--host', default='0.0.0.0', help='Host to bind to')
|
15
|
-
@click.option('--port', default=5000, help='Port to bind to')
|
15
|
+
@click.option('--port', default=5000, type=int, help='Port to bind to')
|
16
16
|
@click.option('--bootstrap', help='Comma-separated list of bootstrap nodes (host:port)')
|
17
17
|
def start(host, port, bootstrap):
|
18
18
|
"""Start a new DHT node."""
|
19
19
|
bootstrap_nodes = bootstrap.split(',') if bootstrap else []
|
20
|
-
node = DHTNode(host, port, bootstrap_nodes)
|
21
|
-
node.start()
|
22
20
|
|
23
21
|
try:
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
22
|
+
node = DHTNode(host, port, bootstrap_nodes)
|
23
|
+
node.start()
|
24
|
+
|
25
|
+
click.echo(f"Node started successfully on {host}:{port}")
|
26
|
+
click.echo(f"Public IP: {node.public_ip}")
|
27
|
+
click.echo(f"Node ID: {node.id}")
|
28
|
+
|
29
|
+
if bootstrap_nodes:
|
30
|
+
click.echo(f"Connected to bootstrap nodes: {', '.join(bootstrap_nodes)}")
|
31
|
+
|
32
|
+
click.echo("Press Ctrl+C to stop the node...")
|
33
|
+
|
34
|
+
try:
|
35
|
+
while True:
|
36
|
+
time.sleep(1)
|
37
|
+
except KeyboardInterrupt:
|
38
|
+
click.echo("Stopping node...")
|
39
|
+
node.stop()
|
40
|
+
click.echo("Node stopped.")
|
41
|
+
except Exception as e:
|
42
|
+
click.echo(f"Error starting node: {e}")
|
43
|
+
sys.exit(1)
|
28
44
|
|
29
45
|
def _send_message(host: str, port: int, message: dict, timeout: int = 5) -> dict:
|
30
46
|
"""Send a message to a DHT node and wait for response."""
|
@@ -34,7 +50,7 @@ def _send_message(host: str, port: int, message: dict, timeout: int = 5) -> dict
|
|
34
50
|
sock.settimeout(timeout)
|
35
51
|
|
36
52
|
# Send message
|
37
|
-
click.echo(f"
|
53
|
+
click.echo(f"Connecting to node at {host}:{port}...")
|
38
54
|
sock.sendto(json.dumps(message).encode(), (host, port))
|
39
55
|
|
40
56
|
# Wait for response
|
@@ -70,8 +86,10 @@ def put(host, port, timeout, key, value):
|
|
70
86
|
|
71
87
|
if response.get('type') == 'store_ack':
|
72
88
|
click.echo(f"Successfully stored {key}={value}")
|
89
|
+
click.echo("Value has been replicated to all nodes in the network")
|
73
90
|
else:
|
74
91
|
click.echo(f"Failed to store {key}={value}")
|
92
|
+
click.echo(f"Response: {response}")
|
75
93
|
|
76
94
|
@cli.command()
|
77
95
|
@click.option('--host', required=True, help='Host of the DHT node')
|
@@ -93,6 +111,32 @@ def get(host, port, timeout, key):
|
|
93
111
|
click.echo(f"Value for {key}: {value}")
|
94
112
|
else:
|
95
113
|
click.echo(f"Failed to retrieve value for key: {key}")
|
114
|
+
click.echo(f"Response: {response}")
|
115
|
+
|
116
|
+
@cli.command()
|
117
|
+
@click.option('--host', required=True, help='Host of the DHT node')
|
118
|
+
@click.option('--port', required=True, type=int, help='Port of the DHT node')
|
119
|
+
@click.option('--timeout', default=5, help='Timeout in seconds')
|
120
|
+
def info(host, port, timeout):
|
121
|
+
"""Get information about a DHT node."""
|
122
|
+
response = _send_message(host, port, {
|
123
|
+
'type': 'info_request'
|
124
|
+
}, timeout)
|
125
|
+
|
126
|
+
if response.get('type') == 'info_response':
|
127
|
+
node_id = response.get('node_id', 'Unknown')
|
128
|
+
peer_count = response.get('peer_count', 0)
|
129
|
+
data_count = response.get('data_count', 0)
|
130
|
+
|
131
|
+
click.echo(f"Node ID: {node_id}")
|
132
|
+
click.echo(f"Connected peers: {peer_count}")
|
133
|
+
click.echo(f"Stored key-value pairs: {data_count}")
|
134
|
+
else:
|
135
|
+
click.echo("Failed to get node information")
|
136
|
+
|
137
|
+
def main():
|
138
|
+
"""Entry point for the CLI."""
|
139
|
+
cli()
|
96
140
|
|
97
141
|
if __name__ == '__main__':
|
98
|
-
|
142
|
+
main()
|
@@ -128,6 +128,24 @@ class DHTNode:
|
|
128
128
|
# Update our routing table with the received nodes
|
129
129
|
received_table = message.get('routing_table', {})
|
130
130
|
self.routing_table.update(received_table)
|
131
|
+
elif msg_type == 'sync_request':
|
132
|
+
# Send our data to the requesting node
|
133
|
+
self._send_response(addr, {
|
134
|
+
'type': 'sync_response',
|
135
|
+
'data': self.data
|
136
|
+
})
|
137
|
+
elif msg_type == 'sync_response':
|
138
|
+
# Update our data with the received data
|
139
|
+
received_data = message.get('data', {})
|
140
|
+
self.data.update(received_data)
|
141
|
+
elif msg_type == 'info_request':
|
142
|
+
# Send information about this node
|
143
|
+
self._send_response(addr, {
|
144
|
+
'type': 'info_response',
|
145
|
+
'node_id': self.id,
|
146
|
+
'peer_count': len(self.routing_table) - 1, # Exclude self
|
147
|
+
'data_count': len(self.data)
|
148
|
+
})
|
131
149
|
|
132
150
|
def _replicate_data(self, key: str, value: str):
|
133
151
|
"""Replicate data to other nodes in the network."""
|
@@ -164,11 +182,31 @@ class DHTNode:
|
|
164
182
|
'value': None
|
165
183
|
})
|
166
184
|
|
167
|
-
def _send_message_with_response(self, addr: Tuple[str, int], message: dict) -> dict:
|
168
|
-
"""Send a message and wait for response.
|
185
|
+
def _send_message_with_response(self, addr: Tuple[str, int], message: dict, timeout: int = 5) -> dict:
|
186
|
+
"""Send a message and wait for response.
|
187
|
+
|
188
|
+
Args:
|
189
|
+
addr: The address to send the message to
|
190
|
+
message: The message to send
|
191
|
+
timeout: The timeout in seconds
|
192
|
+
|
193
|
+
Returns:
|
194
|
+
The response message
|
195
|
+
"""
|
169
196
|
self._send_message(addr, message)
|
170
|
-
|
171
|
-
|
197
|
+
|
198
|
+
# Set timeout for receiving response
|
199
|
+
self.socket.settimeout(timeout)
|
200
|
+
try:
|
201
|
+
data, _ = self.socket.recvfrom(4096)
|
202
|
+
# Reset timeout to None (blocking mode)
|
203
|
+
self.socket.settimeout(None)
|
204
|
+
return json.loads(data.decode())
|
205
|
+
except socket.timeout:
|
206
|
+
raise
|
207
|
+
finally:
|
208
|
+
# Make sure we reset the timeout even if an exception occurs
|
209
|
+
self.socket.settimeout(None)
|
172
210
|
|
173
211
|
def _send_response(self, addr: Tuple[str, int], response: dict):
|
174
212
|
"""Send a response to a node."""
|
@@ -208,4 +246,131 @@ class DHTNode:
|
|
208
246
|
def stop(self):
|
209
247
|
"""Stop the DHT node."""
|
210
248
|
self.running = False
|
211
|
-
self.socket.close()
|
249
|
+
self.socket.close()
|
250
|
+
|
251
|
+
def put(self, key: str, value: str) -> bool:
|
252
|
+
"""Store a key-value pair in the DHT.
|
253
|
+
|
254
|
+
Args:
|
255
|
+
key: The key to store
|
256
|
+
value: The value to store
|
257
|
+
|
258
|
+
Returns:
|
259
|
+
bool: True if successful, False otherwise
|
260
|
+
"""
|
261
|
+
# Store locally
|
262
|
+
self.data[key] = value
|
263
|
+
|
264
|
+
# Replicate to other nodes
|
265
|
+
self._replicate_data(key, value)
|
266
|
+
|
267
|
+
return True
|
268
|
+
|
269
|
+
def get(self, key: str) -> Optional[str]:
|
270
|
+
"""Retrieve a value from the DHT.
|
271
|
+
|
272
|
+
Args:
|
273
|
+
key: The key to retrieve
|
274
|
+
|
275
|
+
Returns:
|
276
|
+
The value if found, None otherwise
|
277
|
+
"""
|
278
|
+
# Check if we have it locally
|
279
|
+
if key in self.data:
|
280
|
+
return self.data[key]
|
281
|
+
|
282
|
+
# If not, ask other nodes
|
283
|
+
for node_id, (host, port) in self.routing_table.items():
|
284
|
+
if node_id != self.id: # Don't ask self
|
285
|
+
try:
|
286
|
+
response = self._send_message_with_response((host, port), {
|
287
|
+
'type': 'get',
|
288
|
+
'key': key
|
289
|
+
})
|
290
|
+
|
291
|
+
if response.get('type') == 'get_response' and response.get('value') is not None:
|
292
|
+
return response.get('value')
|
293
|
+
except Exception as e:
|
294
|
+
print(f"Failed to get value from {host}:{port}: {e}")
|
295
|
+
|
296
|
+
# Not found anywhere
|
297
|
+
return None
|
298
|
+
|
299
|
+
def bootstrap(self, node_address: str):
|
300
|
+
"""Connect to another node to join the network.
|
301
|
+
|
302
|
+
Args:
|
303
|
+
node_address: The address of the node to connect to in the format 'host:port'
|
304
|
+
|
305
|
+
Returns:
|
306
|
+
bool: True if successfully connected, False otherwise
|
307
|
+
"""
|
308
|
+
try:
|
309
|
+
parts = node_address.split(':')
|
310
|
+
if len(parts) != 2:
|
311
|
+
print(f"Invalid bootstrap node format: {node_address}. Expected format: IP:PORT")
|
312
|
+
return False
|
313
|
+
|
314
|
+
host, port = parts
|
315
|
+
try:
|
316
|
+
port = int(port)
|
317
|
+
except ValueError:
|
318
|
+
print(f"Invalid port number in bootstrap node: {node_address}")
|
319
|
+
return False
|
320
|
+
|
321
|
+
print(f"Bootstrapping with node {host}:{port}")
|
322
|
+
|
323
|
+
# Try multiple times in case of network issues
|
324
|
+
max_attempts = 3
|
325
|
+
for attempt in range(max_attempts):
|
326
|
+
try:
|
327
|
+
response = self._send_message_with_response((host, port), {
|
328
|
+
'type': 'join',
|
329
|
+
'node_id': self.id,
|
330
|
+
'host': self.public_ip,
|
331
|
+
'port': self.port
|
332
|
+
}, timeout=5)
|
333
|
+
|
334
|
+
if response.get('type') == 'join_ack':
|
335
|
+
received_table = response.get('routing_table', {})
|
336
|
+
self.routing_table.update(received_table)
|
337
|
+
print(f"Successfully connected to {host}:{port}")
|
338
|
+
|
339
|
+
# Sync data with the network
|
340
|
+
self._sync_data_with_network()
|
341
|
+
return True
|
342
|
+
break
|
343
|
+
except socket.timeout:
|
344
|
+
print(f"Connection attempt {attempt+1}/{max_attempts} timed out, retrying...")
|
345
|
+
if attempt == max_attempts - 1:
|
346
|
+
print(f"Failed to connect to {host}:{port} after {max_attempts} attempts")
|
347
|
+
return False
|
348
|
+
except Exception as e:
|
349
|
+
print(f"Error during connection attempt {attempt+1}: {e}")
|
350
|
+
break
|
351
|
+
except Exception as e:
|
352
|
+
print(f"Failed to bootstrap with {node_address}: {e}")
|
353
|
+
|
354
|
+
return False
|
355
|
+
|
356
|
+
def _sync_data_with_network(self):
|
357
|
+
"""Sync data with other nodes in the network after joining."""
|
358
|
+
if not self.routing_table:
|
359
|
+
return
|
360
|
+
|
361
|
+
# Request data from a random node in the routing table
|
362
|
+
for node_id, (host, port) in self.routing_table.items():
|
363
|
+
if node_id != self.id: # Don't ask self
|
364
|
+
try:
|
365
|
+
response = self._send_message_with_response((host, port), {
|
366
|
+
'type': 'sync_request'
|
367
|
+
})
|
368
|
+
|
369
|
+
if response.get('type') == 'sync_response':
|
370
|
+
# Update our data with the received data
|
371
|
+
received_data = response.get('data', {})
|
372
|
+
self.data.update(received_data)
|
373
|
+
print(f"Synced {len(received_data)} key-value pairs from the network")
|
374
|
+
return
|
375
|
+
except Exception as e:
|
376
|
+
print(f"Failed to sync data from {host}:{port}: {e}")
|
@@ -0,0 +1,27 @@
|
|
1
|
+
Metadata-Version: 2.4
|
2
|
+
Name: simpledht
|
3
|
+
Version: 0.1.2
|
4
|
+
Summary: A simple distributed hash table implementation
|
5
|
+
Home-page: https://github.com/dhruvldrp9/simpledht
|
6
|
+
Author: Dhruvkumar Patel
|
7
|
+
Author-email: dhruv.ldrp9@gmail.com
|
8
|
+
Classifier: Programming Language :: Python :: 3
|
9
|
+
Classifier: License :: OSI Approved :: MIT License
|
10
|
+
Classifier: Operating System :: OS Independent
|
11
|
+
Requires-Python: >=3.6
|
12
|
+
Description-Content-Type: text/markdown
|
13
|
+
License-File: LICENSE
|
14
|
+
Requires-Dist: requests>=2.25.1
|
15
|
+
Requires-Dist: click>=8.0.1
|
16
|
+
Dynamic: author
|
17
|
+
Dynamic: author-email
|
18
|
+
Dynamic: classifier
|
19
|
+
Dynamic: description
|
20
|
+
Dynamic: description-content-type
|
21
|
+
Dynamic: home-page
|
22
|
+
Dynamic: license-file
|
23
|
+
Dynamic: requires-dist
|
24
|
+
Dynamic: requires-python
|
25
|
+
Dynamic: summary
|
26
|
+
|
27
|
+
A simple distributed hash table implementation
|
simpledht-0.1.0/PKG-INFO
DELETED
@@ -1,193 +0,0 @@
|
|
1
|
-
Metadata-Version: 2.4
|
2
|
-
Name: simpledht
|
3
|
-
Version: 0.1.0
|
4
|
-
Summary: A simple distributed hash table implementation
|
5
|
-
Home-page: https://github.com/yourusername/simpledht
|
6
|
-
Author: Your Name
|
7
|
-
Author-email: your.email@example.com
|
8
|
-
Classifier: Programming Language :: Python :: 3
|
9
|
-
Classifier: License :: OSI Approved :: MIT License
|
10
|
-
Classifier: Operating System :: OS Independent
|
11
|
-
Requires-Python: >=3.6
|
12
|
-
Description-Content-Type: text/markdown
|
13
|
-
License-File: LICENSE
|
14
|
-
Requires-Dist: requests>=2.25.1
|
15
|
-
Requires-Dist: click>=8.0.1
|
16
|
-
Dynamic: author
|
17
|
-
Dynamic: author-email
|
18
|
-
Dynamic: classifier
|
19
|
-
Dynamic: description
|
20
|
-
Dynamic: description-content-type
|
21
|
-
Dynamic: home-page
|
22
|
-
Dynamic: license-file
|
23
|
-
Dynamic: requires-dist
|
24
|
-
Dynamic: requires-python
|
25
|
-
Dynamic: summary
|
26
|
-
|
27
|
-
# Distributed Hash Table (DHT) Implementation
|
28
|
-
|
29
|
-
A Python-based Distributed Hash Table implementation that allows nodes to connect across different networks using IP addresses. This implementation supports key-value storage and retrieval across multiple nodes.
|
30
|
-
|
31
|
-
## Features
|
32
|
-
|
33
|
-
- Cross-network node communication
|
34
|
-
- Key-value storage and retrieval
|
35
|
-
- Automatic node discovery
|
36
|
-
- Data replication between nodes
|
37
|
-
- Simple CLI interface
|
38
|
-
- Public IP detection
|
39
|
-
- Local network support
|
40
|
-
|
41
|
-
## Installation
|
42
|
-
|
43
|
-
### From PyPI (Recommended)
|
44
|
-
|
45
|
-
```bash
|
46
|
-
pip install simpledht
|
47
|
-
```
|
48
|
-
|
49
|
-
### From Source
|
50
|
-
|
51
|
-
1. Clone the repository:
|
52
|
-
```bash
|
53
|
-
git clone <repository-url>
|
54
|
-
cd SimpleDHT
|
55
|
-
```
|
56
|
-
|
57
|
-
2. Create and activate a virtual environment:
|
58
|
-
```bash
|
59
|
-
python -m venv env
|
60
|
-
source env/bin/activate # On Windows: env\Scripts\activate
|
61
|
-
```
|
62
|
-
|
63
|
-
3. Install the package in development mode:
|
64
|
-
```bash
|
65
|
-
pip install -e .
|
66
|
-
```
|
67
|
-
|
68
|
-
## Usage
|
69
|
-
|
70
|
-
### Starting a Node
|
71
|
-
|
72
|
-
To start a new DHT node:
|
73
|
-
```bash
|
74
|
-
python cli.py start --host 0.0.0.0 --port 5000
|
75
|
-
```
|
76
|
-
|
77
|
-
To start a node and connect to existing nodes:
|
78
|
-
```bash
|
79
|
-
python cli.py start --host 0.0.0.0 --port 5001 --bootstrap "PUBLIC_IP:5000"
|
80
|
-
```
|
81
|
-
|
82
|
-
### Storing Data
|
83
|
-
|
84
|
-
To store a key-value pair:
|
85
|
-
```bash
|
86
|
-
python cli.py put --host PUBLIC_IP --port 5000 mykey "my value"
|
87
|
-
```
|
88
|
-
|
89
|
-
### Retrieving Data
|
90
|
-
|
91
|
-
To retrieve a value:
|
92
|
-
```bash
|
93
|
-
python cli.py get --host PUBLIC_IP --port 5000 mykey
|
94
|
-
```
|
95
|
-
|
96
|
-
### Cross-Network Example
|
97
|
-
|
98
|
-
1. Start Node 1 (First network):
|
99
|
-
```bash
|
100
|
-
python cli.py start --host 0.0.0.0 --port 5000
|
101
|
-
```
|
102
|
-
|
103
|
-
2. Start Node 2 (Second network):
|
104
|
-
```bash
|
105
|
-
python cli.py start --host 0.0.0.0 --port 5000 --bootstrap "NODE1_PUBLIC_IP:5000"
|
106
|
-
```
|
107
|
-
|
108
|
-
3. Store and retrieve data:
|
109
|
-
```bash
|
110
|
-
# Store on Node 1
|
111
|
-
python cli.py put --host NODE1_PUBLIC_IP --port 5000 test_key "test_value"
|
112
|
-
|
113
|
-
# Retrieve from Node 2
|
114
|
-
python cli.py get --host NODE2_PUBLIC_IP --port 5000 test_key
|
115
|
-
```
|
116
|
-
|
117
|
-
## Network Configuration
|
118
|
-
|
119
|
-
### Firewall Setup
|
120
|
-
|
121
|
-
Ensure the UDP port (default: 5000) is open in your firewall:
|
122
|
-
|
123
|
-
```bash
|
124
|
-
# For UFW (Ubuntu)
|
125
|
-
sudo ufw allow 5000/udp
|
126
|
-
|
127
|
-
# For iptables
|
128
|
-
sudo iptables -A INPUT -p udp --dport 5000 -j ACCEPT
|
129
|
-
```
|
130
|
-
|
131
|
-
### Port Forwarding
|
132
|
-
|
133
|
-
If your node is behind a NAT router:
|
134
|
-
1. Access your router's admin interface
|
135
|
-
2. Set up port forwarding for UDP port 5000
|
136
|
-
3. Forward to your node's local IP address
|
137
|
-
|
138
|
-
## Troubleshooting
|
139
|
-
|
140
|
-
### Common Issues
|
141
|
-
|
142
|
-
1. **Connection Timeout**
|
143
|
-
- Check if the target node is running
|
144
|
-
- Verify firewall settings
|
145
|
-
- Ensure port forwarding is configured correctly
|
146
|
-
- Try increasing the timeout: `--timeout 10`
|
147
|
-
|
148
|
-
2. **Address Already in Use**
|
149
|
-
- The port is already being used by another process
|
150
|
-
- Try a different port number
|
151
|
-
- Check running processes: `netstat -tuln | grep 5000`
|
152
|
-
|
153
|
-
3. **No Response from Node**
|
154
|
-
- Verify the node is running
|
155
|
-
- Check network connectivity: `ping NODE_IP`
|
156
|
-
- Test port connectivity: `nc -vzu NODE_IP 5000`
|
157
|
-
|
158
|
-
### Error Messages
|
159
|
-
|
160
|
-
- `Failed to bootstrap with IP:PORT`: Invalid bootstrap node format
|
161
|
-
- `No response received`: Node is not responding
|
162
|
-
- `Address already in use`: Port conflict
|
163
|
-
- `Failed to get public IP`: Network connectivity issue
|
164
|
-
|
165
|
-
## Architecture
|
166
|
-
|
167
|
-
The DHT implementation uses:
|
168
|
-
- UDP sockets for communication
|
169
|
-
- SHA-256 for node ID generation
|
170
|
-
- Automatic public IP detection
|
171
|
-
- Data replication between nodes
|
172
|
-
- Bootstrap nodes for network discovery
|
173
|
-
|
174
|
-
## Security Considerations
|
175
|
-
|
176
|
-
- This is a basic implementation and should not be used in production without additional security measures
|
177
|
-
- Consider adding:
|
178
|
-
- Encryption for data in transit
|
179
|
-
- Authentication for node joining
|
180
|
-
- Rate limiting to prevent abuse
|
181
|
-
- Input validation
|
182
|
-
|
183
|
-
## Contributing
|
184
|
-
|
185
|
-
1. Fork the repository
|
186
|
-
2. Create a feature branch
|
187
|
-
3. Commit your changes
|
188
|
-
4. Push to the branch
|
189
|
-
5. Create a Pull Request
|
190
|
-
|
191
|
-
## License
|
192
|
-
|
193
|
-
This project is licensed under the MIT License - see the LICENSE file for details.
|
simpledht-0.1.0/README.md
DELETED
@@ -1,167 +0,0 @@
|
|
1
|
-
# Distributed Hash Table (DHT) Implementation
|
2
|
-
|
3
|
-
A Python-based Distributed Hash Table implementation that allows nodes to connect across different networks using IP addresses. This implementation supports key-value storage and retrieval across multiple nodes.
|
4
|
-
|
5
|
-
## Features
|
6
|
-
|
7
|
-
- Cross-network node communication
|
8
|
-
- Key-value storage and retrieval
|
9
|
-
- Automatic node discovery
|
10
|
-
- Data replication between nodes
|
11
|
-
- Simple CLI interface
|
12
|
-
- Public IP detection
|
13
|
-
- Local network support
|
14
|
-
|
15
|
-
## Installation
|
16
|
-
|
17
|
-
### From PyPI (Recommended)
|
18
|
-
|
19
|
-
```bash
|
20
|
-
pip install simpledht
|
21
|
-
```
|
22
|
-
|
23
|
-
### From Source
|
24
|
-
|
25
|
-
1. Clone the repository:
|
26
|
-
```bash
|
27
|
-
git clone <repository-url>
|
28
|
-
cd SimpleDHT
|
29
|
-
```
|
30
|
-
|
31
|
-
2. Create and activate a virtual environment:
|
32
|
-
```bash
|
33
|
-
python -m venv env
|
34
|
-
source env/bin/activate # On Windows: env\Scripts\activate
|
35
|
-
```
|
36
|
-
|
37
|
-
3. Install the package in development mode:
|
38
|
-
```bash
|
39
|
-
pip install -e .
|
40
|
-
```
|
41
|
-
|
42
|
-
## Usage
|
43
|
-
|
44
|
-
### Starting a Node
|
45
|
-
|
46
|
-
To start a new DHT node:
|
47
|
-
```bash
|
48
|
-
python cli.py start --host 0.0.0.0 --port 5000
|
49
|
-
```
|
50
|
-
|
51
|
-
To start a node and connect to existing nodes:
|
52
|
-
```bash
|
53
|
-
python cli.py start --host 0.0.0.0 --port 5001 --bootstrap "PUBLIC_IP:5000"
|
54
|
-
```
|
55
|
-
|
56
|
-
### Storing Data
|
57
|
-
|
58
|
-
To store a key-value pair:
|
59
|
-
```bash
|
60
|
-
python cli.py put --host PUBLIC_IP --port 5000 mykey "my value"
|
61
|
-
```
|
62
|
-
|
63
|
-
### Retrieving Data
|
64
|
-
|
65
|
-
To retrieve a value:
|
66
|
-
```bash
|
67
|
-
python cli.py get --host PUBLIC_IP --port 5000 mykey
|
68
|
-
```
|
69
|
-
|
70
|
-
### Cross-Network Example
|
71
|
-
|
72
|
-
1. Start Node 1 (First network):
|
73
|
-
```bash
|
74
|
-
python cli.py start --host 0.0.0.0 --port 5000
|
75
|
-
```
|
76
|
-
|
77
|
-
2. Start Node 2 (Second network):
|
78
|
-
```bash
|
79
|
-
python cli.py start --host 0.0.0.0 --port 5000 --bootstrap "NODE1_PUBLIC_IP:5000"
|
80
|
-
```
|
81
|
-
|
82
|
-
3. Store and retrieve data:
|
83
|
-
```bash
|
84
|
-
# Store on Node 1
|
85
|
-
python cli.py put --host NODE1_PUBLIC_IP --port 5000 test_key "test_value"
|
86
|
-
|
87
|
-
# Retrieve from Node 2
|
88
|
-
python cli.py get --host NODE2_PUBLIC_IP --port 5000 test_key
|
89
|
-
```
|
90
|
-
|
91
|
-
## Network Configuration
|
92
|
-
|
93
|
-
### Firewall Setup
|
94
|
-
|
95
|
-
Ensure the UDP port (default: 5000) is open in your firewall:
|
96
|
-
|
97
|
-
```bash
|
98
|
-
# For UFW (Ubuntu)
|
99
|
-
sudo ufw allow 5000/udp
|
100
|
-
|
101
|
-
# For iptables
|
102
|
-
sudo iptables -A INPUT -p udp --dport 5000 -j ACCEPT
|
103
|
-
```
|
104
|
-
|
105
|
-
### Port Forwarding
|
106
|
-
|
107
|
-
If your node is behind a NAT router:
|
108
|
-
1. Access your router's admin interface
|
109
|
-
2. Set up port forwarding for UDP port 5000
|
110
|
-
3. Forward to your node's local IP address
|
111
|
-
|
112
|
-
## Troubleshooting
|
113
|
-
|
114
|
-
### Common Issues
|
115
|
-
|
116
|
-
1. **Connection Timeout**
|
117
|
-
- Check if the target node is running
|
118
|
-
- Verify firewall settings
|
119
|
-
- Ensure port forwarding is configured correctly
|
120
|
-
- Try increasing the timeout: `--timeout 10`
|
121
|
-
|
122
|
-
2. **Address Already in Use**
|
123
|
-
- The port is already being used by another process
|
124
|
-
- Try a different port number
|
125
|
-
- Check running processes: `netstat -tuln | grep 5000`
|
126
|
-
|
127
|
-
3. **No Response from Node**
|
128
|
-
- Verify the node is running
|
129
|
-
- Check network connectivity: `ping NODE_IP`
|
130
|
-
- Test port connectivity: `nc -vzu NODE_IP 5000`
|
131
|
-
|
132
|
-
### Error Messages
|
133
|
-
|
134
|
-
- `Failed to bootstrap with IP:PORT`: Invalid bootstrap node format
|
135
|
-
- `No response received`: Node is not responding
|
136
|
-
- `Address already in use`: Port conflict
|
137
|
-
- `Failed to get public IP`: Network connectivity issue
|
138
|
-
|
139
|
-
## Architecture
|
140
|
-
|
141
|
-
The DHT implementation uses:
|
142
|
-
- UDP sockets for communication
|
143
|
-
- SHA-256 for node ID generation
|
144
|
-
- Automatic public IP detection
|
145
|
-
- Data replication between nodes
|
146
|
-
- Bootstrap nodes for network discovery
|
147
|
-
|
148
|
-
## Security Considerations
|
149
|
-
|
150
|
-
- This is a basic implementation and should not be used in production without additional security measures
|
151
|
-
- Consider adding:
|
152
|
-
- Encryption for data in transit
|
153
|
-
- Authentication for node joining
|
154
|
-
- Rate limiting to prevent abuse
|
155
|
-
- Input validation
|
156
|
-
|
157
|
-
## Contributing
|
158
|
-
|
159
|
-
1. Fork the repository
|
160
|
-
2. Create a feature branch
|
161
|
-
3. Commit your changes
|
162
|
-
4. Push to the branch
|
163
|
-
5. Create a Pull Request
|
164
|
-
|
165
|
-
## License
|
166
|
-
|
167
|
-
This project is licensed under the MIT License - see the LICENSE file for details.
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|