id-dom 0.0.4 → 0.0.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/robots.txt ADDED
@@ -0,0 +1,79 @@
1
+ # WATT3D — AI scraping policy
2
+ #
3
+ # This file enumerates known AI / LLM training and inference crawlers
4
+ # that the WATT3D AI Training Rider does not consent to. See
5
+ # ./ADDITIONAL_TERMS.md for the legal scope and definitions.
6
+ #
7
+ # Honoring robots.txt is voluntary. The rider applies whether or not
8
+ # a bot respects this file. The enumeration is non-exhaustive — bots
9
+ # not listed here are still bound by the rider if they perform
10
+ # Training Use as defined in ADDITIONAL_TERMS.md §1.
11
+
12
+ User-agent: GPTBot
13
+ Disallow: /
14
+
15
+ User-agent: ChatGPT-User
16
+ Disallow: /
17
+
18
+ User-agent: OAI-SearchBot
19
+ Disallow: /
20
+
21
+ User-agent: anthropic-ai
22
+ Disallow: /
23
+
24
+ User-agent: ClaudeBot
25
+ Disallow: /
26
+
27
+ User-agent: Claude-Web
28
+ Disallow: /
29
+
30
+ User-agent: Claude-SearchBot
31
+ Disallow: /
32
+
33
+ User-agent: Google-Extended
34
+ Disallow: /
35
+
36
+ User-agent: Applebot-Extended
37
+ Disallow: /
38
+
39
+ User-agent: Meta-ExternalAgent
40
+ Disallow: /
41
+
42
+ User-agent: Meta-ExternalFetcher
43
+ Disallow: /
44
+
45
+ User-agent: FacebookBot
46
+ Disallow: /
47
+
48
+ User-agent: CCBot
49
+ Disallow: /
50
+
51
+ User-agent: PerplexityBot
52
+ Disallow: /
53
+
54
+ User-agent: Perplexity-User
55
+ Disallow: /
56
+
57
+ User-agent: cohere-ai
58
+ Disallow: /
59
+
60
+ User-agent: Bytespider
61
+ Disallow: /
62
+
63
+ User-agent: Amazonbot
64
+ Disallow: /
65
+
66
+ User-agent: Omgilibot
67
+ Disallow: /
68
+
69
+ User-agent: AI2Bot
70
+ Disallow: /
71
+
72
+ User-agent: Diffbot
73
+ Disallow: /
74
+
75
+ # Everyone else: full access. Search engines and human-driven crawlers
76
+ # remain welcome.
77
+
78
+ User-agent: *
79
+ Allow: /