literature-map.com
robots.txt

Robots Exclusion Standard data for literature-map.com

Resource Scan

Scan Details

Site Domain literature-map.com
Base Domain literature-map.com
Scan Status Ok
Last Scan2024-11-01T10:19:30+00:00
Next Scan 2024-11-08T10:19:30+00:00

Last Scan

Scanned2024-11-01T10:19:30+00:00
URL https://literature-map.com/robots.txt
Redirect https://www.literature-map.com/robots.txt
Redirect Domain www.literature-map.com
Redirect Base literature-map.com
Domain IPs 78.47.51.37
Redirect IPs 18.155.68.17, 18.155.68.25, 18.155.68.33, 18.155.68.49
Response IP 18.155.68.17
Found Yes
Hash a37e50e58b1e50384bbc0e0c830ff3d87362def27cdbeb13bd84211e2455ebc2
SimHash 9014d4207a13

Groups

anthropic-ai

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /