datacentermap.com
robots.txt

Robots Exclusion Standard data for datacentermap.com

Resource Scan

Scan Details

Site Domain datacentermap.com
Base Domain datacentermap.com
Scan Status Ok
Last Scan2024-10-06T08:43:11+00:00
Next Scan 2024-10-13T08:43:11+00:00

Last Scan

Scanned2024-10-06T08:43:11+00:00
URL https://datacentermap.com/robots.txt
Redirect https://www.datacentermap.com/robots.txt
Redirect Domain www.datacentermap.com
Redirect Base datacentermap.com
Domain IPs 76.76.21.21
Redirect IPs 76.76.21.123, 76.76.21.164
Response IP 76.76.21.98
Found Yes
Hash 5b7770a6554334b4add475ff43d60b15b510fd427f3522de997077e2a2db398e
SimHash 18187342eda8

Groups

*

Rule Path
Disallow /ui/
Disallow /api/
Disallow /visit/
Disallow /as/

rogerbot
exabot
mj12bot
dotbot
gigabot
semrushbot
semrushbot-sa
semrushbot-ba
semrushbot-si
semrushbot-swa
semrushbot-ct
semrushbot-coub
splitsignalbot
friendlycrawler
imagesiftbot
barkrowler
blexbot
dataforseobot
censysinspect
expanse
internet-measurement
serpstatbot
ezoicbot
turnitinbot
senutobot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.datacentermap.com/sitemap.xml