thortspace.com
robots.txt

Robots Exclusion Standard data for thortspace.com

Resource Scan

Scan Details

Site Domain thortspace.com
Base Domain thortspace.com
Scan Status Ok
Last Scan2024-06-27T04:43:19+00:00
Next Scan 2024-07-04T04:43:19+00:00

Last Scan

Scanned2024-06-27T04:43:19+00:00
URL https://thortspace.com/robots.txt
Domain IPs 104.21.69.121, 172.67.208.4, 2606:4700:3032::6815:4579, 2606:4700:3035::ac43:d004
Response IP 104.21.69.121
Found Yes
Hash f3b9882741b2c8674a5964c09c0b20cbdbce2500d16ab640cd0bc24d2a03dc63
SimHash 1557d84042d4

Groups

googlebot
google
adsbot-google
bingbot
bing

Rule Path
Disallow /e/*
Disallow /f/*
Disallow /g/*
Disallow /forums/*
Disallow /forums/users/*

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /