screenhacker.com
robots.txt
Robots Exclusion Standard data for screenhacker.com
Resource Scan
Scan Details
Site Domain | screenhacker.com |
Base Domain | screenhacker.com |
Scan Status | Ok |
Last Scan | 2024-06-04T11:09:52+00:00 |
Next Scan | 2024-06-11T11:09:52+00:00 |
Last Scan
Scanned | 2024-06-04T11:09:52+00:00 |
URL | https://screenhacker.com/robots.txt |
Domain IPs | 104.21.31.103, 172.67.176.53, 2606:4700:3032::ac43:b035, 2606:4700:3035::6815:1f67 |
Response IP | 104.21.31.103 |
Found | Yes |
Hash | b974a5df07179006a0d8cbbd089bae33ffb2df1ff14d0a2bdff978490d7d16ef |
SimHash | 247fb957cfd9 |
Groups
*
Rule | Path |
---|---|
Disallow |
*
Product | Comment |
---|---|
* | directed to all spiders, not just Scooter |
Rule | Path |
---|---|
Disallow | /cgi-bin/ |
Disallow | /screensite/wp-admin/ |
Disallow | /screensite/wp-includes/ |
Other Records
Field | Value |
---|---|
crawl-delay | 300 |
Other Records
Field | Value |
---|---|
sitemap | http://www.ScreenHacker.com/sitemap_index.xml |
Comments