httrack.com
robots.txt

Robots Exclusion Standard data for httrack.com

Resource Scan

Scan Details

Site Domain httrack.com
Base Domain httrack.com
Scan Status Ok
Last Scan2024-11-09T09:55:24+00:00
Next Scan 2024-11-16T09:55:24+00:00

Last Scan

Scanned2024-11-09T09:55:24+00:00
URL https://httrack.com/robots.txt
Redirect https://www.httrack.com/robots.txt
Redirect Domain www.httrack.com
Redirect Base httrack.com
Domain IPs 2001:bc8:30e2::1, 51.15.188.184
Redirect IPs 2001:bc8:30e2::2, 51.15.188.184
Response IP 51.15.188.184
Found Yes
Hash 321ea5b8e443b23c7960b4c7f9399823b6d4bf57e0d74075c8e65d4765c0fb62
SimHash 56498b418752

Groups

googlebot

Rule Path
Allow /
Allow /page
Allow /html
Allow /src
Disallow /*.zip$
Disallow /*.exe$
Disallow /*.tar.gz$
Disallow /*.deb$

*

Rule Path
Disallow

Comments

  • robots.txt for http://www.httrack.com