howtoshopforfree.net
robots.txt
Robots Exclusion Standard data for howtoshopforfree.net
Resource Scan
Scan Details
| Site Domain | howtoshopforfree.net |
| Base Domain | howtoshopforfree.net |
| Scan Status | Ok |
| Last Scan | 2024-11-08T19:03:58+00:00 |
| Next Scan | 2024-11-15T19:03:58+00:00 |
Last Scan
| Scanned | 2024-11-08T19:03:58+00:00 |
| URL | https://howtoshopforfree.net/robots.txt |
| Domain IPs | 104.21.18.152, 172.67.182.160, 2606:4700:3030::6815:1298, 2606:4700:3030::ac43:b6a0 |
| Response IP | 104.21.18.152 |
| Found | Yes |
| Hash | 02175129ae54a1952c52fbc63d1ad872f482b19dfd0d19280fff85f2c79e1a80 |
| SimHash | 6300dd4476f0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /images/ |
| Disallow | /forums/ |
| Disallow | /htsff_forums/ |
| Disallow | /cgi-bin/ |
| Disallow | /formatable/ |
| Disallow | /dump/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 60 |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.howtoshopforfree.net/sitemap.xml |
| sitemap | http://www.howtoshopforfree.net//sitemap.xml |
| sitemap | https://www.howtoshopforfree.net/sitemap_4.xml |
| sitemap | https://www.howtoshopforfree.net/sitemap_1.xml |
| sitemap | https://www.howtoshopforfree.net/sitemap_5.xml |
| sitemap | https://www.howtoshopforfree.net/sitemap_2.xml |
| sitemap | https://www.howtoshopforfree.net/sitemap_3.xml |