bitcompare.net
robots.txt
Robots Exclusion Standard data for bitcompare.net
Resource Scan
Scan Details
| Site Domain | bitcompare.net |
| Base Domain | bitcompare.net |
| Scan Status | Ok |
| Last Scan | 2024-11-05T10:50:07+00:00 |
| Next Scan | 2024-12-05T10:50:07+00:00 |
Last Scan
| Scanned | 2024-11-05T10:50:07+00:00 |
| URL | https://bitcompare.net/robots.txt |
| Domain IPs | 76.76.21.241, 76.76.21.93 |
| Response IP | 76.76.21.61 |
| Found | Yes |
| Hash | 603dd516cbe473d9a0e3853dabcdaa545e949c1d7e637bb1dc42d54b42fb44d1 |
| SimHash | cc248e728370 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /admin |
| Disallow | /click |
| Disallow | /go |
| Disallow | /widgets |
| Disallow | /unavailable |
| Disallow | /blocked |
| Disallow | /blocked/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://bitcompare.net/sitemap.xml |
| sitemap | https://bitcompare.net/server-sitemap.xml |
| sitemap | https://bitcompare.net/sitemap-bytes.xml |
Warnings
- `host` is not a known field.
Comments