bidfood.nl
robots.txt
Robots Exclusion Standard data for bidfood.nl
Resource Scan
Scan Details
| Site Domain | bidfood.nl |
| Base Domain | bidfood.nl |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-10-22T00:32:55+00:00 |
| Next Scan | 2024-11-21T00:32:55+00:00 |
Last Successful Scan
| Scanned | 2024-08-31T00:21:00+00:00 |
| URL | https://bidfood.nl/robots.txt |
| Redirect | https://www.bidfood.nl/robots.txt |
| Redirect Domain | www.bidfood.nl |
| Redirect Base | bidfood.nl |
| Domain IPs | 145.222.46.129 |
| Redirect IPs | 145.222.46.129 |
| Response IP | 145.222.46.129 |
| Found | Yes |
| Hash | 16ece81c4c76c40cfedb900d05250eef444f5cc6f6f2e2c69a5b2a08f7fa9f5a |
| SimHash | e51954628c93 |
Groups
yahoo! slurp
| Rule | Path |
|---|---|
| Disallow | /shoppingCart/ |
| Disallow | /checkout/ |
| Disallow | /myaccount/ |
| Disallow | /purchaseList/ |
| Disallow | /specialpages/ |
| Disallow | /*jsessionid%3D* |
| Disallow | /*Ntt%3D* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
*
| Rule | Path |
|---|---|
| Disallow | /shoppingCart/ |
| Disallow | /checkout/ |
| Disallow | /myaccount/ |
| Disallow | /purchaseList/ |
| Disallow | /specialpages/ |
| Disallow | /*jsessionid%3D* |
| Disallow | /*Ntt%3D* |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.bidfood.nl/sitemap.xml |