be-one.nl
robots.txt
Robots Exclusion Standard data for be-one.nl
Resource Scan
Scan Details
| Site Domain | be-one.nl |
| Base Domain | be-one.nl |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2/27/2025, 2:32:26 AM |
| Next Scan | 5/28/2025, 2:32:26 AM |
Last Successful Scan
| Scanned | 7/10/2024, 2:29:36 AM |
| URL | https://www.be-one.nl/robots.txt |
| Domain IPs | 104.21.71.240, 172.67.172.97, 2606:4700:3033::6815:47f0, 2606:4700:3034::ac43:ac61 |
| Response IP | 104.21.71.240 |
| Found | Yes |
| Hash | 538d14d7370507aaca9306b9aeebeba9ecb572eb500891e9b95bb666d129d035 |
| SimHash | 81510208ad12 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /now |
| Disallow | /now/ |
| Disallow | /*?q=* |
| Disallow | /*?filteritem=* |
| Disallow | /*?price=* |
| Disallow | /*?vert=* |
| Disallow | /*?count=* |
| Disallow | /*?horiz=* |
| Disallow | /*q%3D* |
| Disallow | /*filteritem%3D* |
| Disallow | /*price%3D* |
| Disallow | /*vert%3D* |
| Disallow | /*count%3D* |
| Disallow | /*horiz%3D* |
| Disallow | /winkelwagen/ |
| Disallow | /zoeken/ |
| Disallow | /pages/testing/ |