bookworld.com.au
robots.txt
Robots Exclusion Standard data for bookworld.com.au
Resource Scan
Scan Details
Site Domain | bookworld.com.au |
Base Domain | bookworld.com.au |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Couldn't connect to server. |
Last Scan | 2024-05-26T10:50:31+00:00 |
Next Scan | 2024-06-09T10:50:31+00:00 |
Last Successful Scan
Scanned | 2024-04-18T10:49:42+00:00 |
URL | https://bookworld.com.au/robots.txt |
Redirect | https://www.angusrobertson.com.au/robots.txt |
Redirect Domain | www.angusrobertson.com.au |
Redirect Base | angusrobertson.com.au |
Domain IPs | 13.227.254.108, 13.227.254.112, 13.227.254.127, 13.227.254.13 |
Redirect IPs | 13.33.33.124, 13.33.33.28, 13.33.33.55, 13.33.33.76, 2600:9000:229f:2600:e:823d:cb80:93a1, 2600:9000:229f:4e00:e:823d:cb80:93a1, 2600:9000:229f:5200:e:823d:cb80:93a1, 2600:9000:229f:6400:e:823d:cb80:93a1, 2600:9000:229f:7600:e:823d:cb80:93a1, 2600:9000:229f:8a00:e:823d:cb80:93a1, 2600:9000:229f:ae00:e:823d:cb80:93a1, 2600:9000:229f:c00:e:823d:cb80:93a1 |
Response IP | 13.33.33.124 |
Found | Yes |
Hash | 6d6879ff2d7ea72a2f73cf71066e239dac3eb8337ccff3ba3c84e870d12ee1db |
SimHash | 3e44dc9ef8fb |
Groups
*
Rule | Path |
---|---|
Disallow | /cart |
Disallow | /checkout |
Disallow | /my-account |
Disallow | /logout |
Disallow | /*q%3D* |
Disallow | /*gclid%3D* |
Disallow | /*source%3D* |
Disallow | /*utm_medium%3D* |
Disallow | /*utm_source%3D* |
Disallow | /*utm_campaign%3D* |
Disallow | /*attachment_id%3D* |
Disallow | /*clickid%3D* |
Disallow | /*src%3D* |
Disallow | /*sc_src%3D* |
Disallow | /*sc_customer%3D* |
Disallow | /*sc_lid%3D* |
Disallow | /*sc_llid%3D* |
Disallow | /*sc_uid%3D* |
Disallow | /*utm_content%3D* |
Disallow | /*irclickid%3D* |
Disallow | /*zsrc%3D* |
Disallow | /*voucherCode%3D* |
Disallow | /*msclkid%3D* |
Disallow | /*addedToWishlist%3D* |
Disallow | /*uiel%3D* |
Disallow | /*sort%3D* |
Disallow | /*token%3D* |
Disallow | /*orderNumber%3D* |
Disallow | /*fbclid%3D* |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
Other Records
Field | Value |
---|---|
sitemap | https://www.angusrobertson.com.au/sitemap-index.xml |
Warnings
- `request-rate` is not a known field.
- `visit-time` is not a known field.
Comments