boostbot.org
robots.txt
Robots Exclusion Standard data for boostbot.org
Resource Scan
Scan Details
| Site Domain | boostbot.org |
| Base Domain | boostbot.org |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Request timed out. |
| Last Scan | 2025-11-08T03:27:33+00:00 |
| Next Scan | 2026-01-07T03:27:33+00:00 |
Last Successful Scan
| Scanned | 2025-08-16T18:23:21+00:00 |
| URL | https://boostbot.org/robots.txt |
| Domain IPs | 104.21.18.28, 172.67.179.172, 2606:4700:3033::ac43:b3ac, 2606:4700:3036::6815:121c |
| Response IP | 104.21.18.28 |
| Found | Yes |
| Hash | 3414148462ae0b4e581596d2d644d65ad19880acbe7146e13f0961f23d304c0a |
| SimHash | eb818822cfb3 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /wp-content/uploads/wc-logs/ |
| Disallow | /wp-content/uploads/woocommerce_transient_files/ |
| Disallow | /wp-content/uploads/woocommerce_uploads/ |
| Disallow | /wp-admin/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| sitemap | https://boostbot.org/sitemap_index.xml |