darwinbox.io
robots.txt
Robots Exclusion Standard data for darwinbox.io
Resource Scan
Scan Details
| Site Domain | darwinbox.io |
| Base Domain | darwinbox.io |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-11-17T05:27:18+00:00 |
| Next Scan | 2024-12-01T05:27:18+00:00 |
Last Successful Scan
| Scanned | 2022-10-31T23:28:37+00:00 |
| URL | http://darwinbox.io/robots.txt |
| Redirect | https://darwinbox.com/robots.txt |
| Redirect Domain | darwinbox.com |
| Redirect Base | darwinbox.com |
| Response IP | 3.109.39.237 |
| Found | Yes |
| Hash | ee90d4912cbaa4b32201a7e8fa7901f6bcafc38f79521b9396347cb8139fdef4 |
| SimHash | 7945ddaaa1b3 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /wp-login.php |
| Disallow | /blog* |
| Disallow | /readme.html |
| Disallow | /*?utm |
| Disallow | /*.pdf$ |
| Disallow | /?s= |
| Disallow | /search/ |
| Disallow | /sub-processors.pdf |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| sitemap | https://darwinbox.com/sitemap_index.xml |
| sitemap | https://darwinbox.com/page-sitemap.xml |