joshimilestoner.com
robots.txt
Robots Exclusion Standard data for joshimilestoner.com
Resource Scan
Scan Details
| Site Domain | joshimilestoner.com |
| Base Domain | joshimilestoner.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-11-21T16:32:44+00:00 |
| Next Scan | 2025-12-21T16:32:44+00:00 |
Last Successful Scan
| Scanned | 2025-10-22T08:40:58+00:00 |
| URL | https://joshimilestoner.com/robots.txt |
| Domain IPs | 191.101.228.176, 2a02:4780:84:5f4a:e9a6:20ee:4fdf:ab43, 2a02:4780:84:d933:a1f4:9441:3bcc:557e, 93.127.196.167 |
| Response IP | 93.127.201.44 |
| Found | Yes |
| Hash | e79538a9186e3f74d37613e583a25ccbd0d9f943924e412bb19b14f06c0de719 |
| SimHash | 630b096344b1 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-content/plugins/ |
| Disallow | /wp-content/cache/ |
| Disallow | /wp-content/themes |
| Disallow | /xmlrpc.php |
| Disallow | */feed/ |
| Disallow | /feed/ |
| Disallow | /*/feed/ |
| Disallow | *?s=* |
| Disallow | /category/*/feed/ |
| Disallow | /amp/ |
| Allow | /wp-content/uploads/ |
| Allow | /wp-content/uploads/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://joshimilestoner.com/sitemap_index.xml |