moneyweb.co.za
robots.txt
Robots Exclusion Standard data for moneyweb.co.za
Resource Scan
Scan Details
| Site Domain | moneyweb.co.za |
| Base Domain | moneyweb.co.za |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-10-27T04:02:03+00:00 |
| Next Scan | 2025-01-25T04:02:03+00:00 |
Last Successful Scan
| Scanned | 2024-04-01T02:36:58+00:00 |
| URL | https://moneyweb.co.za/robots.txt |
| Redirect | https://www.moneyweb.co.za/robots.txt |
| Redirect Domain | www.moneyweb.co.za |
| Redirect Base | moneyweb.co.za |
| Domain IPs | 104.26.2.107, 104.26.3.107, 172.67.74.36, 2606:4700:20::681a:26b, 2606:4700:20::681a:36b, 2606:4700:20::ac43:4a24 |
| Redirect IPs | 104.26.2.107, 104.26.3.107, 172.67.74.36, 2606:4700:20::681a:26b, 2606:4700:20::681a:36b, 2606:4700:20::ac43:4a24 |
| Response IP | 172.67.74.36 |
| Found | Yes |
| Hash | e262d908fceee6d204ac685ae5f9e306a65c2c5229541ef7577ebd5aefef227d |
| SimHash | 7018dd54cd32 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /wp-content/uploads/ftp/ |
| Disallow | /apple-app-site-association |
| Disallow | /search/ |
| Disallow | /wp-json/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.moneyweb.co.za/sitemaps/sitemaps_post.xml |
Comments