iol.co.za
robots.txt
Robots Exclusion Standard data for iol.co.za
Resource Scan
Scan Details
| Site Domain | iol.co.za |
| Base Domain | iol.co.za |
| Scan Status | Ok |
| Last Scan | 2024-11-02T16:53:31+00:00 |
| Next Scan | 2024-11-09T16:53:31+00:00 |
Last Scan
| Scanned | 2024-11-02T16:53:31+00:00 |
| URL | https://iol.co.za/robots.txt |
| Redirect | https://www.iol.co.za/robots.txt |
| Redirect Domain | www.iol.co.za |
| Redirect Base | iol.co.za |
| Domain IPs | 104.22.54.115, 104.22.55.115, 172.67.4.195, 2606:4700:10::6816:3673, 2606:4700:10::6816:3773, 2606:4700:10::ac43:4c3 |
| Redirect IPs | 104.22.54.115, 104.22.55.115, 172.67.4.195, 2606:4700:10::6816:3673, 2606:4700:10::6816:3773, 2606:4700:10::ac43:4c3 |
| Response IP | 104.22.54.115 |
| Found | Yes |
| Hash | 5fce38735b9f3c5a24d079db6ed19cbfbf5406cf16d5193652af0762c7d1c279 |
| SimHash | c9f61c04fff1 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | *?z= |
| Disallow | *?q= |
| Disallow | */preview/ |
| Disallow | /menu/ |
| Disallow | /menu |
| Disallow | /menu?* |
| Disallow | /print-content* |
| Disallow | /widgets/rss_redirect.php?* |
| Disallow | /index.php?* |
| Disallow | /test$ |
| Disallow | /test1$ |
| Disallow | /test2$ |
| Disallow | /my-notifications |
| Disallow | /my-bookmarks |
| Disallow | /business-report/test$ |
| Disallow | /news/test$ |
| Disallow | /personal-finance/test$ |
| Disallow | /sport/test$ |
| Disallow | /entertainment/test$ |
| Disallow | /lifestyle/test$ |
| Disallow | /travel/test$ |
| Disallow | /motoring/test$ |
| Disallow | /wap.iol.co.za/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.iol.co.za/sitemap.xml |