persfin.co.za
robots.txt
Robots Exclusion Standard data for persfin.co.za
Resource Scan
Scan Details
| Site Domain | persfin.co.za |
| Base Domain | persfin.co.za |
| Scan Status | Ok |
| Last Scan | 2024-06-10T12:25:02+00:00 |
| Next Scan | 2024-06-17T12:25:02+00:00 |
Last Scan
| Scanned | 2024-06-10T12:25:02+00:00 |
| URL | https://persfin.co.za/robots.txt |
| Domain IPs | 104.21.13.184, 172.67.133.12, 2606:4700:3033::6815:db8, 2606:4700:3036::ac43:850c |
| Response IP | 172.67.133.12 |
| Found | Yes |
| Hash | 5fce38735b9f3c5a24d079db6ed19cbfbf5406cf16d5193652af0762c7d1c279 |
| SimHash | c9f61c04fff1 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | *?z= |
| Disallow | *?q= |
| Disallow | */preview/ |
| Disallow | /menu/ |
| Disallow | /menu |
| Disallow | /menu?* |
| Disallow | /print-content* |
| Disallow | /widgets/rss_redirect.php?* |
| Disallow | /index.php?* |
| Disallow | /test$ |
| Disallow | /test1$ |
| Disallow | /test2$ |
| Disallow | /my-notifications |
| Disallow | /my-bookmarks |
| Disallow | /business-report/test$ |
| Disallow | /news/test$ |
| Disallow | /personal-finance/test$ |
| Disallow | /sport/test$ |
| Disallow | /entertainment/test$ |
| Disallow | /lifestyle/test$ |
| Disallow | /travel/test$ |
| Disallow | /motoring/test$ |
| Disallow | /wap.iol.co.za/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.iol.co.za/sitemap.xml |