investmentnetwork.co.za
robots.txt
Robots Exclusion Standard data for investmentnetwork.co.za
Resource Scan
Scan Details
| Site Domain | investmentnetwork.co.za |
| Base Domain | investmentnetwork.co.za |
| Scan Status | Ok |
| Last Scan | 2024-11-07T18:03:01+00:00 |
| Next Scan | 2024-11-21T18:03:01+00:00 |
Last Scan
| Scanned | 2024-11-07T18:03:01+00:00 |
| URL | https://investmentnetwork.co.za/robots.txt |
| Redirect | https://www.investmentnetwork.co.za/robots.txt |
| Redirect Domain | www.investmentnetwork.co.za |
| Redirect Base | investmentnetwork.co.za |
| Domain IPs | 13.41.47.86, 13.42.120.140, 18.133.23.123 |
| Redirect IPs | 13.41.47.86, 13.42.120.140, 18.133.23.123 |
| Response IP | 13.42.120.140 |
| Found | Yes |
| Hash | 6b9c44246283c47ec40b70f6873455328198df7e3e12f6167017184d1856af49 |
| SimHash | 72784030f0ee |
Groups
*
| Rule | Path |
|---|---|
| Disallow | |
| Disallow | /ads/* |
| Disallow | /cgi-bin/* |
| Disallow | /scripts/* |
| Disallow | /admin/* |
| Disallow | /tmp/* |
| Disallow | /tmp_/* |
| Disallow | /plugins/* |
| Disallow | /lib/* |
| Disallow | /config/* |
| Disallow | /proposal-tag/* |
| Disallow | /investor_sign_up/* |
| Disallow | /entrepreneur_sign_up/* |
| Disallow | /unsubscribe/* |
| Disallow | /investor/report/* |
| Disallow | /proposal/get_document/* |
| Disallow | /app_demo/* |
| Disallow | /app/* |
| Disallow | /appo/* |
| Disallow | /business-proposal/location* |
| Disallow | /business-proposal/tag* |
| Disallow | /business-proposal/country* |
| Disallow | /learning/ |
| Disallow | /overlay/* |
| Disallow | /users/forgot_password |
| Disallow | /users/login |
| Disallow | /affiliate/register |
| Disallow | /entrepreneur/register |
| Disallow | /investor/register |
| Disallow | /print/investor_legal |
| Disallow | /learn/ |
| Disallow | /learn |
| Disallow | /html/* |
| Disallow | /livlist/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Warnings
- 2 invalid lines.
Comments