stuvia.co.za
robots.txt
Robots Exclusion Standard data for stuvia.co.za
Resource Scan
Scan Details
| Site Domain | stuvia.co.za |
| Base Domain | stuvia.co.za |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-06-12T09:09:57+00:00 |
| Next Scan | 2024-07-12T09:09:57+00:00 |
Last Successful Scan
| Scanned | 2024-04-21T09:00:01+00:00 |
| URL | https://stuvia.co.za/robots.txt |
| Redirect | https://www.stuvia.co.za:443/robots.txt |
| Redirect Domain | www.stuvia.co.za |
| Redirect Base | stuvia.co.za |
| Domain IPs | 34.253.67.111, 52.49.220.22, 54.76.184.28 |
| Redirect IPs | 13.225.4.110, 13.225.4.2, 13.225.4.69, 13.225.4.76 |
| Response IP | 13.225.4.110 |
| Found | Yes |
| Hash | d3d320d20467d999a8704eb21ae2d1efe03439d1509214caa376b5664d981f97 |
| SimHash | 1a889dafdad6 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /view |
| Disallow | /mijnprofiel |
| Disallow | /myprofile |
| Disallow | /cross-domain-login |
| Disallow | /auto-login |
| Disallow | /authenticate/ |
| Disallow | /download |
| Disallow | /downloads |
| Disallow | /uploads |
| Disallow | /en-nl/ |
| Disallow | /en-be/ |
| Disallow | /en-es/ |
| Disallow | /en-fr/ |
| Disallow | /en-de/ |
| Disallow | /nl-nl/ajax-mvc/update-document-type-filter* |
| Disallow | /nl-be/ajax-mvc/update-document-type-filter* |
| Disallow | /en-gb/ajax-mvc/update-document-type-filter* |
| Disallow | /en-za/ajax-mvc/update-document-type-filter* |
| Disallow | /en-us/ajax-mvc/update-document-type-filter* |
| Disallow | /es-es/ajax-mvc/update-document-type-filter* |
| Disallow | /fr-fr/ajax-mvc/update-document-type-filter* |
| Disallow | /de-de/ajax-mvc/update-document-type-filter* |
| Disallow | /ajax-get-institution-dropdown-content* |
| Disallow | /nl-nl/ajax-get-institution-dropdown-content* |
| Disallow | /nl-be/ajax-get-institution-dropdown-content* |
| Disallow | /en-gb/ajax-get-institution-dropdown-content* |
| Disallow | /en-za/ajax-get-institution-dropdown-content* |
| Disallow | /en-us/ajax-get-institution-dropdown-content* |
| Disallow | /es-es/ajax-get-institution-dropdown-content* |
| Disallow | /fr-fr/ajax-get-institution-dropdown-content* |
| Disallow | /de-de/ajax-get-institution-dropdown-content* |
| Disallow | /ajax-mvc* |
| Disallow | /nl-nl/ajax-mvc* |
| Disallow | /nl-be/ajax-mvc* |
| Disallow | /en-gb/ajax-mvc* |
| Disallow | /en-za/ajax-mvc* |
| Disallow | /en-us/ajax-mvc* |
| Disallow | /es-es/ajax-mvc* |
| Disallow | /fr-fr/ajax-mvc* |
| Disallow | /de-de/ajax-mvc* |
| Disallow | /live/views/content/document/documentPreview.php* |
googlebot-image
| Rule | Path |
|---|---|
| Disallow | /nl-nl/doc/*/*.jp* |
| Disallow | /nl-be/doc/*/*.jp* |
| Disallow | /en-gb/doc/*/*.jp* |
| Disallow | /en-us/doc/*/*.jp* |
| Disallow | /en-za/doc/*/*.jp* |
| Disallow | /fr-fr/doc/*/*.jp* |
| Disallow | /es-es/doc/*/*.jp* |
| Disallow | /de-de/doc/*/*.jp* |
| Disallow | /nl-nl/bundle/*/*.jp* |
| Disallow | /nl-be/bundle/*/*.jp* |
| Disallow | /en-gb/bundle/*/*.jp* |
| Disallow | /en-us/bundle/*/*.jp* |
| Disallow | /en-za/bundle/*/*.jp* |
| Disallow | /fr-fr/bundle/*/*.jp* |
| Disallow | /es-es/bundle/*/*.jp* |
| Disallow | /de-de/bundle/*/*.jp* |
| Disallow | /nl-nl/doc/*/*.png |
| Disallow | /nl-be/doc/*/*.png |
| Disallow | /en-gb/doc/*/*.png |
| Disallow | /en-us/doc/*/*.png |
| Disallow | /en-za/doc/*/*.png |
| Disallow | /fr-fr/doc/*/*.png |
| Disallow | /es-es/doc/*/*.png |
| Disallow | /de-de/doc/*/*.png |
| Disallow | /nl-nl/bundle/*/*.png |
| Disallow | /nl-be/bundle/*/*.png |
| Disallow | /en-gb/bundle/*/*.png |
| Disallow | /en-us/bundle/*/*.png |
| Disallow | /en-za/bundle/*/*.png |
| Disallow | /fr-fr/bundle/*/*.png |
| Disallow | /es-es/bundle/*/*.png |
| Disallow | /de-de/bundle/*/*.png |
| Disallow | /doc/*/*.jp* |
| Disallow | /doc/*/*.png |
| Disallow | /bundle/*/*.jp* |
| Disallow | /bundle/*/*.png |
| Disallow | /ajax-mvc/user-log* |
| Disallow | /nl-nl/ajax-mvc/user-log* |
| Disallow | /nl-be/ajax-mvc/user-log* |
| Disallow | /en-gb/ajax-mvc/user-log* |
| Disallow | /en-za/ajax-mvc/user-log* |
| Disallow | /en-us/ajax-mvc/user-log* |
| Disallow | /es-es/ajax-mvc/user-log* |
| Disallow | /fr-fr/ajax-mvc/user-log* |
| Disallow | /de-de/ajax-mvc/user-log* |
Comments