xneelo.co.za
robots.txt

Robots Exclusion Standard data for xneelo.co.za

Resource Scan

Scan Details

Site Domain xneelo.co.za
Base Domain xneelo.co.za
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-04-20T14:39:01+00:00
Next Scan 2024-06-19T14:39:01+00:00

Last Successful Scan

Scanned2024-01-29T13:14:58+00:00
URL https://xneelo.co.za/robots.txt
Domain IPs 104.26.14.197, 104.26.15.197, 172.67.72.76, 2606:4700:20::681a:ec5, 2606:4700:20::681a:fc5, 2606:4700:20::ac43:484c
Response IP 104.26.15.197
Found Yes
Hash 2543d203f2f18b5fb6d6dd7385ca37e012cc0f4d964be118e55405e2a96e2208
SimHash 6001d802c3b2

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://xneelo.co.za/sitemap_index.xml