cj.dev6.rt.gw
robots.txt

Robots Exclusion Standard data for cj.dev6.rt.gw

Resource Scan

Scan Details

Site Domain cj.dev6.rt.gw
Base Domain rt.gw
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-07-20T20:09:15+00:00
Next Scan 2025-10-18T20:09:15+00:00

Last Successful Scan

Scanned2024-09-01T04:36:55+00:00
URL http://cj.dev6.rt.gw/robots.txt
Redirect https://akash-pract.rt.gw/robots.txt
Redirect Domain akash-pract.rt.gw
Redirect Base rt.gw
Domain IPs 65.108.76.106
Redirect IPs 65.108.76.106
Response IP 65.108.76.106
Found Yes
Hash c54c15de0881a3ca74883256e3c1ca62c90d4c23a49325537e9c36a2f687ed7c
SimHash b852dd546bf3

Groups

*

Rule Path
Disallow /

Comments

  • Crawling is blocked for non-production environment