tahwil.net
robots.txt

Robots Exclusion Standard data for tahwil.net

Resource Scan

Scan Details

Site Domain tahwil.net
Base Domain tahwil.net
Scan Status Ok
Last Scan2025-06-25T21:10:00+00:00
Next Scan 2025-07-02T21:10:00+00:00

Last Scan

Scanned2025-06-25T21:10:00+00:00
URL https://tahwil.net/robots.txt
Redirect https://www.tahwil.net/robots.txt
Redirect Domain www.tahwil.net
Redirect Base tahwil.net
Domain IPs 3.214.108.168, 35.169.89.216
Redirect IPs 3.214.108.168, 35.169.89.216
Response IP 35.169.89.216
Found Yes
Hash 472def8972f9a870963cbfd0a50b543d12276f20d05dc5cfcd58e6b54615618d
SimHash c692da3ee61a

Groups

yandex

No rules defined. All paths allowed.

Other Records

Field Value Comment
crawl-delay 5 sets a 2 second time-out