wble-sl.utar.edu.my
robots.txt

Robots Exclusion Standard data for wble-sl.utar.edu.my

Resource Scan

Scan Details

Site Domain wble-sl.utar.edu.my
Base Domain utar.edu.my
Scan Status Ok
Last Scan2024-11-03T09:23:15+00:00
Next Scan 2024-12-03T09:23:15+00:00

Last Scan

Scanned2024-11-03T09:23:15+00:00
URL https://wble-sl.utar.edu.my/robots.txt
Domain IPs 121.123.29.113
Response IP 121.123.29.113
Found Yes
Hash 13c59f11f1697a38e00d613125aeff94904082f265cc9638e351cca06dc49cc7
SimHash 241dfb6101f3

Groups

googlebot
google
bingbot
bing
yahoo! slurp
yahoo! slurp china
baiduspider
baiduspider-image
sogou spider
yandexbot
yandexbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 15

*

Rule Path
Disallow /

Warnings

  • 1 invalid line.