ntu.edu.sg
robots.txt

Robots Exclusion Standard data for ntu.edu.sg

Resource Scan

Scan Details

Site Domain ntu.edu.sg
Base Domain ntu.edu.sg
Scan Status Ok
Last Scan2024-10-26T05:34:35+00:00
Next Scan 2024-11-25T05:34:35+00:00

Last Scan

Scanned2024-10-26T05:34:35+00:00
URL https://ntu.edu.sg/robots.txt
Redirect https://www.ntu.edu.sg/robots.txt
Redirect Domain www.ntu.edu.sg
Redirect Base ntu.edu.sg
Domain IPs 104.16.4.14
Redirect IPs 104.16.4.14
Response IP 104.16.4.14
Found Yes
Hash 76c356cccf5bf8668cfe6b30f31e0fc62b1097df070b4d2e47989638e91a71a3
SimHash 6c107b226137

Groups

*

Rule Path
Disallow /CustomWebForms
Disallow /Sitefinity/*

Comments

  • www.robotstxt.org/
  • Allow crawling of all content