ctanujit.org
robots.txt

Robots Exclusion Standard data for ctanujit.org

Resource Scan

Scan Details

Site Domain ctanujit.org
Base Domain ctanujit.org
Scan Status Ok
Last Scan2025-11-30T05:27:07+00:00
Next Scan 2025-12-30T05:27:07+00:00

Last Scan

Scanned2025-11-30T05:27:07+00:00
URL https://ctanujit.org/robots.txt
Redirect https://www.ctanujit.org/robots.txt
Redirect Domain www.ctanujit.org
Redirect Base ctanujit.org
Domain IPs 199.34.228.147
Redirect IPs 199.34.228.147
Response IP 199.34.228.147
Found Yes
Hash 9d555c6a6ec2e209efe7aa4d8fd4d461c94e28745662c99f299927102241b416
SimHash 814dd4128f92

Groups

nerdybot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /ajax/
Disallow /apps/
Disallow /talkpdfs.html
Disallow /workshops.html
Disallow /publications.html
Disallow /research.html
Disallow /library.html
Disallow /foreml-lab1.html
Disallow /spotlight.html

Other Records

Field Value
sitemap https://www.ctanujit.org/sitemap.xml