pavo.ucsd.edu
robots.txt

Robots Exclusion Standard data for pavo.ucsd.edu

Resource Scan

Scan Details

Site Domain pavo.ucsd.edu
Base Domain ucsd.edu
Scan Status Ok
Last Scan2025-08-31T12:48:36+00:00
Next Scan 2025-09-30T12:48:36+00:00

Last Scan

Scanned2025-08-31T12:48:36+00:00
URL https://pavo.ucsd.edu/robots.txt
Domain IPs 132.239.26.31
Response IP 132.239.26.31
Found Yes
Hash be4d375b495cf058e7d8227bc1f31eddfc376848315b3392ecf1749462dbe6e1
SimHash 3a1dd860eb10

Groups

amazonbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

semrushbot
semrushbot-ba

Rule Path
Disallow /

yandex
yandexbot

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

academicbotrtu

Rule Path
Disallow /

*

Rule Path
Disallow