www.courses.teach.ucdavis.edu
robots.txt

Robots Exclusion Standard data for www.courses.teach.ucdavis.edu

Resource Scan

Scan Details

Site Domain www.courses.teach.ucdavis.edu
Base Domain ucdavis.edu
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-05-03T03:33:46+00:00
Next Scan 2024-08-01T03:33:46+00:00

Last Successful Scan

Scanned2022-06-11T19:49:38+00:00
URL https://www.courses.teach.ucdavis.edu/robots.txt
Response IP 23.185.0.4
Found Yes
Hash a5c793f2a8c2b1b68501d193d2bc5740f423b76356fdc0bec822b6874f2a220a
SimHash 8018d70266a5

Groups

*

Rule Path
Allow /

ravencrawler
rogerbot
dotbot
semrushbot
semrushbot-sa
powermapper
swiftbot

Rule Path
Allow /

Comments

  • Pantheon's documentation on robots.txt: https://pantheon.io/docs/bots-and-indexing/