cise.ufl.edu
robots.txt

Robots Exclusion Standard data for cise.ufl.edu

Resource Scan

Scan Details

Site Domain cise.ufl.edu
Base Domain ufl.edu
Scan Status Ok
Last Scan2025-08-23T01:52:40+00:00
Next Scan 2025-09-22T01:52:40+00:00

Last Scan

Scanned2025-08-23T01:52:40+00:00
URL https://cise.ufl.edu/robots.txt
Domain IPs 128.227.19.86
Response IP 128.227.19.86
Found Yes
Hash 8bd23df39d516123db134c9e136e2afea7d3af949834be83b846d9a3e7fb9933
SimHash 5010c940c592

Groups

anthropic-ai
applebot
bytespider
claude-web
ccbot
claudebot
chatgpt-user
diffbot
facebookbot
google-extended
gptbot
imagesiftbot
piplbot
perplexitybot
amazonbot
omgilibot
omgili
youbot

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Disallow /*s%3D
Disallow /search

siteimprovebot
siteimprovebot-crawler

Rule Path
Allow /