university-directory.eu
robots.txt

Robots Exclusion Standard data for university-directory.eu

Resource Scan

Scan Details

Site Domain university-directory.eu
Base Domain university-directory.eu
Scan Status Ok
Last Scan2024-06-15T23:06:35+00:00
Next Scan 2024-06-22T23:06:35+00:00

Last Scan

Scanned2024-06-15T23:06:35+00:00
URL https://university-directory.eu/robots.txt
Redirect https://www.university-directory.eu/robots.txt
Redirect Domain www.university-directory.eu
Redirect Base university-directory.eu
Domain IPs 104.26.8.65, 104.26.9.65, 172.67.74.45, 2606:4700:20::681a:841, 2606:4700:20::681a:941, 2606:4700:20::ac43:4a2d
Redirect IPs 104.26.8.65, 104.26.9.65, 172.67.74.45, 2606:4700:20::681a:841, 2606:4700:20::681a:941, 2606:4700:20::ac43:4a2d
Response IP 104.26.8.65
Found Yes
Hash 4ceff3cf2dd9916abc551df1087a2bf4708a7ebdc9e1a819d02828f74aca8206
SimHash 2366da53cb30

Groups

*
mediapartners-google

Rule Path
Disallow
Allow /
Disallow /ud/
Disallow /testudxd1/
Disallow /s23/
Disallow /js/Backups/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.university-directory.eu/sitemapindex.xml