werkenbij.uva.nl
robots.txt

Robots Exclusion Standard data for werkenbij.uva.nl

Resource Scan

Scan Details

Site Domain werkenbij.uva.nl
Base Domain uva.nl
Scan Status Ok
Last Scan2025-07-08T04:06:35+00:00
Next Scan 2025-07-22T04:06:35+00:00

Last Scan

Scanned2025-07-08T04:06:35+00:00
URL https://werkenbij.uva.nl/robots.txt
Domain IPs 104.22.40.114, 104.22.41.114, 172.67.6.217, 2606:4700:10::6816:2872, 2606:4700:10::6816:2972, 2606:4700:10::ac43:6d9
Response IP 104.22.41.114
Found Yes
Hash 7fdc981725f1549fe5d083e2bec3151f6dc01299c2c00c0096ec28bf3e340955
SimHash 5225c4006236

Groups

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

magnetmebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

imagesiftbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

openindexspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

*

Rule Path
Disallow *?t=
Disallow *%26t%3D
Disallow *%26n%3D
Disallow *%26hh%3D
Disallow *%26hl%3D
Disallow *%26o%3D
Disallow *%26of%3D
Disallow *%26f%3D
Disallow *%26fo%3D
Disallow *%26ln%3D
Disallow *%26lc%3D
Disallow *%26lp%3D
Disallow *%26lct%3D
Disallow *%26s%3D
Disallow *%26location%3D
Disallow *%26keywords%3D
Disallow *%26spotlight%3D
Disallow */job-alert?email=

Other Records

Field Value
sitemap https://werkenbij.uva.nl/sitemap.xml