Log In Sign Up

dom-zu-speyer.de
robots.txt

Robots Exclusion Standard data for dom-zu-speyer.de

Resource Scan

Scan Details

Site Domain dom-zu-speyer.de
Base Domain dom-zu-speyer.de
Scan Status Ok
Last Scan2025-08-26T19:24:10+00:00
Next Scan 2025-09-25T19:24:10+00:00

Last Scan

Scanned2025-08-26T19:24:10+00:00
URL https://www.dom-zu-speyer.de/robots.txt
Domain IPs 217.151.147.70
Response IP 217.151.147.70
Found Yes
Hash ece5bfee751c5e81e4875e9b1387c8c79cd9b3fefed696d9649ba56c3bc1adf9
SimHash b33746c80fa5

Groups

*

Rule Path
Disallow /typo3/
Disallow /typo3_src/
Disallow /t3lib/
Disallow /typo3conf/
Disallow *?print=*

Other Records

Field Value
crawl-delay 30

bingbot

Rule Path
Disallow /typo3/
Disallow /typo3_src/
Disallow /t3lib/
Disallow /typo3conf/

Other Records

Field Value
crawl-delay 40

amazonbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

newsnow

Rule Path
Disallow /

bloodhound

Rule Path
Disallow /

cydralspider

Rule Path
Disallow /

downloadexpress

Rule Path
Disallow /

gammaspider

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

pimptrain

Rule Path
Disallow /

raven

Rule Path
Disallow /

wapspider

Rule Path
Disallow /

webzinger

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

fidget-spinner-bot

Rule Path
Disallow /
Back to top

Comments

  • robots.txt updated 2023-11-04
  • Legal notice: this domain expressly reserves the right to use its content for commercial text and data mining (§ 44b UrhG).
  • The use of robots or other automated means to access this domain or collect or mine data without the express permission of this domain is strictly prohibited.
  • If you would like to apply for permission to crawl this domain, collect or use data, please contact pressestelle@bistum-speyer.de
Back to top

Warnings

  • `dissalow` is not a known field.
Back to top