fh-mainz.de
robots.txt

Robots Exclusion Standard data for fh-mainz.de

Resource Scan

Scan Details

Site Domain fh-mainz.de
Base Domain fh-mainz.de
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-09-12T21:18:30+00:00
Next Scan 2024-12-11T21:18:30+00:00

Last Successful Scan

Scanned2021-12-15T17:15:53+00:00
URL http://fh-mainz.de/robots.txt
Redirect https://www.hs-mainz.de/robots.txt
Redirect Domain www.hs-mainz.de
Redirect Base hs-mainz.de
Response IP 143.93.114.43
Found Yes
Hash 0122a2b9aa4bd49a73011a20e4de8c2b8e3e7e967feac1aac0ed0a58a8f2fa22
SimHash 6b1803044cea

Groups

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

majesticseo

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

search17

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

slysearch

Rule Path
Disallow /

findlinks

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

lb-spider

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

ec2linkfinder

Rule Path
Disallow /

htdig

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

edisterbot

Rule Path
Disallow /

swebot

Rule Path
Disallow /

picmole

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yeti-mobile

Rule Path
Disallow /

pagepeeker

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

netestatenecrawler

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

comodosslchecker

Rule Path
Disallow /

comodo-certificates-spider

Rule Path
Disallow /

gonzo

Rule Path
Disallow /

schrein

Rule Path
Disallow /

afiliaswebminingtool

Rule Path
Disallow /

suggybot

Rule Path
Disallow /

bdbrandprotect

Rule Path
Disallow /

bpimagewalker

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

lex

Rule Path
Disallow /

contentcrawler

Rule Path
Disallow /

dcpbot

Rule Path
Disallow /

kaloogabot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

icjobs

Rule Path
Disallow /

obot

Rule Path
Disallow /

webmastercoffee

Rule Path
Disallow /

qualidator

Rule Path
Disallow /

webinator

Rule Path
Disallow /

scooter

Rule Path
Disallow /

thunderstone

Rule Path
Disallow /

larbin

Rule Path
Disallow /

opidoobot

Rule Path
Disallow /

ips-agent

Rule Path
Disallow /

tineye

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

unister

Rule Path
Disallow /

reverseget

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

*

Rule Path
Disallow /typo3/
Disallow /fileadmin/intern/
Disallow /uploads/
Disallow /beispiel-inhalte/
Disallow /interne-seiten/
Disallow /en/internal-pages/
Disallow /online/
Disallow /online-tutorial/
Disallow /*?id=*
Disallow /*%26id%3D*
Disallow /*/Private/*
Disallow /*/Configuration/*
Disallow /typo3temp/
Disallow *.sql
Disallow *.sql.gz
Allow /typo3temp/*.css
Allow /typo3temp/*.css.*.gzip
Allow /typo3temp/*.js
Allow /typo3temp/*.js.*.gzip
Allow /typo3temp/*.jpg
Allow /typo3temp/*.gif
Allow /typo3temp/*.png

Other Records

Field Value
sitemap https://www.hs-mainz.de/sitemap.xml

Warnings

  • 2 invalid lines.