mittelalter-lexikon.de
robots.txt

Robots Exclusion Standard data for mittelalter-lexikon.de

Resource Scan

Scan Details

Site Domain mittelalter-lexikon.de
Base Domain mittelalter-lexikon.de
Scan Status Ok
Last Scan2024-09-09T02:01:39+00:00
Next Scan 2024-10-09T02:01:39+00:00

Last Scan

Scanned2024-09-09T02:01:39+00:00
URL https://mittelalter-lexikon.de/robots.txt
Redirect https://www.mittelalter-lexikon.de/robots.txt
Redirect Domain www.mittelalter-lexikon.de
Redirect Base mittelalter-lexikon.de
Domain IPs 138.201.132.95
Redirect IPs 138.201.132.95
Response IP 138.201.132.95
Found Yes
Hash 69906d90a7159aa012e30ef3d7d837caa8ec35a381b6b6152497c975ebb2d59d
SimHash 984a1f80c6f3

Groups

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

blexbot/1.0

Rule Path
Disallow /

domainappender

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

domaincrawler/3.0

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

facebookexternalhit/1.1

Rule Path
Disallow /

facebot

Rule Path
Disallow /

fetch

Rule Path
Disallow /

httrack

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

linko

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

pixray

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

seoscanners.net/1

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webzip

Rule Path
Disallow /

xenu

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

*

Rule Path
Allow /w/api.php?action=mobileview&
Allow /w/load.php?
Allow /w/images/
Allow /w/index.php?title=Kategorie%3A
Allow /w/index.php?title=Kategoria%3A
Disallow /w/
Disallow /api/
Disallow /wiki/User%3A
Disallow /wiki/Benutzer%3A
Disallow /wiki/Benutzerin%3A
Disallow /wiki/User_talk%3A
Disallow /wiki/Benutzer_Diskussion%3A
Disallow /wiki/Benutzerin_Diskussion%3A
Disallow /wiki/MediaWiki%3A
Disallow /wiki/Vorlage%3A
Disallow /wiki/Template%3A
Disallow /wiki/Modul%3A
Disallow /wiki/Spezial%3A
Disallow /wiki/Spezial%3ASuche
Disallow /wiki/Special%3ASuche
Disallow /wiki/Spezial%3AZuf%C3%A4llige_Seite
Disallow /wiki/Spezial%3AZuf%C3%A4llige_Seite
Disallow /wiki/Special%3AZuf%C3%A4llige_Seite
Disallow /wiki/Special%3AZuf%C3%A4llige_Seite
Disallow /wiki/Spezial%3AVersionsgeschichte

Comments

  • robots.txt for https://marjorie-wiki.de/ and friends