abo-frosch.de
robots.txt

Robots Exclusion Standard data for abo-frosch.de

Resource Scan

Scan Details

Site Domain abo-frosch.de
Base Domain abo-frosch.de
Scan Status Ok
Last Scan2024-11-05T02:37:42+00:00
Next Scan 2024-12-05T02:37:42+00:00

Last Scan

Scanned2024-11-05T02:37:42+00:00
URL https://abo-frosch.de/robots.txt
Redirect https://www.abo-frosch.de/robots.txt
Redirect Domain www.abo-frosch.de
Redirect Base abo-frosch.de
Domain IPs 217.160.0.153
Redirect IPs 217.160.0.153
Response IP 217.160.0.153
Found Yes
Hash 10cc25c94f471895936004d6a4b5bf4394e54cf88a53ba5c89c155446e42d320
SimHash f154c7b24c13

Groups

*

Rule Path
Disallow /sweet_wrapper.php
Disallow /impressum.html
Disallow /datenschutz.html
Disallow /jump.php
Disallow /anbieter_linkout.php
Disallow /social_go.php
Disallow /werbung.html
Disallow /ls_gs/
Disallow /out/
Disallow /leserservice-gutschein-aktion.html

adsbot-google

Rule Path
Allow /

almaden

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

discobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

envolk

Rule Path
Disallow /

eurobot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

geniebot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

gigabot/2.0

Rule Path
Disallow /

gonzo1

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

iccrawler - icjobs

Rule Path
Disallow /

lexxebot

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

mozilla/5.0 (compatible; yahoo! slurp china; http://misc.yahoo.com.cn/help.html)

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

oceli

Rule Path
Disallow /

psbot

Rule Path
Disallow /

rufusbot

Rule Path
Disallow /

ruky-roboter

Rule Path
Disallow /

sbider

Rule Path
Disallow /

seekbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semager

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

teoma

Rule Path
Disallow /

thesubot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

vedensbot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

voyager

Rule Path
Disallow /

webalta

Rule Path
Disallow /

yahoo! slurp china

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yeti

Rule Path
Disallow /

zeusbot

Rule Path
Disallow /