Log In Sign Up

aphorismen.de
robots.txt

Robots Exclusion Standard data for aphorismen.de

Resource Scan

Scan Details

Site Domain aphorismen.de
Base Domain aphorismen.de
Scan Status Ok
Last Scan2024-11-12T23:53:01+00:00
Next Scan 2024-11-19T23:53:01+00:00

Last Scan

Scanned2024-11-12T23:53:01+00:00
URL https://aphorismen.de/robots.txt
Redirect https://www.aphorismen.de/robots.txt
Redirect Domain www.aphorismen.de
Redirect Base aphorismen.de
Domain IPs 104.26.14.187, 104.26.15.187, 172.67.68.190, 2606:4700:20::681a:ebb, 2606:4700:20::681a:fbb, 2606:4700:20::ac43:44be
Redirect IPs 104.26.14.187, 104.26.15.187, 172.67.68.190, 2606:4700:20::681a:ebb, 2606:4700:20::681a:fbb, 2606:4700:20::ac43:44be
Response IP 104.26.14.187
Found Yes
Hash cb8c902ea932e281ea9739c29e7397e51b4d2b6ef9c50e9598061de71c0de964
SimHash 1a9c5d10cd59

Groups

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

lcc

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

httrack 3.0

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

bubing

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

seekbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

semrushbot-seoab

Rule Path
Disallow /
Back to top

Comments

  • robots.txt
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/wc/robots.html
  • For syntax checking, see:
  • http://www.sxw.org.uk/computing/robots/check.html
  • Common Crawlers
  • Disallow: DownloadBot HTTrack zum offline Browsen ausschließen
Back to top