Log In Sign Up

business-wissen.de
robots.txt

Robots Exclusion Standard data for business-wissen.de

Resource Scan

Scan Details

Site Domain business-wissen.de
Base Domain business-wissen.de
Scan Status Ok
Last Scan2024-05-08T06:13:10+00:00
Next Scan 2024-05-15T06:13:10+00:00

Last Scan

Scanned2024-05-08T06:13:10+00:00
URL https://business-wissen.de/robots.txt
Redirect https://www.business-wissen.de/robots.txt
Redirect Domain www.business-wissen.de
Redirect Base business-wissen.de
Domain IPs 82.165.100.56
Redirect IPs 82.165.100.56
Response IP 82.165.100.56
Found Yes
Hash 6730449201e7ba59e25c1efb793280c9716b1fc96ecd6b5531f431c282640755
SimHash c70cf8608d1a

Groups

nutch

Rule Path
Disallow /

spiderbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

iccrawler - icjobs

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

semantic-visions.com crawler

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

xovionpagecrawler

Rule Path
Disallow /

*

Rule Path
Disallow /img/
Disallow /css/
Disallow /pdf/
Disallow /premium/
Disallow /produktdownload/
Disallow /suche/
Disallow /shop/suche/
Back to top

Other Records

Field Value
sitemap https://www.business-wissen.de/sitemap/index.xml
Back to top

Comments

  • Sistrix
  • User-agent: 007ac9
  • Disallow: /
  • 80legs ausschließen
  • 27.10.2012
  • Crawl-delay: 5
Back to top

Warnings

  • 4 invalid lines.
Back to top