Log In Sign Up

indies.eu
robots.txt

Robots Exclusion Standard data for indies.eu

Resource Scan

Scan Details

Site Domain indies.eu
Base Domain indies.eu
Scan Status Ok
Last Scan2024-10-25T18:25:32+00:00
Next Scan 2024-11-24T18:25:32+00:00

Last Scan

Scanned2024-10-25T18:25:32+00:00
URL https://www.indies.eu/robots.txt
Domain IPs 89.185.244.42
Response IP 89.185.244.42
Found Yes
Hash 2e7e77fde65837ddc7e11940cc9313558d2c232b29ef936c362d764e43709220
SimHash b9dded43cd9f

Groups

*

Rule Path
Disallow /vyhledavani/
Disallow /en/vyhledavani/
Disallow /en/label/vyhledavani/
Disallow /en/agentura/vyhledavani/
Disallow /en/studio/vyhledavani/
Disallow /us/vyhledavani/
Disallow /us/label/vyhledavani/
Disallow /us/agentura/vyhledavani/
Disallow /us/studio/vyhledavani/
Disallow /moje/
Disallow /en/moje/
Disallow /en/label/moje/
Disallow /en/agentura/moje/
Disallow /en/studio/moje/
Disallow /us/moje/
Disallow /us/label/moje/
Disallow /us/agentura/moje/
Disallow /us/studio/moje/
Disallow /download-zdarma/
Disallow /en/download-zdarma/
Disallow /en/label/download-zdarma/
Disallow /en/agentura/download-zdarma/
Disallow /en/studio/download-zdarma/
Disallow /us/download-zdarma/
Disallow /us/label/download-zdarma/
Disallow /us/agentura/download-zdarma/
Disallow /us/studio/download-zdarma/
Disallow /download/
Disallow /en/download/
Disallow /en/label/download/
Disallow /en/agentura/download/
Disallow /en/studio/download/
Disallow /us/download/
Disallow /us/label/download/
Disallow /us/agentura/download/
Disallow /us/studio/download/
Disallow /newsletter/
Disallow /en/newsletter/
Disallow /en/label/newsletter/
Disallow /en/agentura/newsletter/
Disallow /en/studio/newsletter/
Disallow /us/newsletter/
Disallow /us/label/newsletter/
Disallow /us/agentura/newsletter/
Disallow /us/studio/newsletter/
Disallow /cron/
Disallow /admin/
Disallow /datastore/
Disallow /data/alba/mp3/
Disallow /data/historicke-okenko/
Disallow /data/knihovna-medii/
Disallow /data/objednane-balicky/
Disallow /support/sendy/

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

webzip

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

web downloader

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

offline explorer pro

Rule Path
Disallow /

httrack website copier

Rule Path
Disallow /

offline commander

Rule Path
Disallow /

leech

Rule Path
Disallow /

websnake

Rule Path
Disallow /

blackwidow

Rule Path
Disallow /

http weazel

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://www.indies.eu/cache/sitemap-index.xml
Back to top

Comments

  • Request-rate: 30/1h
  • Crawl-delay: 60
  • vyhledavani
  • moje
  • download-zdarma
  • download
  • newsletter
  • globalni
  • prevent website from being ripped
  • we like you, but we suffer from this because server is overloaded then
  • $a = array('label','agentura','studio');
  • $a2 = array('en', 'us');
  • $t = 'vyhledavani';
  • echo "# $t<br>";
  • echo "Disallow: /$t/<br>";
  • foreach ($a2 as $two) {
  • echo "Disallow: /$two/$t/<br>";
  • foreach ($a as $one) {
  • echo "Disallow: /$two/$one/$t/<br>";
  • }
  • }
Back to top