Log In Sign Up

kalaman-nas.com
robots.txt

Robots Exclusion Standard data for kalaman-nas.com

Resource Scan

Scan Details

Site Domain kalaman-nas.com
Base Domain kalaman-nas.com
Scan Status Ok
Last Scan2025-03-09T15:58:05+00:00
Next Scan 2025-03-16T15:58:05+00:00

Last Scan

Scanned2025-03-09T15:58:05+00:00
URL https://kalaman-nas.com/robots.txt
Domain IPs 179.61.189.131, 2a02:4780:39:a52a:6d9f:6499:3acb:47ec
Response IP 77.37.75.206
Found Yes
Hash 05326109acf7440b9436a126ec52d2000a94dfd78767de93a58b0829b8d775bf
SimHash 713585db8ef4

Groups

*

Rule Path
Allow /

*

Rule Path
Disallow /cgi-bin/
Disallow /tmp/
Disallow /app/
Disallow /app/webroot/img/upload/*.jpg$
Disallow /img/upload/*.jpg$
Disallow /img/
Disallow /gallery/

*

Rule Path
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/
Disallow /trackback/
Disallow /index.php
Disallow /xmlrpc.com
Disallow /wp-login.php
Disallow /wp-content/plugins/
Disallow /comments/feed/

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

yandex

Rule Path
Disallow /

proximic

Rule Path
Disallow /php/
Back to top

Comments

  • Blocks robots from specific folders / directories
  • AVOID SPAM AND AVOID BEING PLAGIARIZED!!!
  • NO GOOD BOTS !!! Needs to be blocked !!!
  • ahrefs.com
  • http://ahrefs.com/robot IP 173.199.115.163 - 173.199.115.163.choopa.net - WARNING !!!
  • Mozilla/5.0 (compatible; AhrefsBot/4.0; +http://ahrefs.com/robot/)
  • One of the worst bots on the net!!! Ahrefs.com take advantage of YOU, and you get absolutely NOTHING in return.
  • Ahrefs slows down your site and make you loose visitors !!!
  • Since ahrefs.com don't care about your robots.txt for all of their bots you need to block the whole IP-range (in .htaccess);
  • Ahrefs.com Network Range: IP 173.199.64.0 - 173.199.127.255. CIDR: 173.199.64.0/18
  • See Bots vs Browswers
  • SEOPROFILER.COM
  • IP 107.21.197.234 (ec2-107-21-197-234.compute-1.amazonaws.com)
  • Mozilla/5.0 (compatible; spbot/3.0; +http://www.seoprofiler.com/bot )
  • Mozilla/5.0 (compatible; spbot/3.1; +http://www.seoprofiler.com/bot )
  • IP-adresses starts with;
  • 23.20.*.*
  • 23.21.*.*
  • 23.22.*.*
  • 23.23.*.*
  • 50.16.*.*
  • 50.17.*.*
  • 50.19.*.*
  • 54.242.*.*
  • 67.202.*.*
  • 72.44.*.*
  • 75.101.*.*
  • 107.20.*.*
  • 107.21.*.*
  • 107.22.*.*
  • 174.129.*.*
  • 184.72.*.*
  • 184.73.*.*
  • 204.236.*.*
  • ezooms.com - One of the absolute must to block in every way you can from spying on you !!!
  • IP 208.115.113.82 Ezooms.com Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com)
  • Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com)
  • 208.115.111.66 208.115.111.67 208.115.111.68 208.115.111.70 208.115.111.71 208.115.111.74 208.115.111.75
  • IP-range: 208.115.96.0 - 208.115.127.255 (they don't give out bot name!). The CIDR is 208.115.111.64/28
  • wowrack dot com says that ezooms.com IP belongs to one of their clients; dotnetdotcom.org and that their main purpose for this machine is to crawl/index the content just like google bot.
  • The spider from ezooms.com visits robots.txt frequently but ignore the rules written in robots.txt.
  • Therefore the only way to stop this secret spider is to block the IP-range.
  • One of the theories is that the spider belongs to http://www.seomoz.org/ (anagram for ezooms) who tries to hide their bot in this way.
  • The email they give out is fake, just as their web site obviously is !!!
  • Ezooms is a parasite and they are definitely up to no good !!!
  • sistrix (IP 5.9.112.64 - 5.9.112.95)
  • Yandex bot - A rule breaker, just as Baidu spiders
  • proximic.com/info/spider.php
  • Amazonaws.com - watch out for all IP's coming from amazonasws.com. There are hundreds and hundreds of bad bots! Block everything coming from this part of the Internet !!! AWS is a myriad of site scrapers or bad bots (eg dozens of twitter/facebook me-too bots) that have no obvious use and no definite IP to block.
  • All IP's from AWS;
  • 216.182.224.0/20 (216.182.224.0 - 216.182.239.255)
  • 72.44.32.0/19 (72.44.32.0 - 72.44.63.255)
  • 67.202.0.0/18 (67.202.0.0 - 67.202.63.255)
  • 75.101.128.0/17 (75.101.128.0 - 75.101.255.255)
  • 174.129.0.0/16 (174.129.0.0 - 174.129.255.255)
  • 204.236.192.0/18 (204.236.192.0 - 204.236.255.255) [previously 204.236.224.0/19]
  • US West (Northern California):
  • 204.236.128.0/18 (216.236.128.0 - 216.236.191.255)
  • EU (Ireland):
  • 79.125.0.0/17 (79.125.0.0 - 79.125.127.255)
  • In .htaccess write;
  • Deny from 216.182.224.0/20
  • Deny from 72.44.32.0/19
  • Deny from 67.202.0.0/18
  • Deny from 75.101.128.0/17
  • Deny from 174.129.0.0/16
  • Deny from 204.236.192.0/18
  • Deny from 79.125.0.0/17
  • Deny from 184.72.0.0/18
  • Deny from 184.73.0.0/16
  • Deny from 175.41.128.0/18
  • Deny from 184.72.128.0/17
  • Deny from 204.236.128.0/18
  • Deny from 184.72.64.0/18 (184.72.64.0 - 184.72.127.255)
  • Deny from 50.16.0.0/15 (50.16.0.0 - 50.17.255.255)
  • Deny from 50.19.0.0/16 (50.19.0.0 - 50.19.255.255)
  • Deny from 50.18.0.0/17 (50.18.0.0 - 50.18.127.255)
  • Deny from 46.51.128.0/18 (46.51.128.0 - 46.51.191.255)
  • Deny from 46.51.192.0/20 (46.51.192.0 - 46.51.207.255)
  • Deny from 46.137.0.0/17 (46.137.0.0 - 46.137.127.255)
  • Deny from 175.41.128.0/18 (175.41.128.0 - 175.41.191.255)
  • Deny from 122.248.192.0/18 (122.248.192.0 - 122.248.255.255)
  • Deny from 175.41.192.0/18 (175.41.192.0 - 175.41.255.255)
  • Deny from 46.51.224.0/19 (46.51.224.0 - 46.51.255.254)
  • List of spam IP's used to spam forums (recently) on Internet.
  • Inachive.com
  • http://www.robotstxt.org
Back to top

Warnings

  • 1 invalid line.
Back to top