Log In Sign Up

venturecapital.20m.com
robots.txt

Robots Exclusion Standard data for venturecapital.20m.com

Resource Scan

Scan Details

Site Domain venturecapital.20m.com
Base Domain 20m.com
Scan Status Ok
Last Scan2024-09-18T22:27:04+00:00
Next Scan 2024-10-18T22:27:04+00:00

Last Scan

Scanned2024-09-18T22:27:04+00:00
URL http://venturecapital.20m.com/robots.txt
Domain IPs 64.136.20.60
Response IP 64.136.20.60
Found Yes
Hash e6100af15b8edec3f825ea9e9a640bd648000bee1b6de3c8e32f0b35c5fade5d
SimHash f3121533c5d6

Groups

googlebot

Rule Path
Disallow

*

Rule Path
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cgi/wp/
Disallow /cgi-bin/util/cgi_access/
Disallow /cgi-bin/util/fm/
Disallow /cgi-bin/util/site_admin/
Disallow /cgi-bin/util/mail_admin/
Disallow /cgi-bin/mail/
Disallow /cgi-bin/login/
Disallow /cgi-bin/util/upgrade/
Disallow /cgi-bin/forgot/
Disallow /404.html

mediapartners-google*

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

bingbot

Rule Path
Disallow
Disallow /cgi/
Disallow /cgi-bin/
Disallow /cgi/wp/
Disallow /cgi-bin/util/cgi_access/
Disallow /cgi-bin/util/fm/
Disallow /cgi-bin/util/site_admin/
Disallow /cgi-bin/util/mail_admin/
Disallow /cgi-bin/mail/
Disallow /cgi-bin/login/
Disallow /cgi-bin/util/upgrade/
Disallow /cgi-bin/forgot/

megaindex

Rule Path
Disallow

msnbot-media

Rule Path
Disallow

msnbot

Rule Path
Disallow

slurp

Rule Path
Disallow

ahrefsbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexantivirus

Rule Path
Disallow /

yandexblogs

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexpagechecker

Rule Path
Disallow /

yandexwebmaster

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

a6-indexer

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

gomezagent

Rule Path
Disallow /

netseer

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

webvac

Rule Path
Disallow /

wesee

Rule Path
Disallow /

wotbox

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://www.americafundinglending.com/sitemap.xml
sitemap http://www.americafundinglending.com/sitemap.html
sitemap http://www.americaonecapital.com/sitemap.xml
sitemap https://www.america1funding.com/sitemap.xml
sitemap https://bizfinancesite.weebly.com/sitemap.xml
sitemap https://quickcreditfunding.weebly.com/sitemap.xml
sitemap https://unsecuredpersonalloanfunding.weebly.com/sitemap.xml
sitemap https://unsecuredcreditfunding.weebly.com/sitemap.xml
sitemap https://unsecuredbadcreditpersonalloans.blogspot.com/feeds/posts/default
sitemap https://americaonecapital.blogspot.com/feeds/posts/default
sitemap https://unsecuredbusinessstartupcreditlines.blogspot.com/feeds/posts/default
sitemap https://america1funding.blogspot.com/feeds/posts/default
sitemap https://commercialcapitalfunding.blogspot.com/feeds/posts/default
sitemap https://hardmoneyloanfinancing.blogspot.com/feeds/posts/default
sitemap https://unsecuredbadcreditcashpersonalloans.blogspot.com/feeds/posts/default
sitemap https://startupbusinessunsecuredloan.blogspot.com/feeds/posts/default
sitemap https://personalstartupbusinessloan.blogspot.com/feeds/posts/default
sitemap https://unsecuredpersonalfunding.blogspot.com/feeds/posts/default
sitemap https://americafunding.blogspot.com/feeds/posts/default
sitemap https://bizfinancesite.blogspot.com/feeds/posts/default
sitemap https://quickcreditstartupfunding.blogspot.com/feeds/posts/default
sitemap https://americanunsecuredcapital.blogspot.com/feeds/posts/default
sitemap https://www.sbaguaranteedloans.com/feeds/posts/default
sitemap https://www.unsecuredstartupbusinessloans.com/feeds/posts/default
sitemap https://americafundinglending.wordpress.com/
sitemap https://southendcap.weebly.com/
sitemap https://www.pinterest.com/unsecured/
sitemap https://www.pinterest.com/americafundinglending/
sitemap https://startupventurecapitalloans.blogspot.com/feeds/posts/default
Back to top

Comments

  • Default /robots.txt File for all Community Architect Partner pages
  • Choopa.net
  • user-agent: AhrefsBot
  • Disallow: /
  • Crawl-delay: 3600
  • Choopa.net
  • user-agent: AhrefsBot/4.0
  • Disallow: /
  • Choopa.net
  • user-agent: AhrefsBot/3.1
  • Disallow: /
  • Choopa.net
  • user-agent: AhrefsBot/3.0
  • Disallow: /
  • Choopa.net
  • user-agent: AhrefsBot/2.0
  • Disallow: /
  • Choopa.net
  • user-agent: AhrefsBot/1.0
  • Disallow: /
  • Allow: /
  • User-agent: dotbot
  • Disallow: /
  • SEOPROFILER.COM
  • IP 107.21.197.234 (ec2-107-21-197-234.compute-1.amazonaws.com)
  • Mozilla/5.0 (compatible; spbot/3.0; +http://www.seoprofiler.com/bot )
  • Mozilla/5.0 (compatible; spbot/3.1; +http://www.seoprofiler.com/bot )
  • IP-adresses starts with;
  • 23.20.*.*
  • 23.21.*.*
  • 23.22.*.*
  • 23.23.*.*
  • 50.16.*.*
  • 50.17.*.*
  • 50.19.*.*
  • 54.242.*.*
  • 67.202.*.*
  • 72.44.*.*
  • 75.101.*.*
  • 107.20.*.*
  • 107.21.*.*
  • 107.22.*.*
  • 174.129.*.*
  • 184.72.*.*
  • 184.73.*.*
  • 204.236.*.*
  • ezooms.com - One of the absolute must to block in every way you can from spying on you !!!
  • IP 208.115.113.82 Ezooms.com Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com)
  • Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com)
  • 208.115.111.66 208.115.111.67 208.115.111.68 208.115.111.70 208.115.111.71 208.115.111.74 208.115.111.75
  • IP-range: 208.115.96.0 - 208.115.127.255 (they don't give out bot name!). The CIDR is 208.115.111.64/28
  • wowrack dot com says that ezooms.com IP belongs to one of their clients; dotnetdotcom.org and that their main purpose for this machine is to crawl/index the content just like google bot.
  • The spider from ezooms.com visits robots.txt frequently but ignore the rules written in robots.txt.
  • Therefore the only way to stop this secret spider is to block the IP-range.
  • One of the theories is that the spider belongs to http://www.seomoz.org/ (anagram for ezooms) who tries to hide their bot in this way.
  • The email they give out is fake, just as their web site obviously is !!!
  • Ezooms is a parasite and they are definitely up to no good !!!
  • sistrix (IP 5.9.112.64 - 5.9.112.95)
  • Yandex bot - A rule breaker, just as Baidu spiders
  • A6-Indexer
  • Spiders a lot but do not include in their index (wastes bandwidth)
  • User-agent: Baiduspider
  • Crawl-delay: 10
  • Is not a functioning search site
  • Does not respect robots.txt
  • https://www.facebook.com/externalhit_uatext.php
  • User-agent: facebookexternalhit
  • Crawl-delay: 5
  • No benefit
  • No benefit
  • No benefit
  • No benefit
Back to top

Warnings

  • 11 invalid lines.
Back to top