Log In Sign Up

apropoba.es
robots.txt

Robots Exclusion Standard data for apropoba.es

Resource Scan

Scan Details

Site Domain apropoba.es
Base Domain apropoba.es
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-05-01T23:46:45+00:00
Next Scan 2024-06-30T23:46:45+00:00

Last Successful Scan

Scanned2024-02-09T20:16:01+00:00
URL https://apropoba.es/robots.txt
Domain IPs 104.21.69.48, 172.67.204.89, 2606:4700:3037::6815:4530, 2606:4700:3037::ac43:cc59
Response IP 172.67.204.89
Found Yes
Hash 4c6ed1c8ad4cca54d81d3521d0ab618b089260089e428741a4dfc278f65917a8
SimHash b2e6c74245cb

Groups

admantx

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

twitterbot

Rule Path
Disallow

xenu's link sleuth 1.1c

Rule Path
Disallow /

xenu's

Rule Path
Disallow /

xenu link sleuth 1.2e

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

psbot

Rule Path
Disallow /

npbot-1/2.0

Rule Path
Disallow /

npbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

proximic

Rule Path
Disallow /php/

ahrefsbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

inarchive.com

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

spiderling

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

urlappendbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

nutch-1.4

Rule Path
Disallow /

lwnutch

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

xovibot/2.0

Rule Path
Disallow /

synapse

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

ecommercebot

Rule Path
Disallow /

ineturl

Rule Path
Disallow /

email exractor

Rule Path
Disallow /

trendictionbot0.5.0

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

bot/0.1

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

linkdexbot/2.0

Rule Path
Disallow /

kraken/0.1

Rule Path
Disallow /

ccbot/2.0

Rule Path
Disallow /

wget/1.1

Rule Path
Disallow /

analyticsseo

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

nett.io

Rule Path
Disallow /

riddler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /
Back to top

Comments

  • Majestic SEO
  • Yandex bot
  • proximic.com/info/spider.php
  • ahrefs.com/robot/
  • 2014-10-26 dotbot = ezooms = opensiteexplorer
  • 2013-06-28
  • 2013-12-22
  • 2014-01-21 Ignores this 2014-01-23!
  • 2014-02-11 nlp.fi.muni.cz/projects/biwec/ ignoreras
  • 2014-02-20 fulltextrobot-77-75-77-32.seznam.cz
  • 2014-03-14 (Gbook)
  • 2014-03-14 (Gbook) webmeup-crawler.com/
  • 2014-03-14
  • 2014-03-19
  • 2014-04-25 law.di.unimi.it/BUbiNG.html
  • 2014-06-09 help.coccoc.com/
  • 2014-06-10 linkdex.com/bots/
  • 2014-07-04 code.google.com/p/crawler4j/
  • 2014-07-04 suki.ling.helsinki.fi/eng/project.html heritrix/3.1.1
  • 2014-07-12 LWNutch/Nutch-1.4 (another scientific bot - we check your robots.txt!
  • 2014-10-22 LWNutch/Nutch-1.4 (another scientific bot - we check your robots.txt!
  • 2014-07-15 xovi.de/ xovi.com/ Block funkar EJ TROTS ATT DET UTLOVAS !
  • 2014-07-15 xovibot.net
  • 2014-07-16 Scammer Synapse: 46.36.139.209 Mozilla/4.0 (compatible; Block funkar EJ !
  • 2014-07-18 Mozilla/5.0 (compatible; IstellaBot/1.18.81 +tiscali.it/)
  • 2014-07-18 Platform Semantic Analyzer - ADmantX Inc. - admantx.com
  • 2014-09-15 NY
  • 2014-07-18 InetURL:/1.0 IP 47.22.0.142
  • 2014-07-18 IP 217.211.17.41
  • 2014-07-26 IP 144.76.23. Se ovan: trendictionbot (2014-03-14)
  • 2014-07-26 commoncrawl.org/faq/ Skiter helt i denna
  • 2014-07-26 BOT/0.1 (BOT for JCE)
  • 2014-07-26 radian6.com/crawler
  • 2014-07-26 linkdex.com/bots/
  • 2014-07-31 linkfluence.net 188.165.203.61 94.23.27.149 91.121.67.216 Struntas i helt
  • 2014-08-02
  • 2014-08-02
  • 2014-10-16 91.212.182.75 United Kingdom Rainham Vooservers Ltd
  • 2014-10-21 64.79.85.205 United States Columbus Xlhost.com Inc
  • 2014-10-23 5.9.97.180 Hetzner Online Ag
  • 2014-11-19 riddler.io/about
  • 2014-12-17 213.239.211.141 crawler.sistrix.net
  • sistrix (5.9.112.64 - 5.9.112.95)
  • Test
  • 2014-08-02 138.91.73.109 United States San Francisco Microsoft Corp Insitesbot/1.0
  • User-agent: Curious George
  • Disallow: /
Back to top

Warnings

  • 1 invalid line.
Back to top