Log In Sign Up

zeuspetshop.com
robots.txt

Robots Exclusion Standard data for zeuspetshop.com

Resource Scan

Scan Details

Site Domain zeuspetshop.com
Base Domain zeuspetshop.com
Scan Status Ok
Last Scan2024-06-02T19:19:34+00:00
Next Scan 2024-07-02T19:19:34+00:00

Last Scan

Scanned2024-06-02T19:19:34+00:00
URL https://zeuspetshop.com/robots.txt
Domain IPs 2a02:24b8:11:2a9::100, 86.107.32.169
Response IP 86.107.32.169
Found Yes
Hash 524315fa5f0b0678c3d99e0a21a57f873e8efa4ca66d96325d395259807bd6f0
SimHash b3175382eab2

Groups

*

Rule Path
Disallow /cgi-bin/

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /wp-signup.php
Disallow /readme.html
Disallow /license.txt
Disallow /xmlrpc.php
Disallow /my-account/
Disallow /cart/
Disallow /checkout/
Allow /wp-admin/admin-ajax.php
Allow /wp-content/uploads/

googlebot-mobile

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

yandex
yandexturbo
yandexbot
yandexbot/3.0
yisouspider
petalbot
bytespider
sogou web spider
sogou inst spider
linespider
barkrowler

Rule Path
Disallow /

baiduspider
baiduspider-video
baiduspider-image

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

ahrefsbot
ahrefsbot/5.2
blexbot
dotbot
dotbot/1.1
screaming frog seo spider
screaming frog seo spider/2,50
mj12bot
semrushbot
semrushbot-ba
semrushbot-ct
semrushbot-bm
semrushbot-sa
semrushbot-swa
semrushbot-si
semrushbot-si/0.97
siteauditbot
splitsignalbot
proximic
rogerbot/1.0
yandexmetrika
yandexmetrika/2.0
xenu's
xenu's link sleuth 1.1c
sistrix
seokicks
seokicks-robot
yisouspider
qwantify
uptimebot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

ruby
python-requests/2.11.0
python-requests
libwww-perl/6.26
dotbot/1.1
mail.ru_bot/2.0
barkrowler/0.5.1 (experimenting / debugging - sorry for your logs ) http://www.exensa.com/crawl - admin@exensa.com -- based on bubing
alexibot
aqua_products
asterias
b2w/0.1
backdoorbot/1.0
becomebot
bloglovin
blowfish/1.0
bookmark search tool
bot
botalot
builtbottough
bullseye/1.0
bunnyslippers
calculon spider
cheesebot
cherrypicker
cherrypickerelite/1.0
cherrypickerse/1.0
coccoc
copernic
copyrightcheck
cosmos
crescent
crescent internet toolpak http ole control v.1.0
daum
dittospyder
dumbot
emailcollector
emailsiphon
emailwolf
enterprise_search
enterprise_search/1.0
erocrawler
es
exabot
extractorpro
ezooms
fairad client
fatbot
flaming attackbot
foobot
freefind
gaisbot
getright/4.2
grub
grub-client
harvest/1.5
hatena antenna
hloader
httplib
humanlinks
idg
infonavirobot
inoreader.com
iron33/1.0.2
jennybot
jetbot
jetbot/1.0
jikespider
kenjin spider
keyword density/0.9
larbin
lexibot
libweb/clshttp
linkextractorpro
linkscan/8.1a unix
linkwalker
lnspiderguy
lwp-trivial
lwp-trivial/1.34
mata hari
megaindex.ru
megaindex.ru/
megaindex.ru/2.0
microsoft url control
microsoft url control - 5.01.4511
microsoft url control - 6.00.8169
miixpc
miixpc/4.2
mister pix
mozilla/5.0 (compatible; megaindex.ru/2.0; +https://www.megaindex.ru/?tab=linkanalyze)
mozilla/5.0 (compatible; spbot/5.0.3; +http://openlinkprofiler.org/bot )
msiecrawler
naver
netants
netmechanic
nicerspro
nutch
offline explorer
omniexplorer_bot
openbot
openfind
openfind data gathere
optimizer
oracle ultra search
parser
paperlibot
pcore-http/v0.25.0
perman
php
propowerbot/2.14
prowebwalker
psbot
pu_in crawler
python-urllib
pyton-requests
qwantify
queryn metasearch
radiation retriever 1.1
repomonkey
repomonkey bait & tackle/v1.01
rma
searchpreview
seznambot
sitebot
sitesnagger
smtbot
sogou
sootle
sosospider
spankbot
spanner
spinn3r
stanford
stanford comp sci
suzuran
swiftbot
szukacz/1.4
teleport
teleportpro
telesoft
the intraformant
thenomad
tocrawl/urldispatcher
trident
true_robot
true_robot/1.0
turingos
updownerbot
url control
url_spider_pro
urly warning
vci
vci webviewer vci webviewer win32
voilabot
web image collector
webauto
webbandit
webbandit/3.50
webcopier
webenhancer
websauger
website quester
webster pro
webstripper
webvac
webzip
webzip/4.0
wget
wget/1.5.3
wget/1.6
willybot
wordpress
wordpress/4.7.2
wordpress/4.6.3
www-collector-e
yodaobot
yak
zeus
zeus 32297 webster pro v2.9 win32
zeus link scout

Rule Path
Disallow /

moget
moget/2.1
ichiro

Rule Path
Disallow /

naverbot
yeti

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.zeuspetshop.com/sitemap_index.xml
Back to top

Comments

  • By INGEMATIC Digital Agency www.ingematic.com
  • File robots.txt
  • Blocca tutti i file in queste cartelle
  • CMS
  • Blocca tutti i file in queste cartelle
  • Disallow: /wp-content/cache/
  • Ottimizzazioni SEO by INGEMATIC ***********
  • Modifiche del 29-03-2019
  • Allow: /wp-content/cache/autoptimize/css/*
  • Allow: /wp-content/cache/autoptimize/js/*
  • End SEO by INGEMATIC **********************
  • SEARCH ENGINE ALLOWED
  • SEARCH-ENGINE
  • User-agent: Slurp
  • SEO-TOOLS
  • https://it.semrush.com/bot/
  • SOCIAL
  • User-agent: Twitterbot
  • Disallow: /
  • OTHER - Last Update 16-05-2020
  • .-.
  • |o,o|
  • ,| _\=/_ .-""-.
  • ||/_/_\_\ /[] _ _\
  • |_/|(_)|\\ _|_o_LII|_
  • \._./// / | ==== | \
  • |\_/|"` |_| ==== |_|
  • |_|_| ||" || ||
  • |-|-| ||LI o ||
  • |_|_| ||'----'||
  • /_/ \_\ /__| |__\
  • sitemap
Back to top

Warnings

  • 3 invalid lines.
Back to top