appstorespy.com
robots.txt

Robots Exclusion Standard data for appstorespy.com

Resource Scan

Scan Details

Site Domain appstorespy.com
Base Domain appstorespy.com
Scan Status Ok
Last Scan2024-10-09T10:34:21+00:00
Next Scan 2024-10-16T10:34:21+00:00

Last Scan

Scanned2024-10-09T10:34:21+00:00
URL https://appstorespy.com/robots.txt
Domain IPs 104.21.22.93, 172.67.203.200, 2606:4700:3031::6815:165d, 2606:4700:3037::ac43:cbc8
Response IP 172.67.203.200
Found Yes
Hash 78ae952d43f77a1c93786608569fc9966a96c386dad2f985c76b4eaff98aaaa4
SimHash ca7ecd612dd0

Groups

*

Rule Path
Disallow /bar
Disallow /support
Disallow /privacy
Disallow /terms
Disallow /tos
Disallow /agreement
Disallow /signin
Disallow /sign_in
Disallow /signup
Disallow /accounts
Disallow /c/
Disallow /shoppingcart

yandex
ahrefsbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

openstat.ru/bot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

synthesio crawler release monalisa

Rule Path
Disallow /

sogou

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

deusu

Rule Path
Disallow /

getintent crawler

Rule Path
Disallow /

proximic

Rule Path
Disallow /

adbeat_bot

Rule Path
Disallow /

addthis.com

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

extlinksbot

Rule Path
Disallow /

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

grouphigh

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

ias_crawler

Rule Path
Disallow /

Warnings

  • 2 invalid lines.
  • `clean-param` is not a known field.