Log In Sign Up

hiper.az
robots.txt

Robots Exclusion Standard data for hiper.az

Resource Scan

Scan Details

Site Domain hiper.az
Base Domain hiper.az
Scan Status Ok
Last Scan2024-10-27T11:27:26+00:00
Next Scan 2024-11-26T11:27:26+00:00

Last Scan

Scanned2024-10-27T11:27:26+00:00
URL https://hiper.az/robots.txt
Domain IPs 104.21.86.133, 172.67.220.86, 2606:4700:3033::ac43:dc56, 2606:4700:3035::6815:5685
Response IP 172.67.220.86
Found Yes
Hash 0ca07570d97fa74637218b76eb02217482dfd7a1639966e8f6e2e8041d7160ab
SimHash 801d77400c31

Groups

*

Rule Path
Allow /
Disallow /login/
Disallow /compare/
Disallow /register/
Disallow /customer

adsbot-google

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

baidu

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ip-web-crawler.com

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

siteliner

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

nutch

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

spbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

wesee

Rule Path
Disallow /

yandexmarket

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

advbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ntentbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

deepcrawl

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.hiper.az/sitemaps/sitemap.xml
Back to top

Warnings

  • 2 invalid lines.
Back to top