kremchik.ua
robots.txt

Robots Exclusion Standard data for kremchik.ua

Resource Scan

Scan Details

Site Domain kremchik.ua
Base Domain kremchik.ua
Scan Status Ok
Last Scan2024-06-03T10:22:11+00:00
Next Scan 2024-06-10T10:22:11+00:00

Last Scan

Scanned2024-06-03T10:22:11+00:00
URL https://kremchik.ua/robots.txt
Domain IPs 104.21.63.172, 172.67.148.212, 2606:4700:3032::6815:3fac, 2606:4700:3037::ac43:94d4
Response IP 104.21.63.172
Found Yes
Hash e7ca9af7fa275c7baaa5e48f14ae2ab1466924d6aa1a6bf0140f4871748ebf1b
SimHash c1ae4c6a225c

Groups

*

Rule Path
Disallow /account/*.html%26many
Disallow /account/*.html%26friends
Disallow /account/*.html%26frendinlist
Disallow /account/*.html%26zakazi
Disallow /account/*.html%26tracking_threads
Disallow /account/*.html%26user_external_link
Disallow /account/*.html%26usermailadd
Disallow /item-*/pictures
Disallow /zakazstatus*
Disallow /user_edit
Disallow /order-make
Disallow /api_login/*
Disallow /mail/
Disallow /verify-email/*
Disallow /mail_id*.html
Disallow /customshop-*/customshopedit
Disallow /cache/
Disallow /cfg/
Disallow /configs/
Disallow /core_functions/
Disallow /examples/
Disallow /fck/
Disallow /includes/
Disallow /languages/
Disallow /libs/
Disallow /temp/
Disallow /temp_smarty/
Disallow /templates/special/
Disallow /templates/
Disallow /temp_ulogin/
Disallow /ajax/
Disallow /api/picture/
Disallow /index.php?verifyemail

petalbot
bloglovin
ahrefsbot
mj12bot
ccbot
detectify
dotbot
baiduspider
riddler
linkpadbot
blexbot
flipboardproxy
aihitbot
trovitbot
qwantify
bubing
cliqzbot
seekport
nutch
dataforseobot
barkrowler
megaindex
imagesiftbot
konturbot
mbcrawle
getintent
yak

Rule Path
Disallow /

Other Records

Field Value
sitemap https://kremchik.ua/sitemap.xml

Comments

  • User-Agent: Yandex
  • User-Agent: Googlebot
  • User-Agent: Mail.ru
  • User-Agent: StackRambler
  • Request-rate: 3/5
  • Visit-time: 0200-0845
  • Crawl-delay: 2
  • Disallow: /account/*.html&wishlist
  • Disallow: /account/*.html&picadduser
  • Disallow: /account/*.html&prudadduser
  • Disallow: /zakazstatus
  • Disallow: /*.php
  • User-Agent: StackRambler
  • User-agent: Pinterest
  • User-Agent: Yandex
  • User-Agent: METASpider
  • User-Agent: MSNBot
  • User-Agent: MSNBot-media