Log In Sign Up

art-holst.com.ua
robots.txt

Robots Exclusion Standard data for art-holst.com.ua

Resource Scan

Scan Details

Site Domain art-holst.com.ua
Base Domain art-holst.com.ua
Scan Status Ok
Last Scan2024-05-12T05:56:49+00:00
Next Scan 2024-06-11T05:56:49+00:00

Last Scan

Scanned2024-05-12T05:56:49+00:00
URL https://art-holst.com.ua/robots.txt
Domain IPs 104.26.4.179, 104.26.5.179, 172.67.71.47, 2606:4700:20::681a:4b3, 2606:4700:20::681a:5b3, 2606:4700:20::ac43:472f
Response IP 104.26.4.179
Found Yes
Hash e0717b39e058a892819a1012bb2708583bfa4043880193a13abd11d1ed0d8a2a
SimHash 605cfdd7ff81

Groups

*

Rule Path
Disallow /order/*

Other Records

Field Value
crawl-delay 1

adsbot-google

Rule Path
Disallow

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

uptimerobotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dotbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /
Disallow /

vebot

Rule Path
Disallow /

barkrowler/0.9

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

gowikibot

Rule Path
Disallow /

gowikibot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

lumtelbot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://art-holst.com.ua/sitemap-auto/sitemap.xml
Back to top

Comments

  • http://www.opensiteexplorer.org/dotbot
  • https://www.semrush.com/bot/
  • https://www.seokicks.de/robot.htm
  • http://webmeup-crawler.com/
  • http://ltx71.com/
  • https://velen.io/
  • https://www.ve.com/vebot
  • https://babbar.tech/crawler
  • https://www.zoominfo.com/
  • http://www.gowikibot.com/
  • https://cliqz.com/en/cliqzbot
  • https://lumtel.com/
  • https://suite.seozoom.it/bot.html
Back to top

Warnings

  • `useragent` is not a known field.
Back to top