ari-build.com
robots.txt

Robots Exclusion Standard data for ari-build.com

Resource Scan

Scan Details

Site Domain ari-build.com
Base Domain ari-build.com
Scan Status Ok
Last Scan2024-10-02T01:30:43+00:00
Next Scan 2024-11-01T01:30:43+00:00

Last Scan

Scanned2024-10-02T01:30:43+00:00
URL https://ari-build.com/robots.txt
Domain IPs 104.17.76.117, 104.17.77.117
Response IP 104.17.76.117
Found Yes
Hash 8269b37f0fe71da9074bc342cc10de0f3c1ac7d524152184c98a9167eeff5c7a
SimHash 434cca45e6ba

Groups

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

adsbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

amazonbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 50

yandex

Rule Path
Disallow /

baiduspider
baiduspider-video
baiduspider-image

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

audioeye

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow

Other Records

Field Value
sitemap https://ari-build.com/sitemap.xml

Comments

  • 08022017: (http) - ARI Network Services
  • Block MegaIndex.ru
  • SEO Star
  • Slow Amazonbot
  • Block YandexBot
  • Block Baidu
  • Block Youdao
  • Block Majestic
  • Slow Rogerbot (Moz's crawler)
  • Slow AudioEye
  • Slow BingBot