Log In Sign Up

heb.com.mx
robots.txt

Robots Exclusion Standard data for heb.com.mx

Resource Scan

Scan Details

Site Domain heb.com.mx
Base Domain heb.com.mx
Scan Status Ok
Last Scan2024-05-16T14:56:57+00:00
Next Scan 2024-06-15T14:56:57+00:00

Last Scan

Scanned2024-05-16T14:56:57+00:00
URL https://heb.com.mx/robots.txt
Redirect https://www.heb.com.mx/robots.txt
Redirect Domain www.heb.com.mx
Redirect Base heb.com.mx
Domain IPs 207.248.80.96
Redirect IPs 13.33.88.33, 13.33.88.40, 13.33.88.57, 13.33.88.95, 2600:9000:223b:2000:d:e8e3:8580:93a1, 2600:9000:223b:2600:d:e8e3:8580:93a1, 2600:9000:223b:2800:d:e8e3:8580:93a1, 2600:9000:223b:3400:d:e8e3:8580:93a1, 2600:9000:223b:3e00:d:e8e3:8580:93a1, 2600:9000:223b:5400:d:e8e3:8580:93a1, 2600:9000:223b:8e00:d:e8e3:8580:93a1, 2600:9000:223b:9a00:d:e8e3:8580:93a1
Response IP 13.33.88.33
Found Yes
Hash 9b815898fe69679badc281ec3807c91ab487ae7676389dc6afe18f63f9cc923c
SimHash 12db111e0c73

Groups

etaospider

Rule Path
Disallow /

dtsearchspider

Rule Path
Disallow /

twengabot-2.0

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

expertsearchspider

Rule Path
Disallow /

jobdiggerspider

Rule Path
Disallow /

joc\ web\spider

Rule Path
Disallow /

kingspider

Rule Path
Disallow /

netspider

Rule Path
Disallow /

openindexspider

Rule Path
Disallow /

openwebspider

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

vikspider

Rule Path
Disallow /

commoncrawler

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

crawlera

Rule Path
Disallow /

crazywebcrawler

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.heb.com.mx/sitemap.xml
Back to top

Comments

  • Version 02.10.2023
  • Heb.com.mx
  • Bloquea herramientas Crawlers/Comparadores de precios
  • Bloqueo de EtaoSpider
  • Más info en http://open.etao.com/dev/EtaoSpider
  • Bloqueo de dtSearchSpider
  • Más info en https://www.dtsearch.com/intro-web-spider.html
  • http://www.twenga.com/bot.html
  • Otros Crawlers extraidos de la blacklist
  • Otros motores de búsqueda
  • Bloqueo de YandexBot (Principal motor de búsqueda en Rusia)
  • Bloqueo de Baiudu (Principal motor de búsqueda en China)
  • http://help.baidu.com/question?prod_en=master&class=Baiduspider
  • Bloqueo de Nandex (Principal motor de búsqueda en Corea del Sur)
  • Bloqueo para herramientas SEO
  • Bloqueo de sistrix
  • Bloqueo de Ahrefs
  • Bloqueo de SemRush
  • Bloqueo de SEOProfilerbot
  • Bloqueo de Screaming Frog
  • Este no garantiza un bloqueo absoluto pero sí limita a los novatos
  • Bloqueo de Majestic
  • Más info en http://www.majestic12.co.uk
Back to top

Warnings

  • 2 invalid lines.
Back to top