Log In Sign Up

rodavigo.net
robots.txt

Robots Exclusion Standard data for rodavigo.net

Resource Scan

Scan Details

Site Domain rodavigo.net
Base Domain rodavigo.net
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-07-20T02:41:54+00:00
Next Scan 2024-10-18T02:41:54+00:00

Last Successful Scan

Scanned2023-09-01T19:55:15+00:00
URL https://rodavigo.net/robots.txt
Domain IPs 176.58.9.195
Response IP 176.58.9.195
Found Yes
Hash 37dd33ccafbc622866a55e4df574d24ad403bf4170eec9a374191fb720e9806b
SimHash 725bcc918894

Groups

*

Rule Path
Disallow */cart/*
Disallow /area-cliente/*
Disallow */auth/*
Disallow /privado/
Disallow /*?search=*
Disallow /*?search2=*
Disallow /*?ob=*
Disallow /*?b=*
Disallow */c/*/*/*/*/f/*
Disallow */c/*/*/*/*/*/f/*
Disallow */c/*/*/*/*/*/*/f/*
Disallow */c/*/*/*/*/*/*/*/f/*
Disallow /gl/*
Disallow */c/f/*
Disallow */c/*/$
Disallow */c/*/$
Disallow */extra/*
Disallow */politica-privacidad
Disallow */aviso-legal
Disallow */cookies

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value Comment
crawl-delay 20 specifies a 2 second timeout

seekport-crawler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yellowbrandprotectionbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

honolulubot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

dataforseobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 40

cincraw

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefssiteaudit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

panscient.com

Rule Path
Disallow /

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

serpstatbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

qwantify

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20
Back to top

Other Records

Field Value
sitemap https://rodavigo.net/sitemaps/sitemap.xml
Back to top

Comments

  • carrito en todos los idiomas
  • área de cliente en todos los idiomas
  • Filtros y otros
  • Retirada de paginación
  • Disallow: /*?p=*
  • Disallow: /*&p=*
  • Retirada de términos de búsqueda
  • Retirada de ordenacion
  • Retirada de filtros conjuntos
  • Restriccion de categorias con fabricante
  • Resticcion idioma GL
  • Restriccion Roma Fabricante
  • Restriccion de barras al final
  • Retirada url /extra
  • Legal Pages
  • Filtros actuales de Rodavigo
Back to top