bodegasfernandocastro.com
robots.txt

Robots Exclusion Standard data for bodegasfernandocastro.com

Resource Scan

Scan Details

Site Domain bodegasfernandocastro.com
Base Domain bodegasfernandocastro.com
Scan Status Ok
Last Scan2024-06-10T20:43:34+00:00
Next Scan 2024-07-10T20:43:34+00:00

Last Scan

Scanned2024-06-10T20:43:34+00:00
URL https://bodegasfernandocastro.com/robots.txt
Domain IPs 34.175.243.32
Response IP 34.175.243.32
Found Yes
Hash 6d0d68382f5d32ba58694a7929dd068be319c8f974be4a60cb9c266f03ca350a
SimHash 62b948c5561a

Groups

googlebot

Rule Path
Disallow /carrito/
Disallow /finalizar-compra/
Disallow /mi-cuenta/
Disallow /aviso-legal/
Disallow /e-commerce-condiciones/
Disallow /politica-privacidad/
Disallow /politica-cookies/
Disallow /wp-login
Disallow /en/cart/
Disallow /en/my-account
Disallow /en/checkout/
Disallow /en/legal-notice/
Disallow /en/e-commerce-terms/
Disallow /en/privacy-policy/
Disallow /en/cookies-policy/
Disallow /fr/politique-de-confidentialite/
Disallow /fr/panier/
Disallow /fr/mon-compte/
Disallow /fr/validation/
Disallow /fr/avis-legal/
Disallow /fr/e-commerce-conditions-generales-de-souscription/
Disallow /fr/politique-de-cookies/

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /cgi-bin/
Disallow /tmp/
Disallow /junk/
Disallow /CHANGELOG.txt
Disallow /cron.php
Disallow /INSTALL.mysql.txt
Disallow /INSTALL.pgsql.txt
Disallow /INSTALL.sqlite.txt
Disallow /install.php
Disallow /INSTALL.txt
Disallow /LICENSE.txt
Disallow /MAINTAINERS.txt
Disallow /update.php
Disallow /UPGRADE.txt
Disallow /xmlrpc.php
Disallow /carrito/
Disallow /finalizar-compra/
Disallow /mi-cuenta/
Disallow /aviso-legal/
Disallow /e-commerce-condiciones/
Disallow /politica-privacidad/
Disallow /politica-cookies/
Disallow /wp-login
Disallow /en/cart/
Disallow /en/my-account
Disallow /en/checkout/
Disallow /en/legal-notice/
Disallow /en/e-commerce-terms/
Disallow /en/privacy-policy/
Disallow /en/cookies-policy/
Disallow /fr/politique-de-confidentialite/
Disallow /fr/panier/
Disallow /fr/mon-compte/
Disallow /fr/validation/
Disallow /fr/avis-legal/
Disallow /fr/e-commerce-conditions-generales-de-souscription/
Disallow /fr/politique-de-cookies/

Other Records

Field Value
crawl-delay 10

mauibot

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

mqqbrowser

Rule Path
Disallow /

nimbostratus-bot/v1.3.2

Rule Path
Disallow /

qwant-news

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sputnikbot/2.3

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

tinytestbot

Rule Path
Disallow /

ucbrowser

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

yeti

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

Other Records

Field Value
sitemap https://bodegasfernandocastro.com/sitemap_index.xml

Warnings

  • 40 invalid lines.