planetahuerto.es
robots.txt

Robots Exclusion Standard data for planetahuerto.es

Resource Scan

Scan Details

Site Domain planetahuerto.es
Base Domain planetahuerto.es
Scan Status Ok
Last Scan2024-05-13T13:47:00+00:00
Next Scan 2024-05-20T13:47:00+00:00

Last Scan

Scanned2024-05-13T13:47:00+00:00
URL https://planetahuerto.es/robots.txt
Redirect https://www.planetahuerto.es/robots.txt
Redirect Domain www.planetahuerto.es
Redirect Base planetahuerto.es
Domain IPs 185.104.134.129
Redirect IPs 34.111.238.203
Response IP 34.111.238.203
Found Yes
Hash ae325e1b27bdce19c9f430d22d6b0fc04b66bf89c3721956d8298eb28de18817
SimHash 6c5d5126ccf7

Groups

*

Rule Path
Allow */*?cnWebLinkClicked=
Allow */*?page=
Allow */*?gclid=
Allow */*?utm
Allow /promo/
Disallow /subir/
Disallow /apagar/
Disallow /*?
Disallow */*?
Disallow /*?order=
Disallow /login*
Disallow /login-redirect*
Disallow /cesta*
Disallow /cesta/add
Disallow /opinar*
Disallow /commente*
Disallow /mi-cuenta*
Disallow /minha-conta*
Disallow /payment*
Disallow /resetear-contrasenya*
Disallow /alterar-a-palavra-passe*
Disallow /scripts/*
Disallow /politica-privacidad
Disallow /cookies
Disallow /articulos-buscar/*
Disallow /artigos-procurar/*
Disallow /artigos-procurar?*
Disallow /revista/procurar/*
Disallow /buscador
Disallow /buscador?*
Disallow /archivos/*.pdf
Disallow *.pdf
Disallow /revista/tag/*
Disallow /marca-*%3A*
Disallow /consultorio/*%3A*
Disallow /revista/autor/*%3A*
Disallow /promo/*?hierarchicalMenu
Disallow /black-friday/ofertas?category
Disallow /black-friday/ofertas?hierarchicalMenu

baiduspider

Rule Path
Disallow /

yahoo! slurp china

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

archive.org_bot
semrushbot
yandeximages

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

orthogaffe

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

xenu

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webcopier v3.2a

Rule Path
Disallow /

webcapture 2.0

Rule Path
Disallow /

webcopier v.2.2

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

sentibot www.sentibot.eu (compatible with googlebot)

Rule Path
Disallow /

mozilla/5.0 (compatible; sentibot/1.0; +https://www.sentibot.eu)

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

jooblebot

Rule Path
Disallow /

mozilla/5.0 (compatible; jooblebot/2.0; windows nt 6.1; wow64; +http://jooble.org/jooblebot) applewebkit/537.36 (khtml, like gecko) safari/537.36

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.planetahuerto.es/shared/sitemapindex-es.xml
sitemap http://www.planetahuerto.pt/shared/sitemapindex-pt.xml

Comments

  • Permitir rastreo PAGINACION, UTMs, Directorio de promociones
  • Private URLS
  • CORPORATIVE
  • QUERYS
  • FILES
  • CRAWLBUDGET
  • SITEMAP ES
  • SITEMAP PT
  • BOTS
  • SPAM BOTS

Warnings

  • `visit-time` is not a known field.