poy.es
robots.txt

Robots Exclusion Standard data for poy.es

Resource Scan

Scan Details

Site Domain poy.es
Base Domain poy.es
Scan Status Ok
Last Scan2025-12-08T09:02:24+00:00
Next Scan 2025-12-15T09:02:24+00:00

Last Scan

Scanned2025-12-08T09:02:24+00:00
URL https://poy.es/robots.txt
Domain IPs 92.205.9.191
Response IP 92.205.9.191
Found Yes
Hash 7e6af0d4659b808a3e121bfd092c0c3b1792c4908af646d82829308e4d49c31b
SimHash 28d38a71ea22

Groups

*

Rule Path
Disallow /semanas
Disallow /archive
Disallow /cgi-bin
Disallow /es
Disallow /_temp/
Disallow /private/
Disallow /admin/
Disallow /login/
Disallow /get/
Disallow /bt/
Disallow /archive/
Disallow /tmp/
Disallow /trackback/
Disallow /comments/
Disallow /cgi-bin/
Disallow /feed/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /cgi-bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /trackback/
Disallow /feed/
Disallow /comments/
Disallow */trackback/
Disallow */feed/
Disallow */comments/
Disallow */embed/
Disallow /xmlrpc.php
Disallow /wp-json/
Disallow /*?s=
Disallow /*?replytocom
Disallow /*?add-to-cart=*
Disallow /*?filter_*
Disallow /*?orderby=*
Disallow /*?s=*
Disallow /*?price=*
Disallow /*?rating=*
Disallow /*?view=*
Disallow /*?product_tag=*
Disallow /?attachment_id=*

*

Rule Path
Disallow /*CFNetwork*
Disallow /*facebookexternalhit*
Disallow /*feed*
Disallow /*Unknown*
Disallow /*Mediapartners-Google*
Disallow /*bingbot*
Disallow /*AhrefsBot*
Disallow /*empty*
Disallow /*proximic*
Disallow /*fetcher*
Disallow /*Googlebot-Image*
Disallow /*Dalvik*
Disallow /*Go-http-client*
Disallow /*GrapeshotCrawler*
Disallow /*Applebot*
Disallow /*YandexBot*
Disallow /*DuckDuckBot-Https*
Disallow /*Lightspeed*
Disallow /*SemrushBot*
Disallow /*MJ12bot*
Disallow /*Firefox*
Disallow /*Discordbot*
Disallow /*semanticbot*
Disallow /*BLEXBot*
Disallow /*DotBot*
Disallow /*Java*
Disallow /*MegaIndex.ru*
Disallow /*CUBOT*
Disallow /*TelegramBot*
Disallow /*archive.org_bot*
Disallow /*DuckDuckGo-Favicons-Bot*
Disallow /*BingPreview*
Disallow /*Apache-HttpClient*
Disallow /*robot*
Disallow /*Mediatoolkitbot*
Disallow /*Curl*
Disallow /*survey*
Disallow /*yak-linkfluence*
Disallow /*Yandex*
Disallow /*Barkrowler*
Disallow /*Python-urllib*
Disallow /*spider*
Disallow /*AdIdxBot*
Disallow /*serendeputy*
Disallow /*Xenu*
Disallow /*Microsoft*
Disallow /*nbot*
Disallow /*Who.is.Bot*
Disallow /*scanner*
Disallow /*LinkedInBot*
Disallow /*SeznamBot*
Disallow /*AdsBot-Google*
Disallow /*Baiduspider*
Disallow /*Google-Adwords-Instant*

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

majestic-12

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

infohelfer

Rule Path
Disallow /

sogou

Rule Path
Disallow /

bspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

aboundex

Rule Path
Disallow /

cubot

Rule Path
Disallow /

crawl

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

proximic

Rule Path
Disallow /

unknown robot identified by bot*

Rule Path
Disallow /

empty user agent string

Rule Path
Disallow /

firefox version 10 and lower - various robots

Rule Path
Disallow /

dalvik

Rule Path
Disallow /

telegrambot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

Other Records

Field Value
sitemap sitemap.xml

Warnings

  • 3 invalid lines.