unpez.com
robots.txt

Robots Exclusion Standard data for unpez.com

Resource Scan

Scan Details

Site Domain unpez.com
Base Domain unpez.com
Scan Status Ok
Last Scan2025-11-28T12:46:06+00:00
Next Scan 2025-12-05T12:46:06+00:00

Last Scan

Scanned2025-11-28T12:46:06+00:00
URL https://unpez.com/robots.txt
Redirect https://www.unpez.com/robots.txt
Redirect Domain www.unpez.com
Redirect Base unpez.com
Domain IPs 164.138.209.234
Redirect IPs 164.138.209.234
Response IP 164.138.209.234
Found Yes
Hash e7ea677f9bd8edcf99520aaa2df98f344b2cfc4068dc2f67d8c15caecb12485f
SimHash 98785942ef21

Groups

*

Rule Path
Allow /
Disallow /es/mujer/top/
Disallow /es/hombre/top/
Disallow /es/privacidad
Disallow /es/terminos-condiciones

netseer

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

sogou develop spider

Rule Path
Disallow /

sogou orion spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm

Product Comment
sogou orion spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm 07)
Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm

Product Comment
sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm 07)
Rule Path
Disallow /

sohu agent

Rule Path
Disallow /

ssurf15a 11

Rule Path
Disallow /

tsurf15a 11

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

moget

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

web core / roots

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

deusu

Rule Path
Disallow /

getintentcrawler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

Warnings

  • 2 invalid lines.