dealsan.es
robots.txt

Robots Exclusion Standard data for dealsan.es

Resource Scan

Scan Details

Site Domain dealsan.es
Base Domain dealsan.es
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan5/14/2025, 10:33:49 AM
Next Scan 8/12/2025, 10:33:49 AM

Last Successful Scan

Scanned10/23/2023, 5:09:19 AM
URL https://www.dealsan.es/robots.txt
Domain IPs 104.21.75.214, 172.67.182.59, 2606:4700:3031::6815:4bd6, 2606:4700:3036::ac43:b63b
Response IP 104.21.75.214
Found Yes
Hash 883fe24cfe6ce02c9c34e8dff6623f63c17bfa5fc8b914a6f3cdf14519246eac
SimHash e60e0d00cfda

Groups

*

Rule Path
Disallow /buscar/*
Disallow /go/*
Disallow /item/*
Disallow /stores/*
Disallow /stores/amazon/*
Disallow /stores/etsy/*
Disallow /amazon.php*
Disallow /etsy.php*
Disallow /ebay.php*
Disallow /comprar/timthumb.php*
Disallow /timthumb.php?src=*

applebot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

httrack

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

brandverity

Rule Path
Disallow /

spiderling

Rule Path
Disallow /

buck

Rule Path
Disallow /

tigerbot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

bot@linkfluence.net

Rule Path
Disallow /

evc-batch

Rule Path
Disallow /

zgrab

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow

seznam

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

amazonbot

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.dealsan.es/sitemap.xml