dealsan.fr
robots.txt

Robots Exclusion Standard data for dealsan.fr

Resource Scan

Scan Details

Site Domain dealsan.fr
Base Domain dealsan.fr
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2/8/2025, 9:26:36 PM
Next Scan 5/9/2025, 9:26:36 PM

Last Successful Scan

Scanned10/18/2023, 4:58:03 PM
URL https://www.dealsan.fr/robots.txt
Domain IPs 104.21.45.207, 172.67.218.221, 2606:4700:3030::6815:2dcf, 2606:4700:3030::ac43:dadd
Response IP 172.67.218.221
Found Yes
Hash 6e9d5a6f02a5cb4cc2ae8c1d1d89a41aca3d891b37ae09637b0f014ca68260dc
SimHash e2060d00ceda

Groups

*

Rule Path
Disallow /chercher/*
Disallow /go/*
Disallow /item/*
Disallow /stores/*
Disallow /stores/amazon/*
Disallow /stores/etsy/*
Disallow /amazon.php*
Disallow /etsy.php*
Disallow /ebay.php*
Disallow /acheter/timthumb.php*
Disallow /timthumb.php?src=*

applebot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

httrack

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

brandverity

Rule Path
Disallow /

spiderling

Rule Path
Disallow /

buck

Rule Path
Disallow /

tigerbot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

bot@linkfluence.net

Rule Path
Disallow /

evc-batch

Rule Path
Disallow /

zgrab

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow

seznam

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

amazonbot

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.dealsan.fr/sitemap.xml