supersales.be
robots.txt

Robots Exclusion Standard data for supersales.be

Resource Scan

Scan Details

Site Domain supersales.be
Base Domain supersales.be
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-15T06:30:25+00:00
Next Scan 2024-11-13T06:30:25+00:00

Last Successful Scan

Scanned2023-12-27T05:08:15+00:00
URL https://supersales.be/robots.txt
Redirect https://www.supersales.be/robots.txt
Redirect Domain www.supersales.be
Redirect Base supersales.be
Domain IPs 104.21.81.61, 172.67.157.46, 2606:4700:3030::6815:513d, 2606:4700:3036::ac43:9d2e
Redirect IPs 104.21.81.61, 172.67.157.46, 2606:4700:3030::6815:513d, 2606:4700:3036::ac43:9d2e
Response IP 104.21.81.61
Found Yes
Hash 2741cb920e18215db0c3851ea3c45350ea6105d559f821d75115251db291a635
SimHash 5257d440ce9b

Groups

*

Rule Path
Disallow /r/
Disallow /gcr/
Disallow /ri/
Disallow /delen
Disallow /share
Disallow /account
Disallow /konto
Disallow /coupon-view/

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

ioncrawl

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

lcc

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /