luxxy.com
robots.txt

Robots Exclusion Standard data for luxxy.com

Resource Scan

Scan Details

Site Domain luxxy.com
Base Domain luxxy.com
Scan Status Ok
Last Scan2024-05-19T18:54:56+00:00
Next Scan 2024-06-18T18:54:56+00:00

Last Scan

Scanned2024-05-19T18:54:56+00:00
URL https://luxxy.com/robots.txt
Redirect https://luxxy.ru/robots.txt
Redirect Domain luxxy.ru
Redirect Base luxxy.ru
Domain IPs 172.66.40.135, 172.66.43.121, 2606:4700:3108::ac42:2887, 2606:4700:3108::ac42:2b79
Redirect IPs 104.21.11.154, 172.67.166.96, 2606:4700:3032::ac43:a660, 2606:4700:3037::6815:b9a
Response IP 104.21.11.154
Found Yes
Hash 30fddf64054389bd187293f6f7592e65bc94502dca1a9c356df47708dc556010
SimHash 5add8bc3d5f1

Groups

*

Rule Path
Disallow /*profile*
Disallow /*ne-opredelena*
Disallow /*order*
Disallow /*page%3D*
Disallow /*page_size%3D*
Disallow /*sort%3D*
Disallow /*search%3D*
Disallow /*category%3D*
Disallow /*price_from%3D*
Disallow /*price_to%3D*
Disallow /*brand%3D*
Disallow /*color%3D*
Disallow /*composition%3D*
Disallow /*size%3D*
Disallow /*season%3D*
Disallow /*last_week%3D*
Disallow /*sale%3D*
Disallow /*popular%3D*
Disallow /search/
Disallow /login/
Disallow /signup/
Disallow /auth/
Disallow /*refresh-comments*
Disallow /*get-comments*
Disallow /error/*

yandex

Rule Path
Disallow /*profile*
Disallow /*ne-opredelena*
Disallow /*order*
Disallow /*page%3D*
Disallow /*page_size%3D*
Disallow /*sort%3D*
Disallow /*search%3D*
Disallow /*category%3D*
Disallow /*price_from%3D*
Disallow /*price_to%3D*
Disallow /*brand%3D*
Disallow /*color%3D*
Disallow /*composition%3D*
Disallow /*size%3D*
Disallow /*season%3D*
Disallow /*last_week%3D*
Disallow /*sale%3D*
Disallow /*popular%3D*
Disallow /search/
Disallow /login/
Disallow /signup/
Disallow /auth/
Disallow /*refresh-comments*
Disallow /*get-comments*
Disallow /error/*

ahrefsbot
semrushbot
searchmetricsbot
mj12bot
dotbot
rogerbot
screaming frog seo spider
scoutjet
linkdexbot
jamesbot
sistrix
curious george

Rule Path
Allow /

ubicrawler
doc
zao
ccbot
clickagy intelligence bot v2
domain re-animator bot
experibot_v1
getintent crawler
livelapbot/0.2
mfibot/1.1
msie 6.0
cliqzbot
discordbot
exabot/3.0
fatbot
imrbot
konqueror
paperlibot/2.0
trovitbot
veoozbot
socialrankiobot
uxcrawlerbot
woobot
yacybot

Rule Path
Disallow /

blexbot
ms search 4.0 robot
ms search 5.0 robot
sogou web spider
updownerbot
surveybot
webmeupbot
xenu link sleuth/1.3.8
xenu's
xenu's link sleuth 1.1c

Rule Path
Disallow /

Comments

  • SEO/SEM Competitor Tools Allowed
  • Bots that obey Robots.txt block
  • Bots that don't always obey robots.txt (might need to be kicked-out on hosting side)
  • User-agent: ia_archiver
  • User-agent: ia_archiver/1.6

Warnings

  • `clean-param` is not a known field.