Log In Sign Up

fletcherstewart.com
robots.txt

Robots Exclusion Standard data for fletcherstewart.com

Resource Scan

Scan Details

Site Domain fletcherstewart.com
Base Domain fletcherstewart.com
Scan Status Ok
Last Scan2026-02-08T18:17:05+00:00
Next Scan 2026-03-10T18:17:05+00:00

Last Scan

Scanned2026-02-08T18:17:05+00:00
URL https://fletcherstewart.com/robots.txt
Redirect https://www.fletcherstewart.com/robots.txt
Redirect Domain www.fletcherstewart.com
Redirect Base fletcherstewart.com
Domain IPs 104.26.2.71, 104.26.3.71, 172.67.69.192, 2606:4700:20::681a:247, 2606:4700:20::681a:347, 2606:4700:20::ac43:45c0
Redirect IPs 104.26.2.71, 104.26.3.71, 172.67.69.192, 2606:4700:20::681a:247, 2606:4700:20::681a:347, 2606:4700:20::ac43:45c0
Response IP 172.67.69.192
Found Yes
Hash 798c4cab7409ac825edda925f09dcdcd0e38eb654d782257e0067d87067452aa
SimHash c9745b1ec69a

Groups

baiduspider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

*

Rule Path
Disallow /lib/
Disallow /*.php$
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /catalog/
Disallow /customer/
Disallow /sendfriend/
Disallow /review/
Disallow /*SID%3D
Disallow /*?
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /customer/
Disallow /customer/account/
Disallow /customer/account/login/
Disallow /catalogsearch/
Disallow /catalog/product_compare/
Disallow /tag/
Disallow /review/
Disallow /*?*product_list_mode=
Disallow /*?*product_list_order=
Disallow /*?*product_list_limit=
Disallow /*?*product_list_dir=
Back to top

Other Records

Field Value
sitemap https://www.fletcherstewart.com/pub/sitemap/sitemap_fs.xml
Back to top

Comments

  • Block Baiduspider Bot - too aggressive
  • Block Ahrefs Bot - too aggressive
  • Block BLEXBot - too aggressive
  • Block PetalBot - too aggressive
  • Block SemrushBot - too aggressive
  • Disable checkout & customer account
  • Disable Search pages
  • Disable Tag & Review (Avoid duplicate content)
  • Disable sorting (Avoid duplicate content)
Back to top