sunbrella.com
robots.txt

Robots Exclusion Standard data for sunbrella.com

Resource Scan

Scan Details

Site Domain sunbrella.com
Base Domain sunbrella.com
Scan Status Ok
Last Scan2024-10-31T20:47:57+00:00
Next Scan 2024-11-30T20:47:57+00:00

Last Scan

Scanned2024-10-31T20:47:57+00:00
URL https://sunbrella.com/robots.txt
Redirect https://www.sunbrella.com/robots.txt
Redirect Domain www.sunbrella.com
Redirect Base sunbrella.com
Domain IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Redirect IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Response IP 199.232.45.124
Found Yes
Hash 52a801113ed2f65b2336576f5f85be260c83b42993e489e4a1d972ca780badbc
SimHash 5524bb50ceb3

Groups

adsbot-google
bingbot
pinterestbot
semrushbot
ahrefsbot
dotbot
*

Rule Path
Disallow /admin/
Disallow /app/
Disallow /downloader/
Disallow /errors/
Disallow /includes/
Disallow /lib/
Disallow /*.php$
Disallow /pkginfo/
Disallow /shell/
Disallow /report/
Disallow /var/
Disallow /catalog/
Disallow /customer/
Disallow /sendfriend/
Disallow /review/
Disallow /*SID%3D
Disallow /amasty_xsearch/
Disallow /.well-known/
Disallow /amblog/
Disallow /cms/noroute/index
Disallow /api.php
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /get.php
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /README.txt
Disallow /RELEASE_NOTES.txt
Disallow /*?dir*
Disallow /*?dir=desc
Disallow /*?dir=asc
Disallow /*?limit=all
Disallow /*?mode*
Disallow /*?fabric_usage_us*
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /customer/account/
Disallow /customer/account/login/
Disallow /catalogsearch/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/

petalbot
siteauditbot
babbar
baiduspider
baiduspider-video
baiduspider-image
yandex
lunabot
neevabot
blexbot
bytespider
mj12bot
claudebot
ccbot
img2dataset
gptbot
chatgpt-user
google-extended
anthropic-ai
claude-web
omgilibot
omgili
facebookbot
bytespider
magpie-crawler

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.sunbrella.com/media/sitemap/consumer_sitemap.xml
sitemap https://www.sunbrella.com/media/sitemap/consumer-faceted-sitemap.xml
sitemap https://www.sunbrella.com/media/sitemap/sitemap-pdfs-2020.xml

Comments

  • Do not index the page Magento admin
  • Directories
  • Do not index the shared files Magento
  • MAGENTO SEA IMPROVEMENT
  • Do not index the page subcategories that are sorted or filtered.
  • Do not index the page checkout and user account
  • Do not index the search page and CEO, non-optimized link categories
  • Blocking No Traffic Bots
  • Delay for Amazonbot
  • Sitemap Location