Log In Sign Up

cakebox.com
robots.txt

Robots Exclusion Standard data for cakebox.com

Resource Scan

Scan Details

Site Domain cakebox.com
Base Domain cakebox.com
Scan Status Ok
Last Scan2025-11-07T09:19:41+00:00
Next Scan 2025-12-07T09:19:41+00:00

Last Scan

Scanned2025-11-07T09:19:41+00:00
URL https://cakebox.com/robots.txt
Redirect https://www.cakebox.com/robots.txt
Redirect Domain www.cakebox.com
Redirect Base cakebox.com
Domain IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Redirect IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Response IP 199.232.113.124
Found Yes
Hash 358db878149a4fd282eb22f14f848e2ed49c0792616faaecf010ee847204e512
SimHash 7524fa5b47f8

Groups

googlebot
bingbot
gptbot
google-extended
applebot-extended
ccbot
perplexitybot
claudebot

Rule Path
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /downloader/
Disallow /errors/
Disallow /includes/
Disallow /lib/
Disallow /pkginfo/
Disallow /phpserver/
Disallow /report/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /vendor/
Disallow /*.php$
Disallow /composer.json
Disallow /composer.lock
Disallow /CONTRIBUTING.md
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow /COPYING.txt
Disallow /Gruntfile.js
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /nginx.conf.sample
Disallow /package.json
Disallow /php.ini.sample
Disallow /RELEASE_NOTES.txt
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /customer/
Disallow /sendfriend/
Disallow /catalogsearch/
Disallow /catalog/product_compare/
Disallow /review/
Disallow /tag/
Disallow /.git
Disallow /.CVS
Disallow /.Svn$
Disallow /.Idea$
Disallow /.Zip$
Disallow /.Sql$
Disallow /*.Tgz$
Disallow /*?*
Allow /pub/media/
Allow /pub/static/

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /downloader/
Disallow /errors/
Disallow /includes/
Disallow /lib/
Disallow /pkginfo/
Disallow /phpserver/
Disallow /report/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /vendor/
Disallow /*.php$
Disallow /composer.json
Disallow /composer.lock
Disallow /CONTRIBUTING.md
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow /COPYING.txt
Disallow /Gruntfile.js
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /nginx.conf.sample
Disallow /package.json
Disallow /php.ini.sample
Disallow /RELEASE_NOTES.txt
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /customer/
Disallow /sendfriend/
Disallow /catalogsearch/
Disallow /catalog/product_compare/
Disallow /review/
Disallow /tag/
Disallow /.git
Disallow /.CVS
Disallow /.Svn$
Disallow /.Idea$
Disallow /.Zip$
Disallow /.Sql$
Disallow /*.Tgz$
Disallow /*?*
Allow /pub/media/
Allow /pub/static/

Other Records

Field Value
crawl-delay 5
Back to top

Other Records

Field Value
sitemap https://www.cakebox.com/media/sitemap/sitemap.xml
Back to top

Comments

  • === GROUP 1: Rules for Reputable AI & Search Bots ===
  • This block applies to all bots listed.
  • Block System Directories & Files
  • Block Core Files
  • Block Private Customer Areas
  • Block Low-Value & Duplicate Content Pages
  • Block Version Control & Archives
  • Block ALL URL Parameters (Filtering, Sorting, etc.)
  • Allow Static Assets (CSS, JS, Images)
  • === GROUP 2: Catch-all for All Other Bots ===
  • This applies the same rules to any bot not listed above.
  • Block System Directories & Files
  • Block Core Files
  • Block Private Customer Areas
  • Block Low-Value & Duplicate Content Pages
  • Block Version Control & Archives
  • Block ALL URL Parameters (Filtering, Sorting, etc.)
  • Allow Static Assets (CSS, JS, Images)
  • === Sitemap (Applies to all) ===
Back to top