befitting.com
robots.txt

Robots Exclusion Standard data for befitting.com

Resource Scan

Scan Details

Site Domain befitting.com
Base Domain befitting.com
Scan Status Ok
Last Scan2025-10-13T03:50:46+00:00
Next Scan 2025-11-12T03:50:46+00:00

Last Scan

Scanned2025-10-13T03:50:46+00:00
URL https://befitting.com/robots.txt
Domain IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Response IP 151.101.193.124
Found Yes
Hash ede5db899e68dc9eed91aad0b3654ec3a1bce3c2898f116d4a8bf8970100e4d4
SimHash b15cfd0a66b0

Groups

googlebot-image

Rule Path
Disallow

*

Rule Path
Disallow */404/
Disallow */app/
Disallow */cgi-bin/
Disallow */downloader/
Disallow */errors/
Disallow */includes/
Disallow */magmi/
Disallow */pkginfo/
Disallow */report/
Disallow */shell/
Disallow */skin/
Disallow */stats/
Disallow */var/
Disallow */admin/
Disallow */wishlist/
Disallow */catalogsearch/
Disallow */enable-cookies/
Disallow */directory/
Disallow */intranet/
Disallow */media/wysiwyg/
Disallow */catalog/product_compare/
Disallow */catalog/category/view/
Disallow */catalog/product/view/
Disallow */catalog/product/gallery/
Disallow */checkout/
Disallow */control/
Disallow */customer/
Disallow */customize/
Disallow */newsletter/
Disallow */sendfriend/
Disallow */lib/
Disallow */review/
Disallow */*SID%3D
Disallow /*?___SID=
Disallow /*?___store=
Disallow */index.php/
Disallow */LICENSE.txt
Disallow */LICENSE.html
Disallow */LICENSE_AFL.txt
Disallow */STATUS.txt
Disallow */*.php$
Disallow */*.sh$

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.befitting.com/en_us/sitemap.xml
sitemap https://www.befitting.com/en_ca/sitemap.xml
sitemap https://www.befitting.com/en_us/sitemap_categories.xml
sitemap https://www.befitting.com/en_ca/sitemap_categories.xml
sitemap https://www.befitting.com/en_us/sitemap_products.xml
sitemap https://www.befitting.com/en_ca/sitemap_products.xml
sitemap https://www.befitting.com/en_us/sitemap_cms.xml
sitemap https://www.befitting.com/en_ca/sitemap_cms.xml
sitemap https://www.befitting.com/en_us/sitemap
sitemap https://www.befitting.com/en_ca/sitemap

Comments

  • Google Image Crawler Setup
  • All Crawlers Setup
  • Directories
  • Parameters
  • Files
  • Sitemaps