caratlane.com
robots.txt

Robots Exclusion Standard data for caratlane.com

Resource Scan

Scan Details

Site Domain caratlane.com
Base Domain caratlane.com
Scan Status Ok
Last Scan2024-10-05T20:23:56+00:00
Next Scan 2024-10-19T20:23:56+00:00

Last Scan

Scanned2024-10-05T20:23:56+00:00
URL https://www.caratlane.com/robots.txt
Domain IPs 18.155.68.100, 18.155.68.26, 18.155.68.56, 18.155.68.60, 2600:9000:23d2:1000:12:2d96:5fc0:93a1, 2600:9000:23d2:4000:12:2d96:5fc0:93a1, 2600:9000:23d2:4400:12:2d96:5fc0:93a1, 2600:9000:23d2:4c00:12:2d96:5fc0:93a1, 2600:9000:23d2:600:12:2d96:5fc0:93a1, 2600:9000:23d2:6c00:12:2d96:5fc0:93a1, 2600:9000:23d2:8800:12:2d96:5fc0:93a1, 2600:9000:23d2:a600:12:2d96:5fc0:93a1
Response IP 18.155.68.56
Found Yes
Hash 75975380d1097ffc30ce7e652dd61b9d07ca5cd33c5a32f17ec0828a862ebcd4
SimHash f624fd824fd3

Groups

semrush

Rule Path
Disallow /

*

Rule Path
Disallow /app/
Disallow /cgi-bin/
Disallow /downloader/
Disallow /includes/
Disallow /js/
Disallow /lib/
Disallow /magento/
Disallow /pkginfo/
Disallow /report/
Disallow /skin/
Disallow /stats/
Disallow /var/
Disallow /priceslider/
Allow /*?locale=en_US
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /checkout/
Disallow /control/
Disallow /contacts/
Disallow /customer/
Disallow /customize/
Disallow /newsletter/
Disallow /poll/
Disallow /review/
Disallow /sendfriend/
Disallow /tag/
Disallow /wishlist/
Disallow /catalogsearch/result/
Allow /catalog/seo_sitemap/category/
Disallow /catalog/
Disallow /home
Disallow /stores/jaipur-gems-hyderabad.html
Disallow /offers/corporate-discounts.html
Disallow /crm/roun.aspx
Disallow /index.html/
Disallow /index.php/
Disallow /diamonds_search.php
Disallow /load_minmax.php
Disallow /diamond_pair_search.php
Disallow /diamonds_count.php
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /install.php
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /STATUS.txt
Disallow /silver-*/?*
Disallow /silver/?*
Allow /blog/wp-content/plugins/*.js
Allow /blog/wp-content/plugins/*.css
Allow /blog/wp-content/themes/*.js
Allow /blog/wp-content/themes/*.css
Allow /blog/wp-content/cache/*.js
Allow /blog/wp-content/cache/*.css
Allow /blog/wp-includes/*.js
Disallow /*.php$
Disallow /*?p=*&
Disallow /*?SID=
Disallow /v5/i18n/mredirect/
Disallow /v5/submit_consultation
Disallow /v5/try-at-home/fetch_city_tah
Disallow /v5/post_tah_form
Disallow /newsletter/subscriber/new/

*

Rule Path
Disallow /design-your-own/design-your-pendant-step1.html
Disallow /design-your-own.html
Disallow /design-your-own/design-your-earring-step1.html
Disallow /design-your-own/design-your-ring-step1.html

*

Rule Path
Disallow /*payref*

*

Rule Path
Disallow /*by-purity*
Disallow /*by-purity%26_escaped_fragment_*

*

Rule Path
Disallow /*by-weight*
Disallow /*by-weight%26_escaped_fragment_*

*

Rule Path
Disallow /*by-design*
Disallow /*by-design%26_escaped_fragment_*

adsbot-google

Rule Path
Disallow /*by-purity*
Disallow /*by-purity%26_escaped_fragment_*

adsbot-google

Rule Path
Disallow /*by-weight*
Disallow /*by-weight%26_escaped_fragment_*

adsbot-google

Rule Path
Disallow /*by-design*
Disallow /*by-design%26_escaped_fragment_*

*

Rule Path
Disallow /priceslider/
Disallow /loose-diamonds/search/
Disallow /catalogsearch/
Disallow /checkout/
Disallow /shopper_home_2col*
Disallow /jewellery/*-mount-*
Disallow /jewellery/*astrological-ring*
Disallow /choose-diamond/sku*
Disallow /glossary/index/list/cat/*
Disallow /customertestimonials/testimonials/
Disallow /concentric-oval-pendant-jp02267-ygp9bs-70833.html
Disallow /jewellery/bezel-mr00904-yg0000.html
Disallow /ai-jewellery-assistant

ahrefsbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

semrush

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.caratlane.com/caratlane-sitemap/sitemap.xml
sitemap https://www.caratlane.com/caratlane-sitemap/product-sitemap.xml
sitemap https://www.caratlane.com/media/image_sitemap.xml
sitemap https://www.caratlane.com/shaya-sitemap/sitemap.xml
sitemap https://www.caratlane.com/shaya-sitemap/image-sitemap.xml

Comments

  • Website Sitemap
  • Directories
  • Allow international page
  • Paths (clean URLs)
  • Page
  • Files
  • Clean Silver URLS
  • Paths (no clean URLs)
  • Disallow: /*?
  • Blog Scripts
  • Post calls
  • Design Pages
  • Payref Pages
  • Purity Pages
  • Weight Pages
  • Design Pages
  • Purity Pages - Adwords agent
  • Weight Pages - Adwords agent
  • Design Pages - Adwords agent
  • Disallow URLs
  • end of robots.txt
  • Crawlers Setup