shop.grandstores.ae
robots.txt

Robots Exclusion Standard data for shop.grandstores.ae

Resource Scan

Scan Details

Site Domain shop.grandstores.ae
Base Domain grandstores.ae
Scan Status Ok
Last Scan2025-12-09T02:05:39+00:00
Next Scan 2026-01-08T02:05:39+00:00

Last Scan

Scanned2025-12-09T02:05:39+00:00
URL https://shop.grandstores.ae/robots.txt
Domain IPs 104.26.10.196, 104.26.11.196, 172.67.74.119, 2606:4700:20::681a:ac4, 2606:4700:20::681a:bc4, 2606:4700:20::ac43:4a77
Response IP 172.67.74.119
Found Yes
Hash 11ff1e39cd879e6ad916986b6a4ca85c9b42af966c95c872f044fea0030e976e
SimHash f92b9b59820f

Groups

*

Rule Path
Disallow /grand_admin/
Disallow /index.php/
Disallow /checkout/
Disallow /app/
Disallow /lib/
Disallow /*.php$
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /customer/
Disallow /sendfriend/
Disallow /review/
Disallow /*SID%3D
Disallow /wishlist/
Disallow /catalogsearch/result/?q=
Disallow /en/customer/
Disallow /en/sendfriend/
Disallow /en/review/
Disallow /en/wishlist/
Disallow /en/checkout/
Disallow /en/catalogsearch/result/?q=
Disallow /en_uae/customer/
Disallow /en_uae/sendfriend/
Disallow /en_uae/review/
Disallow /en_uae/wishlist/
Disallow /en_uae/checkout/
Disallow /en_uae/catalogsearch/result/?q=
Disallow /uae_en/customer/
Disallow /uae_en/sendfriend/
Disallow /uae_en/review/
Disallow /uae_en/wishlist/
Disallow /uae_en/checkout/
Disallow /uae_en/catalogsearch/result/?q=
Disallow /at_uae/customer/
Disallow /at_uae/sendfriend/
Disallow /at_uae/review/
Disallow /at_uae/wishlist/
Disallow /at_uae/checkout/
Disallow /at_uae/catalogsearch/result/?q=
Disallow /en/catalogsearch/result/index/?
Disallow /at_uae/catalogsearch/result/index/?
Disallow /uae_en/catalogsearch/result/index/?
Disallow /en_uae/catalogsearch/result/index/?

baiduspider

Rule Path
Disallow /

proximic

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

seekportbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-cloudvertexbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

kangaroo bot

Rule Path
Disallow /

pangubot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

cohere-training-data-crawler

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /

youbot

Rule Path
Disallow /

gptbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

chatgpt-user

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

perplexitybot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

claude-web

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

google-extended

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

meta-externalagent

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

facebookexternalhit/1.1

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

Other Records

Field Value
sitemap https://shop.grandstores.ae/media/sitemap/sitemap.xml

Comments

  • Baiduspider