shell.sa
robots.txt

Robots Exclusion Standard data for shell.sa

Resource Scan

Scan Details

Site Domain shell.sa
Base Domain shell.sa
Scan Status Ok
Last Scan2024-10-23T06:34:30+00:00
Next Scan 2024-11-22T06:34:30+00:00

Last Scan

Scanned2024-10-23T06:34:30+00:00
URL https://shell.sa/robots.txt
Redirect https://www.shell.sa/robots.txt
Redirect Domain www.shell.sa
Redirect Base shell.sa
Domain IPs 4.210.156.184
Redirect IPs 2600:1413:b000:6::17d5:2bc6, 2600:1413:b000:6::17d5:2bd1, 96.17.96.17, 96.17.96.22
Response IP 23.44.4.160
Found Yes
Hash a3c9bef0e901b1532f04e276c36fe1267deb7d225bd366e09ef6390dd7d33849
SimHash 455c8e50ec81

Groups

siteimprovebot

Rule Path
Allow /

siteimprovebot-crawler

Rule Path
Allow /

*

Rule Path
Disallow /ar_sa/error.html
Disallow /ar_sa/error/
Disallow /ar_sa/external-redirects.html
Disallow /ar_sa/external-redirects/
Disallow /ar_sa/tag-search.html
Disallow /ar_sa/tag-search/
Disallow /en_sa/error.html
Disallow /en_sa/error/
Disallow /en_sa/external-redirects.html
Disallow /en_sa/external-redirects/
Disallow /en_sa/motorists/car-engine-oils/helix-fully-synthetic.html
Disallow /en_sa/motorists/car-engine-oils/helix-fully-synthetic/
Disallow /en_sa/motorists/shell-cafe-food-and-drinks.html
Disallow /en_sa/motorists/shell-cafe-food-and-drinks/
Disallow /en_sa/tag-search.html
Disallow /en_sa/tag-search/

Other Records

Field Value
sitemap https://www.shell.sa/.sitemap.xml
sitemap https://www.shell.sa/ar_sa.sitemap.xml