shell.eg
robots.txt

Robots Exclusion Standard data for shell.eg

Resource Scan

Scan Details

Site Domain shell.eg
Base Domain shell.eg
Scan Status Ok
Last Scan2024-11-05T05:48:16+00:00
Next Scan 2024-12-05T05:48:16+00:00

Last Scan

Scanned2024-11-05T05:48:16+00:00
URL https://shell.eg/robots.txt
Redirect https://www.shell.eg/robots.txt
Redirect Domain www.shell.eg
Redirect Base shell.eg
Domain IPs 4.210.156.184
Redirect IPs 2600:1413:b000:6::17d5:2bc6, 2600:1413:b000:6::17d5:2bd1, 96.17.96.17, 96.17.96.22
Response IP 23.44.4.160
Found Yes
Hash 964af7ee23d759db210e883d14d6ffcc1ac8fe51f2870060983d258a6e6a6646
SimHash 0a14d47454d3

Groups

siteimprovebot

Rule Path
Allow /

siteimprovebot-crawler

Rule Path
Allow /

*

Rule Path
Disallow /ar_eg/error.html
Disallow /ar_eg/error/
Disallow /ar_eg/external-redirects.html
Disallow /ar_eg/external-redirects/
Disallow /ar_eg/motorists/rimula-branches.html
Disallow /ar_eg/motorists/rimula-branches/
Disallow /ar_eg/motorists/shell-stars-program.html
Disallow /ar_eg/motorists/shell-stars-program/
Disallow /ar_eg/motorists/womens-day-offer.html
Disallow /ar_eg/motorists/womens-day-offer/
Disallow /ar_eg/tag-search.html
Disallow /ar_eg/tag-search/
Disallow /en_eg/error.html
Disallow /en_eg/error/
Disallow /en_eg/external-redirects.html
Disallow /en_eg/external-redirects/
Disallow /en_eg/tag-search.html
Disallow /en_eg/tag-search/

Other Records

Field Value
sitemap https://www.shell.eg/.sitemap.xml
sitemap https://www.shell.eg/ar_eg.sitemap.xml