shell.co.th
robots.txt

Robots Exclusion Standard data for shell.co.th

Resource Scan

Scan Details

Site Domain shell.co.th
Base Domain shell.co.th
Scan Status Ok
Last Scan2024-09-13T01:46:09+00:00
Next Scan 2024-10-13T01:46:09+00:00

Last Scan

Scanned2024-09-13T01:46:09+00:00
URL https://shell.co.th/robots.txt
Redirect https://www.shell.co.th/robots.txt
Redirect Domain www.shell.co.th
Redirect Base shell.co.th
Domain IPs 4.210.156.184
Redirect IPs 2600:1413:b000:6::17d5:2bce, 2600:1413:b000:6::17d5:2bcf, 96.17.96.27, 96.17.96.28
Response IP 23.44.5.40
Found Yes
Hash 46c8f80cebaf3c19a90d881ef84b2eba58223dc208a1c5e888bdea5dedd6bbe5
SimHash 484051160713

Groups

siteimprovebot

Rule Path
Allow /

siteimprovebot-crawler

Rule Path
Allow /

*

Rule Path
Disallow /en_th/error.html
Disallow /en_th/error/
Disallow /en_th/external-redirects.html
Disallow /en_th/external-redirects/
Disallow /en_th/tag-search.html
Disallow /en_th/tag-search/
Disallow /th_th/business/shell-fleet-solutions/shell-card/shell-campaign-register.html
Disallow /th_th/business/shell-fleet-solutions/shell-card/shell-campaign-register/
Disallow /th_th/customer/fuels-and-lubricants/fuels.html
Disallow /th_th/customer/fuels-and-lubricants/fuels/
Disallow /th_th/customer/service-stations.html
Disallow /th_th/customer/service-stations/
Disallow /th_th/error.html
Disallow /th_th/error/
Disallow /th_th/external-redirects.html
Disallow /th_th/external-redirects/
Disallow /th_th/tag-search.html
Disallow /th_th/tag-search/

Other Records

Field Value
sitemap https://www.shell.co.th/.sitemap.xml
sitemap https://www.shell.co.th/en_th.sitemap.xml