gujarati.moneycontrol.com
robots.txt

Robots Exclusion Standard data for gujarati.moneycontrol.com

Resource Scan

Scan Details

Site Domain gujarati.moneycontrol.com
Base Domain moneycontrol.com
Scan Status Ok
Last Scan2024-09-01T21:31:57+00:00
Next Scan 2024-10-01T21:31:57+00:00

Last Scan

Scanned2024-09-01T21:31:57+00:00
URL https://gujarati.moneycontrol.com/robots.txt
Domain IPs 23.36.50.99, 2600:1413:1:591::3379, 2600:1413:1:592::3379
Response IP 23.54.58.50
Found Yes
Hash a1758e1cb2be9599aa22addfb5c16edc6f4865e252c48cc1b36a4d4c1da32213
SimHash 6815d1512101

Groups

*

Rule Path
Allow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

mazbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://gujarati.moneycontrol.com/sitemap/news_sitemap.xml
sitemap https://gujarati.moneycontrol.com/category-sitemap.xml
sitemap https://gujarati.moneycontrol.com/index-sitemap-2023.xml