divyamarathi.bhaskar.com
robots.txt

Robots Exclusion Standard data for divyamarathi.bhaskar.com

Resource Scan

Scan Details

Site Domain divyamarathi.bhaskar.com
Base Domain bhaskar.com
Scan Status Ok
Last Scan2024-05-08T16:43:47+00:00
Next Scan 2024-05-22T16:43:47+00:00

Last Scan

Scanned2024-05-08T16:43:47+00:00
URL https://divyamarathi.bhaskar.com/robots.txt
Domain IPs 108.156.133.112, 108.156.133.27, 108.156.133.60, 108.156.133.96
Response IP 108.156.133.27
Found Yes
Hash 1bd74b81abcaa8ed1734ff86f7e00de8e5cc5ca2e56f66d179795aa18a481120
SimHash 401049418135

Groups

*

Rule Path
Allow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

Other Records

Field Value
sitemap https://divyamarathi.bhaskar.com/sitemaps-v1--sitemap-latest-videos.xml
sitemap https://divyamarathi.bhaskar.com/sitemaps-v1--sitemap-google-news-index.xml