mathrubhuminews.in
robots.txt

Robots Exclusion Standard data for mathrubhuminews.in

Resource Scan

Scan Details

Site Domain mathrubhuminews.in
Base Domain mathrubhuminews.in
Scan Status Ok
Last Scan2024-11-01T20:57:16+00:00
Next Scan 2024-11-08T20:57:16+00:00

Last Scan

Scanned2024-11-01T20:57:16+00:00
URL http://mathrubhuminews.in/robots.txt
Redirect https://tv.mathrubhumi.com/robots.txt
Redirect Domain tv.mathrubhumi.com
Redirect Base mathrubhumi.com
Domain IPs 13.232.249.206
Redirect IPs 184.51.96.150
Response IP 23.54.58.34
Found Yes
Hash 1313129735eaf25f34b586a8250a61c87b44e7a26cfcb1cd0bdbd3ebd802e46f
SimHash ad344c13c171

Groups

*

Rule Path
Allow /
Disallow /WEB-INF
Disallow /META-INF
Disallow /polopolydevelopment/
Disallow /status/
Disallow /search-results/
Disallow /poll/
Disallow /logger/
Disallow /cm/
Disallow /error/
Disallow /errorpages/
Disallow /content/
Disallow /captcha/
Disallow /membership/
Disallow /blogimageupload/
Disallow /search*
Disallow /json
Disallow /app

googlebot-news

Rule Path
Allow /

googlebot-mobile
googlebot-image

Rule Path
Allow /polopoly_fs/

twitterbot

Rule Path
Allow /

ia_archiver

Rule Path
Allow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /