mathrubhumi.com
robots.txt

Robots Exclusion Standard data for mathrubhumi.com

Resource Scan

Scan Details

Site Domain mathrubhumi.com
Base Domain mathrubhumi.com
Scan Status Ok
Last Scan2024-04-30T18:54:44+00:00
Next Scan 2024-05-07T18:54:44+00:00

Last Scan

Scanned2024-04-30T18:54:44+00:00
URL https://mathrubhumi.com/robots.txt
Redirect https://www.mathrubhumi.com/?ot=com.atex.plugins.seoplugin.RobotsTxt.ot
Redirect Domain www.mathrubhumi.com
Redirect Base mathrubhumi.com
Domain IPs 13.232.249.206
Redirect IPs 23.50.95.100, 2600:1413:b000:380::deb, 2600:1413:b000:38d::deb
Response IP 104.103.148.114
Found Yes
Hash 3036f0c7e1bcacaf0e8e1615bda10398f8651431c13a3f03a3867c4b975922cf
SimHash 68244a85a5b2

Groups

*

Rule Path
Allow /
Disallow /WEB-INF
Disallow /META-INF
Disallow /polopolydevelopment/
Disallow /status/
Disallow /search-results/
Disallow /poll/
Disallow /logger/
Disallow /cm/
Disallow /cmlink/
Disallow /error/
Disallow /errorpages/
Disallow /content/
Disallow /captcha/
Disallow /membership/
Disallow /blogimageupload/
Disallow /263
Disallow /json
Disallow /webpage
Disallow /tabjson
Disallow /home
Disallow /sponsored/
Disallow /section/

googlebot-news

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

googlebot-image

Rule Path
Allow /image/

twitterbot

Rule Path
Allow /

ia_archiver

Rule Path
Allow /

proximic

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

kraken

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.mathrubhumi.com/sitemap.xml

Comments

  • Sitemaps
  • http://linkfluence.net/
  • http://napoveda.seznam.cz/en/seznambot-intro/
  • https://awario.com/bots.html