english.mathrubhumi.com
robots.txt

Robots Exclusion Standard data for english.mathrubhumi.com

Resource Scan

Scan Details

Site Domain english.mathrubhumi.com
Base Domain mathrubhumi.com
Scan Status Ok
Last Scan2024-11-02T08:01:20+00:00
Next Scan 2024-11-09T08:01:20+00:00

Last Scan

Scanned2024-11-02T08:01:20+00:00
URL https://english.mathrubhumi.com/robots.txt
Redirect https://english.mathrubhumi.com/?ot=com.atex.plugins.seoplugin.RobotsTxt.ot
Domain IPs 23.36.49.240
Response IP 23.54.58.34
Found Yes
Hash a8faf04885ca01f40a5307cd7c2efcec7d49a9498a44fd66bd1771d5bf69b3a5
SimHash 2f34c941a5b4

Groups

*

Rule Path
Allow /
Disallow /WEB-INF
Disallow /META-INF
Disallow /polopolydevelopment/
Disallow /status/
Disallow /search-results/
Disallow /poll/
Disallow /logger/
Disallow /cm/
Disallow /cmlink/
Disallow /error/
Disallow /errorpages/
Disallow /content/
Disallow /captcha/
Disallow /membership/
Disallow /blogimageupload/
Disallow /263
Disallow /json
Disallow /tabjson
Disallow /webpage
Disallow /sponsored/
Disallow /section/

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://english.mathrubhumi.com/sitemap.xml

Comments

  • Sitemaps