Log In Sign Up

trimurty.com
robots.txt

Robots Exclusion Standard data for trimurty.com

Resource Scan

Scan Details

Site Domain trimurty.com
Base Domain trimurty.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-06T02:59:43+00:00
Next Scan 2026-01-04T02:59:43+00:00

Last Successful Scan

Scanned2025-05-16T20:32:37+00:00
URL https://trimurty.com/robots.txt
Redirect https://www.trimurty.com/robots.txt
Redirect Domain www.trimurty.com
Redirect Base trimurty.com
Domain IPs 107.6.170.242
Redirect IPs 107.6.170.242
Response IP 107.6.170.242
Found Yes
Hash 60d3b99026193f2e639b53c7afa8f918e4d14dbc2e126282a4c92f94aa3a8db7
SimHash 6c599b2374f1

Groups

*

Rule Path
Disallow

googlebot

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

msnbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /

yahoo-mmcrawler

Rule Path
Allow /

yandex

Rule Path
Allow /
Disallow /login/
Disallow /admin/
Disallow /admin/login/
Disallow /cgi-bin/
Disallow /temp/
Disallow /public/
Disallow /vendor/
Disallow /node_modules/
Disallow /config/
Disallow /bootstrap/
Disallow /resources/
Disallow /login
Disallow /register
Disallow /password/reset

badbot

Rule Path
Disallow /

evilscraper

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.trimurty.com/sitemap.xml
Back to top

Comments

  • Allow all search engines to crawl all content
  • Disallow specific directories that are not meant to be indexed
  • Disallow specific files that should not be indexed
  • Block specific crawlers (example: BadBot)
Back to top