Log In Sign Up

vismithams.in
robots.txt

Robots Exclusion Standard data for vismithams.in

Resource Scan

Scan Details

Site Domain vismithams.in
Base Domain vismithams.in
Scan Status Ok
Last Scan2025-10-21T21:17:15+00:00
Next Scan 2025-11-04T21:17:15+00:00

Last Scan

Scanned2025-10-21T21:17:15+00:00
URL https://vismithams.in/robots.txt
Domain IPs 104.21.9.99, 172.67.189.55, 2606:4700:3030::ac43:bd37, 2606:4700:3036::6815:963
Response IP 104.21.9.99
Found Yes
Hash d0216a627657ed6ab218793c10879fb622b3e43288be91473a03d0a88bc554b6
SimHash 713cd812ee9a

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

applebot

Rule Path
Allow /

gptbot

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

google-extended

Rule Path
Allow /

claudebot

Rule Path
Allow /

claude-web

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

xai-crawler

Rule Path
Allow /

deepseekbot

Rule Path
Allow /

ccbot

Rule Path
Allow /

amazonbot

Rule Path
Allow /

youbot

Rule Path
Allow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /cart/
Disallow /checkout/
Disallow /my-account/
Disallow /cgi-bin/
Disallow /trackback/
Disallow /xmlrpc.php
Disallow /author/
Disallow /tag/
Disallow /?author=*
Disallow /?s=
Back to top

Other Records

Field Value
sitemap https://vismithams.in/sitemap.xml
Back to top

Comments

  • Allow major search engines and AI bots
  • Block common ad, spam, and bad bots
  • Disallow crawling sensitive areas for security
  • Prevent indexing of duplicate or private content
  • Sitemap location
Back to top