Log In Sign Up

buspar.directory
robots.txt

Robots Exclusion Standard data for buspar.directory

Resource Scan

Scan Details

Site Domain buspar.directory
Base Domain buspar.directory
Scan Status Ok
Last Scan2025-06-24T14:44:10+00:00
Next Scan 2025-07-24T14:44:10+00:00

Last Scan

Scanned2025-06-24T14:44:10+00:00
URL https://buspar.directory/robots.txt
Domain IPs 2a02:4780:15:f990:3696:b082:7bb5:f479, 2a02:4780:39:d6f1:7975:d7c8:3b79:e788, 84.32.84.136, 84.32.84.238
Response IP 77.37.66.27
Found Yes
Hash f54c6b5e13cff0a8d0433a4db8365f964963be9f8641812b14ee4d3682a627b5
SimHash 720e59e27fa7

Groups

*

Rule Path
Allow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

duckduckbot

Rule Path
Allow /
Disallow /admin/
Disallow /private/
Disallow /temp/
Disallow /logs/
Allow /blog/
Allow /css/
Allow /js/
Allow /images/
Allow /php/
Allow /index.html
Allow /blog.html
Allow /about.html
Allow /contact.html
Allow /search.html
Allow /privacy-policy.html
Allow /terms-of-service.html

Other Records

Field Value
crawl-delay 1

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://buspar.directory/sitemap.xml
Back to top

Comments

  • Robots.txt for Buspar Directory
  • Educational website about buspirone (Buspar)
  • Sitemap location
  • Allow all major search engines
  • Block access to admin/backend areas (if any)
  • Allow access to all content directories
  • Allow access to important pages
  • Crawl delay for respectful crawling
  • Additional instructions for specific bots
  • Block common spam bots
  • Note: This is an educational website about buspirone
  • We welcome search engine indexing to help people find medical information
  • Website is not affiliated with official Buspar or pharmaceutical companies
Back to top