sendbot.co
robots.txt

Robots Exclusion Standard data for sendbot.co

Resource Scan

Scan Details

Site Domain sendbot.co
Base Domain sendbot.co
Scan Status Ok
Last Scan2025-10-13T15:02:57+00:00
Next Scan 2025-10-27T15:02:57+00:00

Last Scan

Scanned2025-10-13T15:02:57+00:00
URL https://sendbot.co/robots.txt
Redirect https://sendbot.com.br/robots.txt
Redirect Domain sendbot.com.br
Redirect Base sendbot.com.br
Domain IPs 104.26.6.104, 104.26.7.104, 172.67.74.136, 2606:4700:20::681a:668, 2606:4700:20::681a:768, 2606:4700:20::ac43:4a88
Redirect IPs 104.21.45.12, 172.67.207.73, 2606:4700:3030::6815:2d0c, 2606:4700:3030::ac43:cf49
Response IP 172.67.207.73
Found Yes
Hash 9c2158d82c97f68205f05758072fe73fa1d5b738aea8b15de771a8c373d041fe
SimHash 4c36521065e0

Groups

*

Rule Path
Allow /
Allow /api/health
Allow /api/performance
Disallow /api/contact
Disallow /api/error-tracking
Disallow /api/auth/
Disallow /api/admin/
Disallow /admin/
Disallow /_next/
Disallow /.next/
Disallow /private/
Disallow /tmp/
Disallow /wp-admin/
Disallow /wp-content/
Disallow /wp-includes/
Disallow /cgi-bin/

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

whatsapp

Rule Path
Allow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://sendbot.co/sitemap.xml
sitemap https://sendbot.co/sitemap-images.xml
sitemap https://sendbot.co/sitemap-news.xml

Comments

  • Robots.txt for Sendbot.co
  • Version: 2.0
  • Last Updated: 2024
  • Allow all crawlers to access most content
  • Block sensitive API endpoints
  • Block common bot traps
  • Special rules for specific crawlers
  • Block aggressive crawlers
  • Sitemaps
  • Cache directive
  • Crawlers should respect this

Warnings

  • `host` is not a known field.