Log In Sign Up

typebotbrasil.com
robots.txt

Robots Exclusion Standard data for typebotbrasil.com

Resource Scan

Scan Details

Site Domain typebotbrasil.com
Base Domain typebotbrasil.com
Scan Status Ok
Last Scan2025-10-18T13:16:12+00:00
Next Scan 2025-11-01T13:16:12+00:00

Last Scan

Scanned2025-10-18T13:16:12+00:00
URL https://typebotbrasil.com/robots.txt
Redirect https://sendbot.com.br/robots.txt
Redirect Domain sendbot.com.br
Redirect Base sendbot.com.br
Domain IPs 76.76.21.21
Redirect IPs 104.21.45.12, 172.67.207.73, 2606:4700:3030::6815:2d0c, 2606:4700:3030::ac43:cf49
Response IP 104.21.45.12
Found Yes
Hash 9c2158d82c97f68205f05758072fe73fa1d5b738aea8b15de771a8c373d041fe
SimHash 4c36521065e0

Groups

*

Rule Path
Allow /
Allow /api/health
Allow /api/performance
Disallow /api/contact
Disallow /api/error-tracking
Disallow /api/auth/
Disallow /api/admin/
Disallow /admin/
Disallow /_next/
Disallow /.next/
Disallow /private/
Disallow /tmp/
Disallow /wp-admin/
Disallow /wp-content/
Disallow /wp-includes/
Disallow /cgi-bin/

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

whatsapp

Rule Path
Allow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://sendbot.co/sitemap.xml
sitemap https://sendbot.co/sitemap-images.xml
sitemap https://sendbot.co/sitemap-news.xml
Back to top

Comments

  • Robots.txt for Sendbot.co
  • Version: 2.0
  • Last Updated: 2024
  • Allow all crawlers to access most content
  • Block sensitive API endpoints
  • Block common bot traps
  • Special rules for specific crawlers
  • Block aggressive crawlers
  • Sitemaps
  • Cache directive
  • Crawlers should respect this
Back to top

Warnings

  • `host` is not a known field.
Back to top