Log In Sign Up

configure-seu-dominio.com.br
robots.txt

Robots Exclusion Standard data for configure-seu-dominio.com.br

Resource Scan

Scan Details

Site Domain configure-seu-dominio.com.br
Base Domain configure-seu-dominio.com.br
Scan Status Ok
Last Scan2025-05-27T01:48:39+00:00
Next Scan 2025-06-26T01:48:39+00:00

Last Scan

Scanned2025-05-27T01:48:39+00:00
URL https://configure-seu-dominio.com.br/robots.txt
Domain IPs 104.21.34.10, 172.67.194.175, 2606:4700:3032::6815:220a, 2606:4700:3033::ac43:c2af
Response IP 172.67.194.175
Found Yes
Hash a7e96bce74c61a9c3300b50e19647cdc0c357ea3c27863602b5a6986092c40a3
SimHash 291fde21c6f8

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

yandexbot

Rule Path
Allow /

semrushbot

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

slurp

Rule Path
Allow /

*

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://configure-seu-dominio.com.br/sitemap.xml
Back to top

Comments

  • Google e Bing
  • Bots de anúncios e análise do Google
  • Bots de redes sociais
  • Bots de análise e SEO
  • Bloquear todos os outros bots
  • Configurar um crawl delay para reduzir impacto
  • Sitemap
Back to top