Log In Sign Up

neogol.com
robots.txt

Robots Exclusion Standard data for neogol.com

Resource Scan

Scan Details

Site Domain neogol.com
Base Domain neogol.com
Scan Status Ok
Last Scan2026-01-21T08:08:46+00:00
Next Scan 2026-01-28T08:08:46+00:00

Last Scan

Scanned2026-01-21T08:08:46+00:00
URL https://neogol.com/robots.txt
Domain IPs 204.93.189.133
Response IP 204.93.189.133
Found Yes
Hash b998307bd0ec95beb91a23957e20ce39b7dc38a904157d7984e1ec4dbdd240e9
SimHash 4f16de133672

Groups

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /
Back to top

Comments

  • Permitir Googlebot (pero con crawl-delay opcional)
  • Permitir Bingbot (opcional, también puede tener delay)
  • Bloquear el spammy DotBot (Moz)
  • Bloquear AhrefsBot (backlink checker)
  • Bloquear SemrushBot si alguna vez aparece (por si acaso)
Back to top