Log In Sign Up

barcelonalternativa.com
robots.txt

Robots Exclusion Standard data for barcelonalternativa.com

Resource Scan

Scan Details

Site Domain barcelonalternativa.com
Base Domain barcelonalternativa.com
Scan Status Ok
Last Scan2026-02-05T02:03:07+00:00
Next Scan 2026-02-12T02:03:07+00:00

Last Scan

Scanned2026-02-05T02:03:07+00:00
URL https://barcelonalternativa.com/robots.txt
Domain IPs 50.31.177.6
Response IP 50.31.177.6
Found Yes
Hash a6154647f7aef79053a269340754c4f9071aa9952c4efebcc4284abadfc0ebe6
SimHash e851dc90cd85

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

*

Rule Path
Allow /feed/$
Disallow /feed/
Disallow /comments/feed/
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$

googlebot

Rule Path
Allow /*.css$
Allow /*.js$
Back to top

Comments

  • vestacp autogenerated robots.txt
  • Ralentizamos algunos bots que se suelen volver locos
  • Bloqueo de feeds para crawlers
  • Previene problemas de recursos bloqueados en Google Webmaster Tools
Back to top