Log In Sign Up

badalona.cat
robots.txt

Robots Exclusion Standard data for badalona.cat

Resource Scan

Scan Details

Site Domain badalona.cat
Base Domain badalona.cat
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-06-02T06:44:32+00:00
Next Scan 2025-08-31T06:44:32+00:00

Last Successful Scan

Scanned2024-01-17T06:27:51+00:00
URL https://badalona.cat/robots.txt
Redirect https://www.badalona.cat/robots.txt
Redirect Domain www.badalona.cat
Redirect Base badalona.cat
Domain IPs 18.156.16.255
Redirect IPs 18.156.16.255
Response IP 18.156.16.255
Found Yes
Hash 4018cc4a17c3f4e752b86dc11851a684deec46bbf29fe2eba91e8a02c450c220
SimHash ac71ab554d61

Groups

*

Rule Path
Disallow /*portada/
Disallow /equipaments/
Disallow /paperera-de-reciclatge/
Disallow /ca/recursos/
Disallow /es/recursos/

googlebot

Rule Path
Disallow /ca/recursos/
Disallow /es/recursos/
Disallow /paperera-de-reciclatge/
Disallow /equipaments/
Disallow /*portada/
Disallow /*?
Disallow /*atct_album_view$
Disallow /*folder_factories$
Disallow /*folder_summary_view$
Disallow /*login_form$
Disallow /*mail_password_form$
Disallow /%40%40search
Disallow /*search_rss$
Disallow /*sendto_form$
Disallow /*summary_view$
Disallow /*thumbnail_view$
Disallow /*view$
Back to top

Other Records

Field Value
sitemap https://www.badalona.cat/sitemap.xml.gz
Back to top

Comments

  • Define access-restrictions for robots/spiders
  • http://www.robotstxt.org/wc/norobots.html
  • By default we allow robots to access all areas of our site
  • already accessible to anonymous users
  • Add Googlebot-specific syntax extension to exclude forms
  • that are repeated for each piece of content in the site
  • the wildcard is only supported by Googlebot
  • http://www.google.com/support/webmasters/bin/answer.py?answer=40367&ctx=sibling
Back to top