Log In Sign Up

rubi.cat
robots.txt

Robots Exclusion Standard data for rubi.cat

Resource Scan

Scan Details

Site Domain rubi.cat
Base Domain rubi.cat
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2026-02-04T08:26:49+00:00
Next Scan 2026-02-18T08:26:49+00:00

Last Successful Scan

Scanned2025-12-28T08:24:16+00:00
URL https://rubi.cat/robots.txt
Redirect https://www.rubi.cat/robots.txt
Redirect Domain www.rubi.cat
Redirect Base rubi.cat
Domain IPs 35.157.26.13
Redirect IPs 18.153.162.82
Response IP 35.157.26.13
Found Yes
Hash 411462bd872753af40e64af134d098f12fd91e0bf05f8aa5e29aa92e71911101
SimHash a861ab5d0d61

Groups

*

Rule Path
Disallow /fitxers/
Disallow /fitxers/documents-ok/area-de-serveis-territorials/mobilitat/pla-de-mobilitat-urbana-sostenible-de-rubi-pmus
Disallow /ca/portada/*
Disallow /es/portada/*
Disallow /*header
Disallow /*header
Disallow /*%40%40embedded
Disallow /*image_view_fullscreen$
Disallow /ca/nadal/programacio/*
Disallow /es/navidad/programacion/*
Disallow /es/navidad/los-reyes-de-oriente/*
Disallow /ca/nadal/els-reis-d-orient/*
Disallow /ca/nadal/els-reis-d-orient/el-mag-rubisenc
Disallow /es/navidad/los-reyes-de-oriente/el-mago-rubisenc
Disallow /ca/nadal/els-reis-d-orient/arribada-de-ses-majestats-els-reis-d-orient
Disallow /es/navidad/los-reyes-de-oriente/llegada-de-sus-majestades-los-reyes-de-oriente

googlebot

Rule Path
Disallow /*?
Disallow /*atct_album_view$
Disallow /*folder_factories$
Disallow /*folder_summary_view$
Disallow /*login_form$
Disallow /*mail_password_form$
Disallow /%40%40search
Disallow /*search_rss$
Disallow /*sendto_form$
Disallow /*summary_view$
Disallow /*thumbnail_view$
Disallow /*view$
Back to top

Other Records

Field Value
sitemap https://www.rubi.cat/sitemap.xml.gz
Back to top

Comments

  • Define access-restrictions for robots/spiders
  • http://www.robotstxt.org/wc/norobots.html
  • By default we allow robots to access all areas of our site
  • already accessible to anonymous users
  • Add Googlebot-specific syntax extension to exclude forms
  • that are repeated for each piece of content in the site
  • the wildcard is only supported by Googlebot
  • http://www.google.com/support/webmasters/bin/answer.py?answer=40367&ctx=sibling
Back to top