Log In Sign Up

etsab.upc.edu
robots.txt

Robots Exclusion Standard data for etsab.upc.edu

Resource Scan

Scan Details

Site Domain etsab.upc.edu
Base Domain upc.edu
Scan Status Failed
Failure StageFetching resource.
Failure ReasonRequest timed out.
Last Scan2025-10-04T20:47:05+00:00
Next Scan 2025-10-18T20:47:05+00:00

Last Successful Scan

Scanned2025-08-27T10:17:23+00:00
URL https://etsab.upc.edu/robots.txt
Domain IPs 147.83.194.28
Response IP 147.83.194.28
Found Yes
Hash 758cf0c38f804028be14183ff35c3d411f4538abe473e67eac69366882a1c430
SimHash af71ab554d65

Groups

*

Rule Path
Disallow */noindex-upc/*
Disallow /*sendto_form$
Disallow /*folder_factories$
Disallow */ca/shared/*
Disallow */es/shared/*
Disallow */en/shared/*
Disallow */plantilles/*
Disallow */ca/banners-ca/*
Disallow */es/banners-es/*
Disallow */en/banners-en/*
Disallow */ca/logosfooter-ca/*
Disallow */es/logosfooter-es/*
Disallow */en/logosfooter-en/*

googlebot

Rule Path
Disallow /*?
Disallow /*atct_album_view$
Disallow /*folder_factories$
Disallow /*folder_summary_view$
Disallow /*login_form$
Disallow /*mail_password_form$
Disallow /%40%40search
Disallow /*search_rss$
Disallow /*sendto_form$
Disallow /*summary_view$
Disallow /*thumbnail_view$
Disallow /*view$
Back to top

Other Records

Field Value
sitemap https://etsab.upc.edu/sitemap.xml.gz
Back to top

Comments

  • Define access-restrictions for robots/spiders
  • http://www.robotstxt.org/wc/norobots.html
  • By default we allow robots to access all areas of our site
  • already accessible to anonymous users
  • Add Googlebot-specific syntax extension to exclude forms
  • that are repeated for each piece of content in the site
  • the wildcard is only supported by Googlebot
  • http://www.google.com/support/webmasters/bin/answer.py?answer=40367&ctx=sibling
Back to top