Log In Sign Up

al.pi.leg.br
robots.txt

Robots Exclusion Standard data for al.pi.leg.br

Resource Scan

Scan Details

Site Domain al.pi.leg.br
Base Domain al.pi.leg.br
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-11-08T05:44:56+00:00
Next Scan 2026-01-07T05:44:56+00:00

Last Successful Scan

Scanned2025-08-16T20:14:47+00:00
URL https://al.pi.leg.br/robots.txt
Redirect https://www.al.pi.leg.br/robots.txt
Redirect Domain www.al.pi.leg.br
Redirect Base al.pi.leg.br
Domain IPs 201.54.51.175
Redirect IPs 201.54.51.175
Response IP 201.54.51.175
Found Yes
Hash 7e1f22deb6eaa03c19e84f73e141c6201aef30f137be2f55476497fc9b734349
SimHash be710b574d65

Groups

*

Rule Path
Disallow

googlebot

Rule Path
Disallow /*sendto_form$
Disallow /*search_form$
Disallow /*folder_factories$
Disallow /*login_form
Disallow /*folder_contents$
Back to top

Other Records

Field Value
sitemap /sitemap.xml.gz
Back to top

Comments

  • Define access-restrictions for robots/spiders
  • http://www.robotstxt.org/wc/norobots.html
  • By default we allow robots to access all areas of our site
  • already accessible to anonymous users
  • Add Googlebot-specific syntax extension to exclude forms
  • that are repeated for each piece of content in the site
  • the wildcard is only supported by Googlebot
  • http://www.google.com/support/webmasters/bin/answer.py?answer=40367&ctx=sibling
Back to top