Log In Sign Up

www.gov.br
robots.txt

Robots Exclusion Standard data for www.gov.br

Resource Scan

Scan Details

Site Domain www.gov.br
Base Domain www.gov.br
Scan Status Ok
Last Scan2024-09-24T18:22:49+00:00
Next Scan 2024-10-24T18:22:49+00:00

Last Scan

Scanned2024-09-24T18:22:49+00:00
URL https://www.gov.br/robots.txt
Domain IPs 161.148.164.31
Response IP 161.148.164.31
Found Yes
Hash 665417a8e7d2184fe1effefd6a242111682a9e59e94c4aa8af17e9c5487a090a
SimHash 2f510fdf8dc5

Groups

yandex

Rule Path
Disallow /ebserh
Disallow /mre

*

Rule Path
Disallow /economia/pt-br/internet/*
Disallow /*sendto_form$
Disallow /*folder_factories$
Disallow /ebserh/*?
Disallow /ebserh/*atct_album_view$
Disallow /ebserh/*folder_summary_view$
Disallow /ebserh/*login_form$
Disallow /ebserh/*mail_password_form$
Disallow /ebserh/%40%40search
Disallow /ebserh/*search_rss$
Disallow /ebserh/*summary_view$
Disallow /ebserh/*thumbnail_view$
Disallow /ebserh/*view$
Disallow /ebserh/*%40%40download/file$
Disallow /ebserh/*portal_javascripts*
Disallow /ebserh/*portal_css*
Disallow /ebserh/*%2B%2Btheme%2B%2B*
Disallow /ebserh/*%2B%2Bresource%2B%2B*
Disallow /ebserh/*%40%40configuracoes_view*
Disallow /ebserh/*.css$
Disallow /ebserh/*.gif$
Disallow /ebserh/*.jpeg$
Disallow /ebserh/*.jpg$
Disallow /ebserh/*.JPG$
Disallow /ebserh/*.js$
Disallow /ebserh/*.pdf$
Disallow /ebserh/*.png$
Disallow /ebserh/*.zip$
Disallow /mre/*?
Disallow /mre/*atct_album_view$
Disallow /mre/*folder_summary_view$
Disallow /mre/*login_form$
Disallow /mre/*mail_password_form$
Disallow /mre/%40%40search
Disallow /mre/*search_rss$
Disallow /mre/*summary_view$
Disallow /mre/*thumbnail_view$
Disallow /mre/*view$
Disallow /mre/*%40%40download/file$
Disallow /mre/*portal_javascripts*
Disallow /mre/*portal_css*
Disallow /mre/*%2B%2Btheme%2B%2B*
Disallow /mre/*%2B%2Bresource%2B%2B*
Disallow /mre/*%40%40configuracoes_view*
Disallow /mre/*.css$
Disallow /mre/*.gif$
Disallow /mre/*.jpeg$
Disallow /mre/*.jpg$
Disallow /mre/*.JPG$
Disallow /mre/*.js$
Disallow /mre/*.pdf$
Disallow /mre/*.png$
Disallow /mre/*.zip$
Back to top

Other Records

Field Value
sitemap https://www.gov.br/sitemap.xml.gz
Back to top

Comments

  • Define access-restrictions for robots/spiders
  • http://www.robotstxt.org/wc/norobots.html
  • By default we allow robots to access all areas of our site
  • already accessible to anonymous users
  • Add Googlebot-specific syntax extension to exclude forms
  • that are repeated for each piece of content in the site
  • the wildcard is only supported by Googlebot
  • http://www.google.com/support/webmasters/bin/answer.py?answer=40367&ctx=sibling
  • User-Agent: Googlebot
Back to top