Log In Sign Up

tre-to.jus.br
robots.txt

Robots Exclusion Standard data for tre-to.jus.br

Resource Scan

Scan Details

Site Domain tre-to.jus.br
Base Domain tre-to.jus.br
Scan Status Ok
Last Scan2024-10-28T02:14:07+00:00
Next Scan 2024-11-27T02:14:07+00:00

Last Scan

Scanned2024-10-28T02:14:07+00:00
URL https://tre-to.jus.br/robots.txt
Redirect https://www.tre-to.jus.br/robots.txt
Redirect Domain www.tre-to.jus.br
Redirect Base tre-to.jus.br
Domain IPs 23.209.46.146, 23.209.46.155, 2600:1413:b000:6::17d5:2bc7, 2600:1413:b000:6::17d5:2be0
Redirect IPs 23.209.46.146, 23.209.46.155, 2600:1413:b000:6::17d5:2bc7, 2600:1413:b000:6::17d5:2be0
Response IP 23.209.46.146
Found Yes
Hash a5530468e041f79895d505fbb6175ff070544b352c8086549a458af671789e37
SimHash 2d35c0574d65

Groups

*

Rule Path
Disallow /*?
Disallow /*atct_album_view$
Disallow /*folder_factories$
Disallow /*folder_summary_view$
Disallow /*login_form$
Disallow /*mail_password_form$
Disallow /*search
Disallow /*search_rss
Disallow /*searchRSS
Disallow /*updated_search
Disallow /*sendto_form$
Disallow /institucional/sala-de-imprensa/sintonia/
Disallow /arquivos
Disallow /eleicoes/eleicoes-2018/jec2018
Disallow /o-tse/sistemas/codigo-fonte-de-sistemas-corporativos-auditse
Disallow /*list-subjects$
Disallow /*search-es$
Disallow /*advanced-search$
Disallow /*search-codigo-eleitoral$
Disallow /*monocraticas-search$
Disallow /*rss
Back to top

Other Records

Field Value
sitemap /sitemap.xml.gz
Back to top

Comments

  • Define access-restrictions for robots/spiders
  • http://www.robotstxt.org/wc/norobots.html
  • By default we allow robots to access all areas of our site accessible to
  • anonymous users, except for search, which burns our CPU for no reason.
  • Block all URLs including query strings (? pattern) - contentish objects expose query string only for actions or status reports which
  • might confuse search results.
  • This will also block ?set_language
  • Add Googlebot-specific syntax extension to exclude forms
  • that are repeated for each piece of content in the site
  • the wildcard is only supported by Googlebot
  • http://www.google.com/support/webmasters/bin/answer.py?answer=40367&ctx=sibling
Back to top