Log In Sign Up

portalnautico.com.br
robots.txt

Robots Exclusion Standard data for portalnautico.com.br

Resource Scan

Scan Details

Site Domain portalnautico.com.br
Base Domain portalnautico.com.br
Scan Status Ok
Last Scan2024-06-05T05:04:05+00:00
Next Scan 2024-06-12T05:04:05+00:00

Last Scan

Scanned2024-06-05T05:04:05+00:00
URL https://portalnautico.com.br/robots.txt
Redirect https://www.portalnautico.com.br/robots.txt
Redirect Domain www.portalnautico.com.br
Redirect Base portalnautico.com.br
Domain IPs 104.21.36.136, 172.67.194.213, 2606:4700:3033::6815:2488, 2606:4700:3037::ac43:c2d5
Redirect IPs 104.21.36.136, 172.67.194.213, 2606:4700:3033::6815:2488, 2606:4700:3037::ac43:c2d5
Response IP 104.21.36.136
Found Yes
Hash d92435215e3b48c42109fc0921ceee9b0e2c49b93daef3a44477a78fc1614b4d
SimHash 2a185dd09650

Groups

*

Rule Path
Allow /*?
Disallow /inc/
Disallow /includes/
Allow /*.js$
Allow /*.css$
Allow /*.png$
Allow /*.jpg$
Allow /*.gif$
Allow /*.ttf$
Allow /*.woff$
Allow /*.svg$
Disallow /*.php$
Disallow /*.phtml$

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /

baidu

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

bspider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

yandex

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://www.portalnautico.com.br/mapeamentodosite/sitemap.xml
Back to top

Comments

  • robots.txt for http://www.portalnautico.com.br/
  • Website Sitemap
  • Crawlers Setup
  • Allowable Index
  • Directories
  • Paths (no clean URLs)
  • Filtros
  • Disallow: /*&SID=
  • Disallow: /*&abrangencia=
  • Disallow: /*?SID=
  • Disallow: /*?abrangencia=
  • Block Bad Crawlers
  • end of robots.txt
Back to top