Log In Sign Up

unaerp.br
robots.txt

Robots Exclusion Standard data for unaerp.br

Resource Scan

Scan Details

Site Domain unaerp.br
Base Domain unaerp.br
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2026-01-13T17:36:48+00:00
Next Scan 2026-04-13T17:36:48+00:00

Last Successful Scan

Scanned2025-08-23T21:02:09+00:00
URL https://unaerp.br/robots.txt
Domain IPs 189.112.110.68
Response IP 189.112.110.68
Found Yes
Hash 32f79ad5fc70f5284b3119e5324fa64932bdb978714b8c02252dbbbc7fa735ea
SimHash 3a1f5c1a81fc

Groups

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

zeus 32297 webster pro v2.9 win32

Rule Path
Disallow /

linkdexbot/2.1

Rule Path
Disallow /

whatsapp

Rule Path
Disallow /

bdcbot

Rule Path
Disallow /

nutch spider/nutch-1.12

Rule Path
Disallow /

snowhaze search/1.0

Rule Path
Disallow /

gluten free crawler/1.0

Rule Path
Disallow /

pcore-http/v0.44.0

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

a6-indexer

Rule Path
Disallow /

phantomjs

Rule Path
Disallow /

phantomjs/2.1.1

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

*

Rule Path
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /modules/
Disallow /plugins/
Disallow /tmp/
Back to top

Comments

  • If the Joomla site is installed within a folder
  • eg www.example.com/joomla/ then the robots.txt file
  • MUST be moved to the site root
  • eg www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to all of the
  • paths.
  • eg the Disallow rule for the /administrator/ folder MUST
  • be changed to read
  • Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://tool.motoricerca.info/robots-checker.phtml
Back to top