Log In Sign Up

netsiter.com
robots.txt

Robots Exclusion Standard data for netsiter.com

Resource Scan

Scan Details

Site Domain netsiter.com
Base Domain netsiter.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-24T11:38:58+00:00
Next Scan 2025-10-08T11:38:58+00:00

Last Successful Scan

Scanned2025-08-17T05:50:48+00:00
URL https://netsiter.com/robots.txt
Redirect https://www.netsiter.com/robots.txt
Redirect Domain www.netsiter.com
Redirect Base netsiter.com
Domain IPs 89.41.38.183
Redirect IPs 89.41.38.183
Response IP 89.41.38.183
Found Yes
Hash b219f346f1045a47d07510cd8c7bf63655855d2fd85fd26800ae3963c8d2c152
SimHash f21e1d1b49f4

Groups

*

Rule Path
Allow /*.js*
Allow /*.css*
Allow /*.png*
Allow /*.jpg*
Allow /*.gif*
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /tmp/
Back to top

Comments

  • If the Joomla site is installed within a folder
  • eg www.example.com/joomla/ then the robots.txt file
  • MUST be moved to the site root
  • eg www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to all of the
  • paths.
  • eg the Disallow rule for the /administrator/ folder MUST
  • be changed to read
  • Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • https://www.robotstxt.org/orig.html
Back to top