Log In Sign Up

urbancontest.com
robots.txt

Robots Exclusion Standard data for urbancontest.com

Resource Scan

Scan Details

Site Domain urbancontest.com
Base Domain urbancontest.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-11-02T03:38:10+00:00
Next Scan 2025-01-31T03:38:10+00:00

Last Successful Scan

Scanned2023-07-12T03:30:41+00:00
URL http://urbancontest.com/robots.txt
Redirect http://www.urbancontest.com/robots.txt
Redirect Domain www.urbancontest.com
Redirect Base urbancontest.com
Domain IPs 5.144.174.142
Redirect IPs 5.144.174.142
Response IP 5.144.174.142
Found Yes
Hash e1f132f3ce695312f01f7905e1f22f957bf25f25c652c70d07c58b0537ede137
SimHash a31e15594fe5

Groups

*

Rule Path
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /modules/
Disallow /plugins/
Disallow /tmp/
Allow /cache/images/
Allow /plugins/system/ajaxinfinity/ajax-loader.gif
Back to top

Other Records

Field Value
sitemap https://www.urbancontest.com/index.php?option=com_osmap&view=xml&tmpl=component&id=1
Back to top

Comments

  • If the Joomla site is installed within a folder such as at
  • e.g. www.example.com/joomla/ the robots.txt file MUST be
  • moved to the site root at e.g. www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to the disallowed
  • path, e.g. the Disallow rule for the /administrator/ folder
  • MUST be changed to read Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://tool.motoricerca.info/robots-checker.phtml
Back to top