Log In Sign Up

nartwor.ru
robots.txt

Robots Exclusion Standard data for nartwor.ru

Resource Scan

Scan Details

Site Domain nartwor.ru
Base Domain nartwor.ru
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-02-03T05:11:10+00:00
Next Scan 2025-05-04T05:11:10+00:00

Last Successful Scan

Scanned2024-06-16T05:07:55+00:00
URL https://nartwor.ru/robots.txt
Domain IPs 31.31.198.119
Response IP 31.31.198.119
Found Yes
Hash 4b46c27e16f80f9a9bcea1227408ca8e4c8f1a6fbae9ce5ef7adb1ae4de2e09b
SimHash a31e1558836d

Groups

*

Rule Path
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /images/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /media/
Disallow /modules/
Disallow /plugins/
Disallow /templates/
Disallow /tmp/

Other Records

Field Value
crawl-delay 1000

claudebot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /
Back to top

Comments

  • If the Joomla site is installed within a folder such as at
  • e.g. www.example.com/joomla/ the robots.txt file MUST be
  • moved to the site root at e.g. www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to the disallowed
  • path, e.g. the Disallow rule for the /administrator/ folder
  • MUST be changed to read Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://tool.motoricerca.info/robots-checker.phtml
  • blocks bot 2024-04-17
  • blocks bot 2024-03-05
  • blocks bot 2024-03-24
Back to top