Log In Sign Up

music.org
robots.txt

Robots Exclusion Standard data for music.org

Resource Scan

Scan Details

Site Domain music.org
Base Domain music.org
Scan Status Ok
Last Scan2024-09-24T20:10:35+00:00
Next Scan 2024-10-08T20:10:35+00:00

Last Scan

Scanned2024-09-24T20:10:35+00:00
URL https://music.org/robots.txt
Redirect https://www.music.org/robots.txt
Redirect Domain www.music.org
Redirect Base music.org
Domain IPs 104.26.8.250, 104.26.9.250, 172.67.71.225, 2606:4700:20::681a:8fa, 2606:4700:20::681a:9fa, 2606:4700:20::ac43:47e1
Redirect IPs 104.26.8.250, 104.26.9.250, 172.67.71.225, 2606:4700:20::681a:8fa, 2606:4700:20::681a:9fa, 2606:4700:20::ac43:47e1
Response IP 172.67.71.225
Found Yes
Hash f2c559ee67667b147b33acfee68c3a440d5c5c07d4756763bc40e88c951d9eec
SimHash ab0f1d78c3bd

Groups

*

Rule Path
Allow /media/system/css/
Allow /media/system/js/
Allow /media/jui/css/
Allow /media/jui/js/
Allow /templates/shaper_university/css/
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Allow /modules/mod_vertical_menu/cache/
Disallow /plugins/
Allow /plugins/system/helix/js/
Disallow /tmp/

wikido

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

clark-crawler2

Rule Path
Disallow /

*

Rule Path
Allow /sitemap-4seo

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.music.org/sitemap-4seo.xml
Back to top

Comments

  • If the Joomla site is installed within a folder
  • eg www.example.com/joomla/ then the robots.txt file
  • MUST be moved to the site root
  • eg www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to all of the
  • paths.
  • eg the Disallow rule for the /administrator/ folder MUST
  • be changed to read
  • Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://tool.motoricerca.info/robots-checker.phtml
  • 4SEO-sitemap: sitemap - added on 2024-05-17 17:44:37 - Please do not modify manually.
  • /4SEO-sitemap
  • 4SEO-aiBots
  • /4SEO-aiBots
Back to top