Log In Sign Up

clayconews.com
robots.txt

Robots Exclusion Standard data for clayconews.com

Resource Scan

Scan Details

Site Domain clayconews.com
Base Domain clayconews.com
Scan Status Ok
Last Scan2026-02-07T14:57:20+00:00
Next Scan 2026-02-14T14:57:20+00:00

Last Scan

Scanned2026-02-07T14:57:20+00:00
URL https://clayconews.com/robots.txt
Redirect https://www.clayconews.com/robots.txt
Redirect Domain www.clayconews.com
Redirect Base clayconews.com
Domain IPs 104.26.8.176, 104.26.9.176, 172.67.71.240, 2606:4700:20::681a:8b0, 2606:4700:20::681a:9b0, 2606:4700:20::ac43:47f0
Redirect IPs 104.26.8.176, 104.26.9.176, 172.67.71.240, 2606:4700:20::681a:8b0, 2606:4700:20::681a:9b0, 2606:4700:20::ac43:47f0
Response IP 104.26.8.176
Found Yes
Hash 31f9863bf3ce7bf145643969aa29f09eb396d74b839a4cb8aefb08bf7663de39
SimHash 621f255a8b65

Groups

amazonbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

*

Rule Path
Disallow /ad-clicks
Disallow /administrator/
Disallow /api/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /modules/
Disallow /plugins/
Disallow /tmp/
Allow /media/vendor/bootstrap/js/
Allow /media/system/css/
Allow /media/system/css/joomla-fontawesome.min.css
Allow /media/system/js/
Allow /media/vendor/joomla-custom-elements/css/
Allow /media/vendor/jquery/js/
Allow /media/mod_latestnewsenhanced/css/
Allow /plugins/system/helixultimate/assets/
Allow /plugins/system/helixultimate/assets/css/
Allow /templates/shaper_helixultimate/css/
Allow /templates/shaper_helixultimate/css/presets/
Allow /plugins/system/jce/css/
Allow /plugins/system/wf_responsive_widgets/css/
Allow /plugins/system/wf_responsive_widgets/js/
Allow /images/
Allow /images2/
Allow /images3/
Allow /ads/preferences/
Allow /dtt/k
Allow /gpt/
Allow /pagead/show_ads.js
Allow /pagead/html/
Allow /pagead/js/
Allow /pagead/js/adsbygoogle.js
Allow /pagead/*/show_ads_impl.js
Allow /pagead/managed/js/adsense/
Allow /static/glade.js
Allow /static/glade/
Allow /tag/js/
Back to top

Other Records

Field Value
sitemap https://www.clayconews.com/sitemap-xml
Back to top

Comments

  • If the Joomla site is installed within a folder
  • eg www.example.com/joomla/ then the robots.txt file
  • MUST be moved to the site root
  • eg www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to all of the
  • paths.
  • eg the Disallow rule for the /administrator/ folder MUST
  • be changed to read
  • Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://tool.motoricerca.info/robots-checker.phtml
Back to top