Log In Sign Up

cmelitegroup.com
robots.txt

Robots Exclusion Standard data for cmelitegroup.com

Resource Scan

Scan Details

Site Domain cmelitegroup.com
Base Domain cmelitegroup.com
Scan Status Ok
Last Scan2025-11-28T20:41:18+00:00
Next Scan 2025-12-28T20:41:18+00:00

Last Scan

Scanned2025-11-28T20:41:18+00:00
URL https://cmelitegroup.com/robots.txt
Redirect https://www.cmelitegroup.com/robots.txt
Redirect Domain www.cmelitegroup.com
Redirect Base cmelitegroup.com
Domain IPs 192.124.249.78
Redirect IPs 192.124.249.78
Response IP 192.124.249.78
Found Yes
Hash bf3666854d65c9d261a3457c3fb92f3d439281780f7a0805982c1e9677924b52
SimHash 6106f992e567

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-includes/
Disallow /xmlrpc.php
Disallow /?s=
Disallow /search/
Disallow /cgi-bin/
Disallow /*/feed/
Disallow /*/rss2/

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

applebot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

slurp

Rule Path
Allow /

google-extended

Rule Path
Allow /

applebot-extended

Rule Path
Allow /

gptbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /

claudebot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

applebot

Rule Path
Allow /

petalbot

Rule Path
Allow /

coccocbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

bravebot

Rule Path
Allow /

ccbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

youbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.cmelitegroup.com/sitemap_index.xml
Back to top

Comments

  • ================================
  • CM Elite Group — robots.txt
  • ================================
  • --- WordPress hygiene (applies to everyone) ---
  • Optional (reduce thin/duplicate pages):
  • --- Allow major search engines (within hygiene above) ---
  • --- Allow reputable AI crawlers for visibility (within hygiene above) ---
  • --- Block aggressive or non-beneficial crawlers ---
  • --- Sitemap ---
Back to top