worldnumerology.com
robots.txt

Robots Exclusion Standard data for worldnumerology.com

Resource Scan

Scan Details

Site Domain worldnumerology.com
Base Domain worldnumerology.com
Scan Status Ok
Last Scan2026-02-25T13:43:17+00:00
Next Scan 2026-03-27T13:43:17+00:00

Last Scan

Scanned2026-02-25T13:43:17+00:00
URL https://worldnumerology.com/robots.txt
Redirect https://www.worldnumerology.com/robots.txt
Redirect Domain www.worldnumerology.com
Redirect Base worldnumerology.com
Domain IPs 104.26.12.10, 104.26.13.10, 172.67.70.82, 2606:4700:20::681a:c0a, 2606:4700:20::681a:d0a, 2606:4700:20::ac43:4652
Redirect IPs 104.26.12.10, 104.26.13.10, 172.67.70.82, 2606:4700:20::681a:c0a, 2606:4700:20::681a:d0a, 2606:4700:20::ac43:4652
Response IP 104.26.12.10
Found Yes
Hash c26732066220d3a20e94586e94223c411f4035e4d5786767807e843ab180e934
SimHash 703ec9c0e691

Groups

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

bingbot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

applebot

Rule Path
Allow /

semrushbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

semrushbot-sa

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

majesticbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

mj12bot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

serpstatbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

seokicks-robot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

*

Rule Path
Allow /css/
Allow /styles/
Allow /scripts/
Allow /icons/
Allow /cdn-cgi/
Allow /admin/heartdesire-numerology.php
Disallow /admin/
Disallow /admin_reportadmin_old/
Disallow /backup/
Disallow /bkfiles/
Disallow /Builds/
Disallow /desktop/
Disallow /dev/
Disallow /new/
Disallow /Download2Cnet/
Disallow /Downloads/
Disallow /email/
Disallow /hans/
Disallow /iframeproject/
Disallow /iframeproject_iphone/
Disallow /iframeproject_oct_27/
Disallow /MAC/
Disallow /master_program_chart_bk/
Disallow /numerology-services.com/
Disallow /numerologyworld.com/
Disallow /oldx/
Disallow /ry/
Disallow /Saravana/
Disallow /test_folder/

Other Records

Field Value
sitemap https://www.worldnumerology.com/sitemap.xml

Comments

  • ALLOWED SEARCH ENGINE BOTS (SEO BENEFIT)
  • SEO TOOL BOTS (THROTTLED TO REDUCE LOAD)
  • AI TRAINING BOTS (FULLY BLOCKED)
  • SCRAPERS / NON-SEO BOTS (FULLY BLOCKED)
  • DEFAULT RULES FOR ALL OTHER BOTS
  • Allow essential assets for SEO rendering
  • Block sensitive, backend, or duplicate folders
  • SITEMAP