investinglive.com
robots.txt

Robots Exclusion Standard data for investinglive.com

Resource Scan

Scan Details

Site Domain investinglive.com
Base Domain investinglive.com
Scan Status Ok
Last Scan2025-12-22T03:55:45+00:00
Next Scan 2025-12-29T03:55:45+00:00

Last Scan

Scanned2025-12-22T03:55:45+00:00
URL https://investinglive.com/robots.txt
Domain IPs 65.8.131.127, 65.8.131.19, 65.8.131.21, 65.8.131.51
Response IP 3.170.229.19
Found Yes
Hash 28cf1c38f1fb8597c509ae87c2d019fa19c397fcec51ada12eb88370d0506a60
SimHash 003aca5301b2

Groups

*

Rule Path
Disallow /search-results
Disallow /terms/search-results
Disallow /tag/search-results
Disallow /signalr/
Disallow /*.pdf$

ai2bot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

deepseekbot

Rule Path
Disallow /

deepseek

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

qwen-agent

Rule Path
Disallow /

Other Records

Field Value
sitemap https://investinglive.com/sitemap.xml
sitemap https://investinglive.com/latest.xml
sitemap https://investinglive.com/articles-sitemap-index.xml

Comments

  • Global Rules for All Bots
  • Bot-Specific Restrictions
  • Sitemap Directives