capitalflowsdata.com
robots.txt

Robots Exclusion Standard data for capitalflowsdata.com

Resource Scan

Scan Details

Site Domain capitalflowsdata.com
Base Domain capitalflowsdata.com
Scan Status Ok
Last Scan2026-03-31T15:09:16+00:00
Next Scan 2026-04-07T15:09:16+00:00

Last Scan

Scanned2026-03-31T15:09:16+00:00
URL https://capitalflowsdata.com/robots.txt
Domain IPs 185.158.133.1
Response IP 185.158.133.1
Found Yes
Hash a4a526db074e0a4bc6779280f951a57b42b43f661045111eee079076a1a0ca80
SimHash 6b045a1081e4

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

*

Rule Path
Allow /
Disallow /auth
Disallow /admin
Disallow /admin/pages
Disallow /profile

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://capitalflowsdata.com/sitemap.xml
sitemap https://capitalflowsdata.com/sitemap-static.xml
sitemap https://capitalflowsdata.com/sitemap-filers.xml
sitemap https://capitalflowsdata.com/sitemap-companies.xml
sitemap https://capitalflowsdata.com/sitemap-news.xml
sitemap https://capitalflowsdata.com/sitemap-pages.xml

Comments

  • Block auth and admin pages from indexing
  • Crawl delay for politeness
  • Sitemap Index (contains all sitemaps)
  • Individual Sitemaps