etfdb.com
robots.txt
Robots Exclusion Standard data for etfdb.com
Resource Scan
Scan Details
| Site Domain | etfdb.com |
| Base Domain | etfdb.com |
| Scan Status | Ok |
| Last Scan | 2024-09-21T14:48:57+00:00 |
| Next Scan | 2024-09-28T14:48:57+00:00 |
Last Scan
| Scanned | 2024-09-21T14:48:57+00:00 |
| URL | https://etfdb.com/robots.txt |
| Domain IPs | 104.26.2.169, 104.26.3.169, 172.67.75.76, 2606:4700:20::681a:2a9, 2606:4700:20::681a:3a9, 2606:4700:20::ac43:4b4c |
| Response IP | 104.26.3.169 |
| Found | Yes |
| Hash | 4dff15924fa5e5fbb0207a1824704de9b7d4c65ede43abe1a9bff313e848fa19 |
| SimHash | 6c6df4d2c8b3 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /etfdb/ |
| Disallow | /feeds/insidefutures/ |
| Disallow | /factsheets/ |
| Disallow | /members/join/free/offers/ |
| Disallow | /members/join/free/info/ |
| Disallow | /members/upgrade/pro/thank-you/ |
| Disallow | /members/join/pro/thank-you/ |
| Disallow | /newsletter/confirmation/ |
| Disallow | /newsletter/confirmed/ |
| Disallow | /widget/ |
| Disallow | /2143012/ |
| Disallow | /21804333245/ |
| Disallow | /wp-content/uploads/ |
| Disallow | */wrap-function |
| Disallow | */contextual-ee |
| Disallow | */wrap-events |
| Disallow | */wrap-raf |
| Disallow | */wrap-timer |
| Disallow | */wrap-xhr |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://etfdb.com/sitemap-index.xml |