topics.barrons.com
robots.txt
Robots Exclusion Standard data for topics.barrons.com
Resource Scan
Scan Details
Site Domain | topics.barrons.com |
Base Domain | barrons.com |
Scan Status | Ok |
Last Scan | 2024-05-09T04:10:37+00:00 |
Next Scan | 2024-05-16T04:10:37+00:00 |
Last Scan
Scanned | 2024-05-09T04:10:37+00:00 |
URL | http://topics.barrons.com/robots.txt |
Redirect | https://www.barrons.com/robots.txt |
Redirect Domain | www.barrons.com |
Redirect Base | barrons.com |
Domain IPs | 18.155.68.108, 18.155.68.15, 18.155.68.24, 18.155.68.42 |
Redirect IPs | 108.156.133.102, 108.156.133.113, 108.156.133.67, 108.156.133.78, 2600:9000:2755:5800:14:c68f:c40:93a1, 2600:9000:2755:6000:14:c68f:c40:93a1, 2600:9000:2755:7800:14:c68f:c40:93a1, 2600:9000:2755:8000:14:c68f:c40:93a1, 2600:9000:2755:8a00:14:c68f:c40:93a1, 2600:9000:2755:be00:14:c68f:c40:93a1, 2600:9000:2755:c400:14:c68f:c40:93a1, 2600:9000:2755:e00:14:c68f:c40:93a1 |
Response IP | 108.156.133.102 |
Found | Yes |
Hash | e619b227530640cdeed98e1332a9f950f90f0137f6e930a349f75efc94ede5c2 |
SimHash | 19303e4ac492 |
Groups
*
Rule | Path |
---|---|
Disallow | /video/search/ |
Disallow | /market-brief/ |
Disallow | /auth/ |
Disallow | /client/login* |
Disallow | /login?* |
Disallow | /public/ |
Disallow | /search?* |
Disallow | /documents/ |
Other Records
Field | Value |
---|---|
sitemap | https://www.barrons.com/bol_news_sitemap.xml |
sitemap | https://www.barrons.com/bol_live_news_sitemap.xml |
sitemap | https://www.barrons.com/sitemaps/web/barrons/en/sitemap_barrons_en_index.xml |
sitemap | https://www.barrons.com/sitemaps/web/afp_news/en/sitemap_afp_news_en_index.xml |
sitemap | https://www.barrons.com/sitemaps/web/afp_news/es/sitemap_afp_news_es_index.xml |
sitemap | https://www.barrons.com/sitemaps/web/barrons_automated/en/sitemap_barrons_automated_en_index.xml |
sitemap | https://www.barrons.com/sitemaps/web/barrons-video/en/sitemap_barrons-video_en_index.xml |
sitemap | https://www.barrons.com/sitemap.xml |
sitemap | https://www.barrons.com/barrons/barrons_quote_index_sitemap.xml |
sitemap | https://www.barrons.com/finder_sitemap.xml |