online.barrons.com
robots.txt
Robots Exclusion Standard data for online.barrons.com
Resource Scan
Scan Details
Site Domain | online.barrons.com |
Base Domain | barrons.com |
Scan Status | Ok |
Last Scan | 2024-05-12T06:55:13+00:00 |
Next Scan | 2024-05-19T06:55:13+00:00 |
Last Scan
Scanned | 2024-05-12T06:55:13+00:00 |
URL | https://online.barrons.com/robots.txt |
Redirect | https://www.barrons.com/robots.txt |
Redirect Domain | www.barrons.com |
Redirect Base | barrons.com |
Domain IPs | 18.155.68.108, 18.155.68.15, 18.155.68.24, 18.155.68.42, 2600:9000:23d2:200:14:ca3f:6000:93a1, 2600:9000:23d2:3800:14:ca3f:6000:93a1, 2600:9000:23d2:5e00:14:ca3f:6000:93a1, 2600:9000:23d2:6400:14:ca3f:6000:93a1, 2600:9000:23d2:9400:14:ca3f:6000:93a1, 2600:9000:23d2:a00:14:ca3f:6000:93a1, 2600:9000:23d2:ec00:14:ca3f:6000:93a1, 2600:9000:23d2:f600:14:ca3f:6000:93a1 |
Redirect IPs | 108.156.133.102, 108.156.133.113, 108.156.133.67, 108.156.133.78, 2600:9000:2755:5000:14:c68f:c40:93a1, 2600:9000:2755:5200:14:c68f:c40:93a1, 2600:9000:2755:800:14:c68f:c40:93a1, 2600:9000:2755:9200:14:c68f:c40:93a1, 2600:9000:2755:b000:14:c68f:c40:93a1, 2600:9000:2755:ba00:14:c68f:c40:93a1, 2600:9000:2755:f400:14:c68f:c40:93a1, 2600:9000:2755:f600:14:c68f:c40:93a1 |
Response IP | 108.156.133.78 |
Found | Yes |
Hash | e619b227530640cdeed98e1332a9f950f90f0137f6e930a349f75efc94ede5c2 |
SimHash | 19303e4ac492 |
Groups
*
Rule | Path |
---|---|
Disallow | /video/search/ |
Disallow | /market-brief/ |
Disallow | /auth/ |
Disallow | /client/login* |
Disallow | /login?* |
Disallow | /public/ |
Disallow | /search?* |
Disallow | /documents/ |
Other Records
Field | Value |
---|---|
sitemap | https://www.barrons.com/bol_news_sitemap.xml |
sitemap | https://www.barrons.com/bol_live_news_sitemap.xml |
sitemap | https://www.barrons.com/sitemaps/web/barrons/en/sitemap_barrons_en_index.xml |
sitemap | https://www.barrons.com/sitemaps/web/afp_news/en/sitemap_afp_news_en_index.xml |
sitemap | https://www.barrons.com/sitemaps/web/afp_news/es/sitemap_afp_news_es_index.xml |
sitemap | https://www.barrons.com/sitemaps/web/barrons_automated/en/sitemap_barrons_automated_en_index.xml |
sitemap | https://www.barrons.com/sitemaps/web/barrons-video/en/sitemap_barrons-video_en_index.xml |
sitemap | https://www.barrons.com/sitemap.xml |
sitemap | https://www.barrons.com/barrons/barrons_quote_index_sitemap.xml |
sitemap | https://www.barrons.com/finder_sitemap.xml |