interactive.wsj.com
robots.txt
Robots Exclusion Standard data for interactive.wsj.com
Resource Scan
Scan Details
Site Domain | interactive.wsj.com |
Base Domain | wsj.com |
Scan Status | Ok |
Last Scan | 2024-11-14T16:54:09+00:00 |
Next Scan | 2024-11-21T16:54:09+00:00 |
Last Scan
Scanned | 2024-11-14T16:54:09+00:00 |
URL | https://interactive.wsj.com/robots.txt |
Redirect | https://www.wsj.com/robots.txt |
Redirect Domain | www.wsj.com |
Redirect Base | wsj.com |
Domain IPs | 3.165.82.122, 3.165.82.129, 3.165.82.47, 3.165.82.57 |
Redirect IPs | 13.35.210.120, 13.35.210.65, 13.35.210.75, 13.35.210.78, 2600:9000:2078:1c00:3:4b0:de80:93a1, 2600:9000:2078:3600:3:4b0:de80:93a1, 2600:9000:2078:6200:3:4b0:de80:93a1, 2600:9000:2078:6c00:3:4b0:de80:93a1, 2600:9000:2078:8600:3:4b0:de80:93a1, 2600:9000:2078:a400:3:4b0:de80:93a1, 2600:9000:2078:ca00:3:4b0:de80:93a1, 2600:9000:2078:da00:3:4b0:de80:93a1 |
Response IP | 13.35.210.78 |
Found | Yes |
Hash | 88f031f0aa7f6d06f9eece81b8b0dc075c8a4e5b23db8688ec0a466b2cdd85d0 |
SimHash | 509850df8670 |
Groups
*
Rule | Path |
---|---|
Disallow | /article_email/* |
Disallow | /user/* |
Disallow | /pdf/documents/* |
Disallow | /login/* |
Disallow | /acct/* |
Disallow | /msgcenter/* |
Disallow | /setup/* |
Disallow | /marketing/* |
Disallow | /public/article/* |
Disallow | /public/search/ |
Disallow | /public/search* |
Disallow | /search* |
Disallow | /public/page/wsj-x-marketing.html |
Disallow | /public/page/news-media-marketing.html |
Disallow | /public/page/0_0_WP_RT_MARKETING.html |
Disallow | /news/articles/SB2* |
Disallow | /news/articles/SB3* |
Disallow | /news/articles/SB4* |
Disallow | /articles/SB2* |
Disallow | /articles/SB3* |
Disallow | /articles/SB4* |
Disallow | /article/AP* |
Disallow | /article/BT-CO* |
Disallow | /article/DN-CO* |
Disallow | /article/PR-CO* |
Disallow | /article/HUG* |
Disallow | /video/search/* |
Disallow | /articles/BT-CO* |
Disallow | /articles/DN-CO* |
Disallow | /articles/PR-CO* |
Disallow | /news/articles/BT-CO* |
Disallow | /news/articles/DN-CO* |
Disallow | /news/articles/PR-CO* |
Disallow | /catchup/* |
Disallow | /articles/the-meaning-behind-juneteenth-11592413234 |
Disallow | /emailservice/* |
Disallow | /emailsignup/* |
Disallow | /insetsrv/v1/* |
Disallow | /user/fpd/api/* |
Disallow | /Date%28* |
Disallow | /auth/sso/proxy-login* |
Disallow | /client/ |
Disallow | /buyside/search-results?*term=* |
msnptc/1.0
Rule | Path |
---|---|
Disallow | /article_email/* |
Disallow | /login/* |
Disallow | /acct/* |
Disallow | /msgcenter/* |
Disallow | /setup/* |
Disallow | /user/* |
Disallow | /marketing/* |
Disallow | /public/article/* |
Disallow | /public/search/ |
Disallow | /public/search* |
Disallow | /search* |
Disallow | /public/page/wsj-x-marketing.html |
Disallow | /public/page/news-media-marketing.html |
Disallow | /public/page/0_0_WP_RT_MARKETING.html |
Disallow | /news/articles/SB2* |
Disallow | /news/articles/SB3* |
Disallow | /news/articles/SB4* |
Disallow | /articles/SB2* |
Disallow | /articles/SB3* |
Disallow | /articles/SB4* |
Disallow | /article/AP* |
Disallow | /article/BT-CO* |
Disallow | /article/DN-CO* |
Disallow | /article/PR-CO* |
Disallow | /article/HUG* |
Disallow | /video/search/* |
Disallow | /articles/BT-CO* |
Disallow | /articles/DN-CO* |
Disallow | /articles/PR-CO* |
Disallow | /news/articles/BT-CO* |
Disallow | /news/articles/DN-CO* |
Disallow | /news/articles/PR-CO* |
Other Records
Field | Value |
---|---|
sitemap | https://www.wsj.com/sitemap.xml |
sitemap | https://www.wsj.com/wsjsitemaps/wsj_google_news.xml |
sitemap | https://www.wsj.com/sitemap_topics.xml |
sitemap | https://www.wsj.com/sitemaps/web/wsj/en/sitemap_wsj_en_index.xml |
sitemap | https://www.wsj.com/live_news_sitemap.xml |
sitemap | https://www.wsj.com/wsj_graphics_sitemap.xml |
sitemap | https://www.wsj.com/sitemaps/web/wsj/en/sitemap_news_archive_index.xml |
sitemap | https://www.wsj.com/wsjsitemaps/wsj_article_types.xml |
sitemap | https://www.wsj.com/authors_sitemap.xml |
sitemap | https://www.wsj.com/wsjsitemaps/wsj_article_list_sitemap.xml |
sitemap | https://www.wsj.com/api-video/sitemaps/google/sitemap-google-news-video-wsj-en.asp |
sitemap | https://www.wsj.com/api-video/sitemaps/google/sitemap-google-video-wsj-en.asp |
sitemap | https://www.wsj.com/sitemaps/web/video/en/sitemap_video_en_index.xml |
sitemap | https://www.wsj.com/buyside/sitemap.xml |
sitemap | https://www.wsj.com/wsjsitemaps/elections2024_index.xml |
Comments