mw3.wsj.net
robots.txt
Robots Exclusion Standard data for mw3.wsj.net
Resource Scan
Scan Details
Site Domain | mw3.wsj.net |
Base Domain | wsj.net |
Scan Status | Ok |
Last Scan | 2024-09-11T17:22:11+00:00 |
Next Scan | 2024-10-11T17:22:11+00:00 |
Last Scan
Scanned | 2024-09-11T17:22:11+00:00 |
URL | https://mw3.wsj.net/robots.txt |
Domain IPs | 3.165.82.110, 3.165.82.117, 3.165.82.29, 3.165.82.66 |
Response IP | 3.165.82.110 |
Found | Yes |
Hash | f8c3c814d4976702b7d3a622b755825c404a4a10964b26e269988c3fdaf16bc1 |
SimHash | 581d7331cf94 |
Groups
*
Rule | Path |
---|---|
Disallow | /3com/ |
Disallow | /admin/ |
Disallow | /archive/ |
Disallow | /bin/ |
Disallow | /cgi-bin/ |
Disallow | /data/ |
Disallow | /dbcfiles/ |
Disallow | /dhtml/ |
Disallow | /dhtmlmenu/ |
Disallow | /doubleclick/ |
Disallow | /dynamiclogic/ |
Disallow | /enf/ |
Disallow | /feeds/ |
Disallow | /free/ |
Disallow | /http2_data/ |
Disallow | /login/ |
Disallow | /MarketPerception/ |
Disallow | /marketperception/ |
Disallow | /MyPortfolio/ |
Disallow | /myportfolio/ |
Disallow | /news/current/ |
Disallow | /portfolio/ |
Disallow | /RealMedia/ |
Disallow | /realmedia/ |
Disallow | /registration/ |
Disallow | /tools/alerts/ |
Disallow | /tpx/ |
Disallow | /tools/quotes/secfilings.asp |
Disallow | /tools/quotes/form10k.asp |
Disallow | /tools/quotes/form10q.asp |
Disallow | /tools/quotes/form8k.asp |
Disallow | /tools/quotes/registrationstate.asp |
Disallow | /tools/quotes/sched13d.asp |
Disallow | /tools/quotes/sched13f.asp |
Disallow | /tools/quotes/sched13g.asp |
Disallow | /tools/quotes/secarticle.asp |
Disallow | /wsj/ |
Disallow | /xhtml/v/ |
msnptc/1.0
Rule | Path |
---|---|
Allow | /registration/ |
Allow | /tools/alerts/ |
Allow | /login/ |
Allow | /myportfolio/ |
Disallow | /3com/ |
Disallow | /admin/ |
Disallow | /archive/ |
Disallow | /bin/ |
Disallow | /cgi-bin/ |
Disallow | /data/ |
Disallow | /dbcfiles/ |
Disallow | /dhtml/ |
Disallow | /dhtmlmenu/ |
Disallow | /doubleclick/ |
Disallow | /dynamiclogic/ |
Disallow | /enf/ |
Disallow | /feeds/ |
Disallow | /free/ |
Disallow | /http2_data/ |
Disallow | /login/ |
Disallow | /MarketPerception/ |
Disallow | /marketperception/ |
Disallow | /news/current/ |
Disallow | /RealMedia/ |
Disallow | /realmedia/ |
Disallow | /tpx/ |
Disallow | /tools/quotes/secfilings.asp |
Disallow | /tools/quotes/form10k.asp |
Disallow | /tools/quotes/form10q.asp |
Disallow | /tools/quotes/form8k.asp |
Disallow | /tools/quotes/registrationstate.asp |
Disallow | /tools/quotes/sched13d.asp |
Disallow | /tools/quotes/sched13f.asp |
Disallow | /tools/quotes/sched13g.asp |
Disallow | /tools/quotes/secarticle.asp |
Other Records
Field | Value |
---|---|
sitemap | http://www.marketwatch.com/sitemap-search-index.xml.gz |