worldindustrialreporter.com
robots.txt
Robots Exclusion Standard data for worldindustrialreporter.com
Resource Scan
Scan Details
| Site Domain | worldindustrialreporter.com |
| Base Domain | worldindustrialreporter.com |
| Scan Status | Ok |
| Last Scan | 2024-06-10T14:40:25+00:00 |
| Next Scan | 2024-06-17T14:40:25+00:00 |
Last Scan
| Scanned | 2024-06-10T14:40:25+00:00 |
| URL | https://worldindustrialreporter.com/robots.txt |
| Redirect | https://www.thomasnet.com/robots.txt |
| Redirect Domain | www.thomasnet.com |
| Redirect Base | thomasnet.com |
| Domain IPs | 13.224.163.113, 13.224.163.29, 13.224.163.45, 13.224.163.46 |
| Redirect IPs | 104.18.22.243, 104.18.23.243, 2606:4700::6812:16f3, 2606:4700::6812:17f3 |
| Response IP | 104.18.23.243 |
| Found | Yes |
| Hash | 9bf281715e70ef1e0ec9f613ffe25ca09f52d2a4068b1274a864fb3da2f1f003 |
| SimHash | e080d7955e88 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /404.html |
| Disallow | /ProductInformationHQ.html |
| Disallow | /RCS/ |
| Disallow | /Seminar/ |
| Disallow | /adlink.html |
| Disallow | /advsearch.html |
| Disallow | /branchheading.html |
| Disallow | /branchprofile.html |
| Disallow | /brands.html |
| Disallow | /browsecatalogcats.html |
| Disallow | /browse.html |
| Disallow | /businessweek.html |
| Disallow | /cadcb.html |
| Disallow | /cadlibrary.html |
| Disallow | /cadlink.html |
| Disallow | /cadpl.html |
| Disallow | /cadps.html |
| Disallow | /cad-taxonomy.xml.gz |
| Disallow | /capabilities-services.html |
| Disallow | /captcha/ |
| Disallow | /catalognavigator.html |
| Disallow | /catalogs/keyword/ |
| Disallow | /catnav/browse.html |
| Disallow | /catnav/itemdetail.html |
| Disallow | /catnav/list.html |
| Disallow | /catnav/productdetail.html |
| Disallow | /catnav/products.html |
| Disallow | /ccpcontent.html |
| Disallow | /cduser/ |
| Disallow | /cofeedback.html |
| Disallow | /cofeedback_send.html |
| Disallow | /commodityresults.html |
| Disallow | /company-certifications.html |
| Disallow | /company-information.html |
| Disallow | /compare.html |
| Disallow | /compsearch.html |
| Disallow | /conews.html |
| Disallow | /contactcompany.html |
| Disallow | /contactcompany_send.html |
| Disallow | /dc/ |
| Disallow | /dnb/ |
| Disallow | /ecatalog/ |
| Disallow | /emailcatalog.html |
| Disallow | /emailcolleague.html |
| Disallow | /emailprofile.html |
| Disallow | /emailresults.html |
| Disallow | /feedback.html |
| Disallow | /feedback_send.html |
| Disallow | /free/ |
| Disallow | /gen/ |
| Disallow | /generalinfo.html |
| Disallow | /gotowin/ |
| Disallow | /heading.html |
| Disallow | /headinginfo.html |
| Disallow | /headingtrdir.html |
| Disallow | /inc/ |
| Disallow | /industrialmarkettrends |
| Disallow | /local.html |
| Disallow | /map-company.html |
| Disallow | /map-print.html |
| Disallow | /multicontact.html |
| Disallow | /multiselect.html |
| Disallow | /mythomas/ |
| Disallow | /newlook.html |
| Disallow | *nsearch.html? |
| Disallow | /pcb-attribute-details.html |
| Disallow | /pnnalerts.html |
| Disallow | /pnnalerts_save.html |
| Disallow | /preview.html |
| Disallow | /print/ |
| Disallow | /product-information.html |
| Disallow | /productsearch/productspecs.html |
| Disallow | /profile.html |
| Disallow | /profileinfo.html |
| Disallow | /profilenews.html |
| Disallow | /register/ |
| Disallow | /results.html |
| Disallow | /rfi/ |
| Disallow | /search/additional_suppliers.php |
| Disallow | /seealso.html |
| Disallow | /seminar/ |
| Disallow | /seminars/ |
| Disallow | /spreadword.html |
| Disallow | /spreadword_send.html |
| Disallow | /subscriber/ |
| Disallow | /survey/ |
| Disallow | /terms.html |
| Disallow | /tgr.html |
| Disallow | /tnetfooter.html |
| Disallow | /tnetheader.html |
| Disallow | /toolbarv2/ |
| Disallow | /tools-gadgets/ |
| Disallow | /trdir.html |
| Disallow | /user/ |
| Disallow | /vimeo.html |
| Disallow | /vlink.html |
| Disallow | /webcontactcompany.html |
| Disallow | /webprofile.html |
| Disallow | /webprofileinfo.html |
| Disallow | /webresults.html |
| Disallow | /websearchlink.html |
| Disallow | /white-papers/publisher/ |
| Disallow | /order.html |
| Disallow | /configure.html |
| Disallow | /product-service-detail.html |
| Disallow | /whitepaper-details.html |
| Disallow | /whitepapers.html |
| Disallow | /catalog-images.html |
| Disallow | /bsearch.html |
| Disallow | /branches.html |
| Disallow | /local/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.thomasnet.com/sitemap_index.xml.gz |
| sitemap | https://www.thomasnet.com/suppliers/sitemap/index.xml |
Warnings
- 2 invalid lines.