anixter.com
robots.txt
Robots Exclusion Standard data for anixter.com
Resource Scan
Scan Details
| Site Domain | anixter.com |
| Base Domain | anixter.com |
| Scan Status | Ok |
| Last Scan | 2025-11-21T11:41:08+00:00 |
| Next Scan | 2025-12-21T11:41:08+00:00 |
Last Scan
| Scanned | 2025-11-21T11:41:08+00:00 |
| URL | https://anixter.com/robots.txt |
| Redirect | https://www.anixter.com/robots.txt |
| Redirect Domain | www.anixter.com |
| Redirect Base | anixter.com |
| Domain IPs | 104.18.24.90 |
| Redirect IPs | 104.18.14.201, 104.18.15.201 |
| Response IP | 104.18.15.201 |
| Found | Yes |
| Hash | 47b7acdbf4e77f5f9ec41fc68643567edde4d695de4e02ab01cd21cd930ec6c5 |
| SimHash | 6622588cccd7 |
Groups
anixter webcrawler
| Rule | Path |
|---|---|
| Disallow | /global/ |
| Disallow | /en/ |
| Disallow | /bin/ |
| Disallow | /*/*.json |
| Disallow | /*/product-set.*.html |
| Disallow | /*/product-detail.*.html |
| Disallow | /*/customer-portals/ |
| Disallow | /*/products/wire-and-cable.html |
| Disallow | /*/products/communications.html |
| Disallow | /*/products/security.html |
| Disallow | /*/products/networking.html |
| Disallow | /*/products/hardware-and-supplies.html |
| Disallow | /*/products/oem.html |
| Disallow | /*/products/fasteners.html |
| Disallow | /*/products/suppliers.html |
| Disallow | /*/products/shopping-cart.html |
| Disallow | /*/resources/news-and-events/events-calendar/eventeditor.html |
| Disallow | /*/resources/news-and-events/events-calendar/eventviewer.html |
| Disallow | /*/resources/news-and-events/registration.html |
| Disallow | /*/resources/news-and-events/registration/multi-lingual-event-registration-forms.html |
| Disallow | /*/resources/news-and-events/registration/multi-lingual-event-registration-forms/*.html |
| Disallow | /*/resources/news-and-events/newsletter-sign-up/thank-you.html |
| Disallow | /*/resources/news-and-events/master-campaign-event-template.html |
| Disallow | /*/about-us/contact-us/global-locations-contact-info.html |
| Disallow | /*/about-us/contact-us/feedback/thank-you.html |
| Disallow | /*/404.html |
| Disallow | /*/Careers-Page-Not-Available.html |
| Disallow | /*/Investor-Relations-Page-is-Not-Available.html |
| Disallow | /*/Product-Catalog-Search-is-Not-Available.html |
| Disallow | /*/product-detail.html |
| Disallow | /*/product-set.html |
| Disallow | /*/supplier-products.html |
| Disallow | /*/search-results.html |
*
| Rule | Path |
|---|---|
| Disallow | /global/ |
| Disallow | /en/ |
| Disallow | /bin/ |
| Disallow | /*/*.json |
| Disallow | /*/customer-portals/ |
| Disallow | /*/products/wire-and-cable.html |
| Disallow | /*/products/communications.html |
| Disallow | /*/products/security.html |
| Disallow | /*/products/networking.html |
| Disallow | /*/products/hardware-and-supplies.html |
| Disallow | /*/products/oem.html |
| Disallow | /*/products/fasteners.html |
| Disallow | /*/products/shopping-cart.html |
| Disallow | /*/resources/news-and-events/events-calendar/eventeditor.html |
| Disallow | /*/resources/news-and-events/events-calendar/eventviewer.html |
| Disallow | /*/resources/news-and-events/registration.html |
| Disallow | /*/resources/news-and-events/registration/multi-lingual-event-registration-forms.html |
| Disallow | /*/resources/news-and-events/registration/multi-lingual-event-registration-forms/*.html |
| Disallow | /*/resources/news-and-events/newsletter-sign-up/thank-you.html |
| Disallow | /*/resources/news-and-events/master-campaign-event-template.html |
| Disallow | /*/about-us/contact-us/global-locations-contact-info.html |
| Disallow | /*/about-us/contact-us/feedback/thank-you.html |
| Disallow | /*/404.html |
| Disallow | /*/Careers-Page-Not-Available.html |
| Disallow | /*/Investor-Relations-Page-is-Not-Available.html |
| Disallow | /*/Product-Catalog-Search-is-Not-Available.html |
| Disallow | /*/product-detail.html |
| Disallow | /*/product-set.html |
| Disallow | /*/etc/design/anixter/images/ |
| Disallow | /*/cart* |
| Disallow | /*/checkout* |
| Disallow | /*/my-account* |
| Disallow | /*/login* |
| Disallow | /*/register* |
| Disallow | /*/reelcuts/enter* |
| Disallow | *zoomImages |
| Disallow | *quickView |
| Disallow | /rb_* |
| Disallow | /*/relatedProducts? |
| Disallow | /*/_ui/* |
| Disallow | /*/etc/designs* |
| Disallow | /*/etc/clientlibs* |
| Disallow | /*/favicon.ico |
| Disallow | /*/search/?text* |
| Disallow | /*/search/?q=* |
| Disallow | /*/product-set* |
| Disallow | /*/search/* |
| Disallow | /*/medias/* |
Other Records
| Field | Value | Comment |
|---|---|---|
| crawl-delay | 10 | 10 seconds between page requests |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.anixter.com/en_mx/sitemap-index.xml |
| sitemap | https://www.anixter.com/en_uk/sitemap-index.xml |
| sitemap | https://www.anixter.com/en_us/sitemap-index.xml |
| sitemap | https://www.anixter.com/en_us/sitemap-index2.xml |
| sitemap | https://www.anixter.com/en_ca/sitemap-index.xml |
| sitemap | https://www.anixter.com/en_ca/sitemap-index2.xml |
| sitemap | https://www.anixter.com/en_gb/sitemap-index.xml |
| sitemap | https://www.anixter.com/en_gb/sitemap-index2.xml |
| sitemap | https://www.anixter.com/fr_ca/sitemap-index.xml |
Warnings
- `request-rate` is not a known field.
- `visit-time` is not a known field.
Comments