carhartt-wip.com
robots.txt

Robots Exclusion Standard data for carhartt-wip.com

Resource Scan

Scan Details

Site Domain carhartt-wip.com
Base Domain carhartt-wip.com
Scan Status Ok
Last Scan2024-11-02T08:05:10+00:00
Next Scan 2024-11-16T08:05:10+00:00

Last Scan

Scanned2024-11-02T08:05:10+00:00
URL https://carhartt-wip.com/robots.txt
Redirect https://www.carhartt-wip.com/robots.txt
Redirect Domain www.carhartt-wip.com
Redirect Base carhartt-wip.com
Domain IPs 35.227.235.17
Redirect IPs 35.227.235.17
Response IP 35.227.235.17
Found Yes
Hash 8fcb03bc9c0e6c932073cc044f349c1928d78f35a67807e2ba9f3ed42d206801
SimHash e04a087065b7

Groups

*

Rule Path
Disallow /*?page=-
Disallow /restservices*
Disallow /en/ajax
Disallow /de/ajax
Disallow /fr/ajax
Disallow /es/ajax
Disallow /it/ajax

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.2

Other Records

Field Value
sitemap https://www.carhartt-wip.com/sitemap.xml