knitpicks.com
robots.txt
Robots Exclusion Standard data for knitpicks.com
Resource Scan
Scan Details
Site Domain | knitpicks.com |
Base Domain | knitpicks.com |
Scan Status | Ok |
Last Scan | 2024-09-28T17:05:43+00:00 |
Next Scan | 2024-10-28T17:05:43+00:00 |
Last Scan
Scanned | 2024-09-28T17:05:43+00:00 |
URL | https://knitpicks.com/robots.txt |
Redirect | https://www.knitpicks.com/robots.txt |
Redirect Domain | www.knitpicks.com |
Redirect Base | knitpicks.com |
Domain IPs | 52.12.245.24, 52.26.38.184 |
Redirect IPs | 52.12.245.24, 52.26.38.184 |
Response IP | 52.26.38.184 |
Found | Yes |
Hash | 4c0e1e1a0463874b6bd2b10117234c5cbcd3efd8f8d10fb02e211f8374451045 |
SimHash | 4a0498507e32 |
Groups
googlebot
Rule | Path |
---|---|
Disallow | |
Disallow | /search?* |
Disallow | /free-download/* |
Disallow | /download?* |
Disallow | /*ROBOTS%3DNO |
Disallow | /checkout/completed?* |
googlebot-image
Rule | Path |
---|---|
Disallow | |
Disallow | /search?* |
Disallow | /free-download/* |
Disallow | /download?* |
Disallow | /*ROBOTS%3DNO |
Disallow | /checkout/completed?* |
*
Rule | Path |
---|---|
Disallow | |
Disallow | /search?* |
Disallow | /free-download/* |
Disallow | /download?* |
Disallow | /*ROBOTS%3DNO |
Disallow | /checkout/completed?* |
rogerbot
Rule | Path |
---|---|
Disallow | |
Disallow | /search?* |
Disallow | /free-download/* |
Disallow | /download?* |
Disallow | /*ROBOTS%3DNO |
Disallow | /checkout/completed?* |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
dotbot
Rule | Path |
---|---|
Disallow | |
Disallow | /search?* |
Disallow | /free-download/* |
Disallow | /download?* |
Disallow | /*ROBOTS%3DNO |
Disallow | /checkout/completed?* |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
mj12bot
Rule | Path |
---|---|
Disallow | |
Disallow | /search?* |
Disallow | /free-download/* |
Disallow | /download?* |
Disallow | /*ROBOTS%3DNO |
Disallow | /checkout/completed?* |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
ahrefsbot
Rule | Path |
---|---|
Disallow | |
Disallow | /search?* |
Disallow | /free-download/* |
Disallow | /download?* |
Disallow | /*ROBOTS%3DNO |
Disallow | /checkout/completed?* |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
semrushbot
Rule | Path |
---|---|
Disallow | |
Disallow | /search?* |
Disallow | /free-download/* |
Disallow | /download?* |
Disallow | /*ROBOTS%3DNO |
Disallow | /checkout/completed?* |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
Other Records
Field | Value |
---|---|
sitemap | https://www.knitpicks.com/sitemaps/kp-sitemap.xml |
Comments