grizzly.com
robots.txt
Robots Exclusion Standard data for grizzly.com
Resource Scan
Scan Details
Site Domain | grizzly.com |
Base Domain | grizzly.com |
Scan Status | Ok |
Last Scan | 2024-05-21T14:46:53+00:00 |
Next Scan | 2024-06-20T14:46:53+00:00 |
Last Scan
Scanned | 2024-05-21T14:46:53+00:00 |
URL | https://grizzly.com/robots.txt |
Redirect | https://www.grizzly.com:443/robots.txt |
Redirect Domain | www.grizzly.com |
Redirect Base | grizzly.com |
Domain IPs | 52.89.213.28, 54.70.215.5, 54.71.92.183 |
Redirect IPs | 52.89.213.28, 54.70.215.5, 54.71.92.183 |
Response IP | 52.89.213.28 |
Found | Yes |
Hash | 57499754c39de505d56653a60cc1896a863637a965ba899111db2a99792d2058 |
SimHash | 7397434ac5c8 |
Groups
*
Rule | Path |
---|---|
Disallow | /user |
Disallow | /multiitem |
Disallow | /order |
Disallow | /cart |
Disallow | /skus |
Disallow | /error |
Disallow | /webservice |
Disallow | |
Disallow | /catalog |
Disallow | /parts/* |
Disallow | /checkout |
Disallow | /search |
Disallow | /testing/ |
istellabot/1.01.18 +http://www.tiscali.it/istellabot/1.10.2 +http://www.tiscali.it/
Rule | Path |
---|---|
Disallow | / |
mozilla/5.0 (compatible; domainappender /1.0; +http://www.profound.net/domainappender)
Rule | Path |
---|---|
Disallow | / |
mozilla/5.0 (compatible; domainsigmacrawler/0.1; +http://domainsigma.com/robot)
Rule | Path |
---|---|
Disallow | / |
mozilla/5.0 (compatible; meanpathbot/1.0; +http://www.meanpath.com/meanpathbot.html)
Rule | Path |
---|---|
Disallow | / |
mozilla/5.0 (compatible; mj12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+)
Rule | Path |
---|---|
Disallow | / |
mozilla/5.0 (compatible; netseer crawler/2.0; +http://www.netseer.com/crawler.html; crawler@netseer.com)
Rule | Path |
---|---|
Disallow | / |
mozilla/5.0 (compatible; pad-bot/9.0; +http://www.descargarprogramagratis.com/)
Rule | Path |
---|---|
Disallow | / |
mozilla/5.0 (compatible; paperlibot/2.1; http://support.paper.li/entries/20023257-what-is-paper-li)
Rule | Path |
---|---|
Disallow | / |
mozilla/5.0 (compatible; umbot-ln/1.0; mailto: crawling@ubermetrics-technologies.com)
Rule | Path |
---|---|
Disallow | / |
mozilla/5.0 (macintosh; intel mac os x 10.9; rv:28.0) gecko/20100101 firefox/28.0 (flipboardproxy/1.1; +http://flipboard.user-agent: com/browserproxy)
Rule | Path |
---|---|
Disallow | / |
mozilla/5.0 (windows; u; windows nt 5.1; en-us; rv:1.8.1.6) gecko/20070725 firefox/2.0.0.6 - james bot - webcrawler httpuser-agent: ://cognitiveseo.com/bot.html
Rule | Path |
---|---|
Disallow | / |
mozilla/5.0 (windows; u; windows nt 5.1; en-us; rv:1.9.1.2) gecko/20090729 firefox/3.5.2 (.net clr 3.5.30729; diffbot/0.user-agent: 1; +http://www.diffbot.com)
Rule | Path |
---|---|
Disallow | / |
mozilla/5.0+(x11;+linux+x86_64)+applewebkit/536.5+(khtml,+like+gecko)+chrome/19.0.1084.56+safari/536.5
Rule | Path |
---|---|
Disallow | / |
Other Records
Field | Value |
---|---|
sitemap | https://www.grizzly.com/sitemap/main/1 |
Warnings
- 8 invalid lines.