casetify.com
robots.txt
Robots Exclusion Standard data for casetify.com
Resource Scan
Scan Details
| Site Domain | casetify.com |
| Base Domain | casetify.com |
| Scan Status | Ok |
| Last Scan | 2024-09-14T09:47:51+00:00 |
| Next Scan | 2024-09-28T09:47:51+00:00 |
Last Scan
| Scanned | 2024-09-14T09:47:51+00:00 |
| URL | https://casetify.com/robots.txt |
| Redirect | https://www.casetify.com/robots.txt |
| Redirect Domain | www.casetify.com |
| Redirect Base | casetify.com |
| Domain IPs | 104.18.28.89, 104.18.29.89 |
| Redirect IPs | 104.18.28.89, 104.18.29.89 |
| Response IP | 104.18.28.89 |
| Found | Yes |
| Hash | 5d87950093b6f14f92ea458590f2c832bdc8ef233bf7725383b0e6ebbc93daa2 |
| SimHash | 6d2e5ab6c5f7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*index.php/ |
| Disallow | /*.php$ |
| Disallow | /*admin/ |
| Disallow | /bin/ |
| Disallow | /*build/ |
| Disallow | /*media/ |
| Disallow | /my/ |
| Disallow | /*checkout/ |
| Disallow | /*cron/ |
| Disallow | /*maintenance/ |
| Disallow | /*private/ |
| Disallow | /*scripts/ |
| Disallow | /*test/ |
| Disallow | /*testing/ |
| Disallow | /*tmp/ |
| Disallow | /*worker/ |
| Disallow | /*vendors/ |
| Disallow | /*includes/ |
| Disallow | /*setting/ |
| Disallow | /*action%3Dcheck_file_exist* |
| Disallow | /*ajaxCall%3D1* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
baiduspider
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /es_MX/* |
| Disallow | /de_DE/* |
| Disallow | /zh_HK/* |
| Disallow | /ja_JP/* |
| Disallow | /ko_KR/* |
| Disallow | /th_TH/* |
| Disallow | /fr_FR/* |
| Disallow | /id_ID/* |
haosouspider
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /es_MX/* |
| Disallow | /de_DE/* |
| Disallow | /zh_HK/* |
| Disallow | /ja_JP/* |
| Disallow | /ko_KR/* |
| Disallow | /th_TH/* |
| Disallow | /fr_FR/* |
| Disallow | /id_ID/* |
sogou web spider
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /es_MX/* |
| Disallow | /de_DE/* |
| Disallow | /zh_HK/* |
| Disallow | /ja_JP/* |
| Disallow | /ko_KR/* |
| Disallow | /th_TH/* |
| Disallow | /fr_FR/* |
| Disallow | /id_ID/* |
sogou inst spider
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /es_MX/* |
| Disallow | /de_DE/* |
| Disallow | /zh_HK/* |
| Disallow | /ja_JP/* |
| Disallow | /ko_KR/* |
| Disallow | /th_TH/* |
| Disallow | /fr_FR/* |
| Disallow | /id_ID/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.casetify.com/sitemap.xml |
Comments