casetify.com
robots.txt
Robots Exclusion Standard data for casetify.com
Resource Scan
Scan Details
Site Domain | casetify.com |
Base Domain | casetify.com |
Scan Status | Ok |
Last Scan | 2024-11-09T09:48:47+00:00 |
Next Scan | 2024-11-23T09:48:47+00:00 |
Last Scan
Scanned | 2024-11-09T09:48:47+00:00 |
URL | https://casetify.com/robots.txt |
Redirect | https://www.casetify.com/robots.txt |
Redirect Domain | www.casetify.com |
Redirect Base | casetify.com |
Domain IPs | 104.18.28.89, 104.18.29.89 |
Redirect IPs | 104.18.28.89, 104.18.29.89 |
Response IP | 104.18.29.89 |
Found | Yes |
Hash | 5d87950093b6f14f92ea458590f2c832bdc8ef233bf7725383b0e6ebbc93daa2 |
SimHash | 6d2e5ab6c5f7 |
Groups
*
Rule | Path |
---|---|
Disallow | /*index.php/ |
Disallow | /*.php$ |
Disallow | /*admin/ |
Disallow | /bin/ |
Disallow | /*build/ |
Disallow | /*media/ |
Disallow | /my/ |
Disallow | /*checkout/ |
Disallow | /*cron/ |
Disallow | /*maintenance/ |
Disallow | /*private/ |
Disallow | /*scripts/ |
Disallow | /*test/ |
Disallow | /*testing/ |
Disallow | /*tmp/ |
Disallow | /*worker/ |
Disallow | /*vendors/ |
Disallow | /*includes/ |
Disallow | /*setting/ |
Disallow | /*action%3Dcheck_file_exist* |
Disallow | /*ajaxCall%3D1* |
Other Records
Field | Value |
---|---|
crawl-delay | 1 |
baiduspider
Rule | Path |
---|---|
Allow | / |
Disallow | /es_MX/* |
Disallow | /de_DE/* |
Disallow | /zh_HK/* |
Disallow | /ja_JP/* |
Disallow | /ko_KR/* |
Disallow | /th_TH/* |
Disallow | /fr_FR/* |
Disallow | /id_ID/* |
haosouspider
Rule | Path |
---|---|
Allow | / |
Disallow | /es_MX/* |
Disallow | /de_DE/* |
Disallow | /zh_HK/* |
Disallow | /ja_JP/* |
Disallow | /ko_KR/* |
Disallow | /th_TH/* |
Disallow | /fr_FR/* |
Disallow | /id_ID/* |
sogou web spider
Rule | Path |
---|---|
Allow | / |
Disallow | /es_MX/* |
Disallow | /de_DE/* |
Disallow | /zh_HK/* |
Disallow | /ja_JP/* |
Disallow | /ko_KR/* |
Disallow | /th_TH/* |
Disallow | /fr_FR/* |
Disallow | /id_ID/* |
sogou inst spider
Rule | Path |
---|---|
Allow | / |
Disallow | /es_MX/* |
Disallow | /de_DE/* |
Disallow | /zh_HK/* |
Disallow | /ja_JP/* |
Disallow | /ko_KR/* |
Disallow | /th_TH/* |
Disallow | /fr_FR/* |
Disallow | /id_ID/* |
Other Records
Field | Value |
---|---|
sitemap | https://www.casetify.com/sitemap.xml |
Comments