au.keepcup.com
robots.txt

Robots Exclusion Standard data for au.keepcup.com

Resource Scan

Scan Details

Site Domain au.keepcup.com
Base Domain keepcup.com
Scan Status Ok
Last Scan2024-09-19T06:15:24+00:00
Next Scan 2024-10-19T06:15:24+00:00

Last Scan

Scanned2024-09-19T06:15:24+00:00
URL https://au.keepcup.com/robots.txt
Domain IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Response IP 199.232.45.124
Found Yes
Hash 6934afb82efe0910e1daa00dda7a466d6e84450aa8b4aaad1dec827651dc2495
SimHash cd14f8afeb9b

Groups

*

Rule Path
Disallow /*.php*
Disallow /*SID%3D*
Disallow /*ajax%3D*
Disallow /*options%3D*
Disallow /*product_list_dir%3D*
Disallow /*product_list_mode%3D*
Disallow /*product_list_order%3D*
Disallow /*product_list_limit%3D*
Disallow /*size%3D*
Disallow /*band%3D*
Disallow /*lid%3D*
Disallow /*plug%3D*
Disallow /*by_features%3D*
Disallow /*colour%3D*
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /onestepcheckout/
Disallow /catalogsearch/
Disallow /newsletter/
Disallow /sendfriend/
Disallow /customize/
Disallow /index.php/
Disallow /checkout/
Disallow /wishlist/
Disallow /contacts/
Disallow /customer/
Disallow /review/
Disallow /ajax/
Disallow /poll/
Disallow /tag/

mj12bot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

femtosearchbot

Rule Path
Disallow /

Comments

  • Crawlers Setup
  • Block bots