Log In Sign Up

cookpad.com
robots.txt

Robots Exclusion Standard data for cookpad.com

Resource Scan

Scan Details

Site Domain cookpad.com
Base Domain cookpad.com
Scan Status Ok
Last Scan2024-06-12T04:45:17+00:00
Next Scan 2024-06-19T04:45:17+00:00

Last Scan

Scanned2024-06-12T04:45:17+00:00
URL https://cookpad.com/robots.txt
Domain IPs 151.101.130.132, 151.101.194.132, 151.101.2.132, 151.101.66.132, 2a04:4e42:200::644, 2a04:4e42:400::644, 2a04:4e42:600::644, 2a04:4e42::644
Response IP 151.101.2.132
Found Yes
Hash 127b5053dee94497725d0f3d518e25ad334c89ef2e9cc6c22a1d5e03b82e7b14
SimHash 80de99edf573

Groups

*

Rule Path
Disallow /user/confirm_premium_navi
Allow /

baiduspider

Rule Path
Allow /cn
Disallow /*?_pxhc=*
Disallow /cn/users
Disallow /

yandex

Rule Path
Allow /
Disallow /*/accounts/new

gptbot

Rule Path
Disallow /
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • See below for how Clean-param works for Yandex crawler
  • https://yandex.ru/support/webmaster/robot-workings/clean-param.html?lang=en
Back to top

Warnings

  • `clean-param` is not a known field.
Back to top