Log In Sign Up

bargain-widgets.cookpad.com
robots.txt

Robots Exclusion Standard data for bargain-widgets.cookpad.com

Resource Scan

Scan Details

Site Domain bargain-widgets.cookpad.com
Base Domain cookpad.com
Scan Status Ok
Last Scan2024-06-21T00:40:08+00:00
Next Scan 2024-06-28T00:40:08+00:00

Last Scan

Scanned2024-06-21T00:40:08+00:00
URL https://bargain-widgets.cookpad.com/robots.txt
Redirect https://tokubai.co.jp/robots.txt
Redirect Domain tokubai.co.jp
Redirect Base tokubai.co.jp
Domain IPs 18.155.68.106, 18.155.68.25, 18.155.68.31, 18.155.68.92, 2600:9000:23d2:2000:1c:1bfb:f1c0:93a1, 2600:9000:23d2:3200:1c:1bfb:f1c0:93a1, 2600:9000:23d2:3e00:1c:1bfb:f1c0:93a1, 2600:9000:23d2:7400:1c:1bfb:f1c0:93a1, 2600:9000:23d2:a000:1c:1bfb:f1c0:93a1, 2600:9000:23d2:d200:1c:1bfb:f1c0:93a1, 2600:9000:23d2:e800:1c:1bfb:f1c0:93a1, 2600:9000:23d2:f200:1c:1bfb:f1c0:93a1
Redirect IPs 13.115.61.119, 18.178.67.233
Response IP 18.178.67.233
Found Yes
Hash dca431c7df60e39a30116c646bdbf3672f2438c4125206f62a9ace27a1d78769
SimHash b2944d8d3654

Groups

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
  • Claude / Anthropic AI
Back to top