Log In Sign Up

rapeutation.com
robots.txt

Robots Exclusion Standard data for rapeutation.com

Resource Scan

Scan Details

Site Domain rapeutation.com
Base Domain rapeutation.com
Scan Status Ok
Last Scan2025-09-08T00:07:38+00:00
Next Scan 2025-10-08T00:07:38+00:00

Last Scan

Scanned2025-09-08T00:07:38+00:00
URL https://rapeutation.com/robots.txt
Domain IPs 104.21.74.231, 172.67.164.80, 2606:4700:3032::6815:4ae7, 2606:4700:3037::ac43:a450
Response IP 104.21.74.231
Found Yes
Hash a9e72f0f04b455c9da5dce1e4b9e4ea2b438fddde69b44858fe38ebd838f27b6
SimHash d167c379a752

Groups

amazonbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

bingpreview

Rule Path
Disallow /

anthropic

Rule Path
Disallow /

huggingface

Rule Path
Disallow /
Back to top

Comments

  • OpenAI’s GPTBot
  • Bing Chat Preview
  • Anthropic’s Claude crawler
  • Hugging Face
  • Any other crawler whose name contains "AI" (if your crawler honors wildcard matching here)
Back to top

Warnings

  • 2 invalid lines.
Back to top