Log In Sign Up

rayankadeh.com
robots.txt

Robots Exclusion Standard data for rayankadeh.com

Resource Scan

Scan Details

Site Domain rayankadeh.com
Base Domain rayankadeh.com
Scan Status Ok
Last Scan2025-12-21T16:04:46+00:00
Next Scan 2026-01-20T16:04:46+00:00

Last Scan

Scanned2025-12-21T16:04:46+00:00
URL https://rayankadeh.com/robots.txt
Domain IPs 185.136.133.66
Response IP 185.136.133.66
Found Yes
Hash 8174380d952db13f1ce683f99323c0d706f947631c238c260475b8d4f2e8dd80
SimHash 127cde228812

Groups

claudebot

Rule Path
Disallow /

claudebot/1.0

Rule Path
Disallow /

claudebot/*

Rule Path
Disallow /

googlebot

Rule Path
Disallow /*?add-to-cart
Disallow /*%26product_orderby
Disallow /?wc-ajax=get_refreshed_fragments

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

sogou

Rule Path
Disallow /

exabot

Rule Path
Disallow /

bspider

Rule Path
Disallow /

sitebulb

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

seoscanners

Rule Path
Disallow /

wget

Rule Path
Disallow /

curl

Rule Path
Disallow /
Back to top

Comments

  • Block Bad Bots / Spam Crawlers
Back to top