yousayhrsay.com
robots.txt

Robots Exclusion Standard data for yousayhrsay.com

Resource Scan

Scan Details

Site Domain yousayhrsay.com
Base Domain yousayhrsay.com
Scan Status Ok
Last Scan2025-10-10T14:55:25+00:00
Next Scan 2025-11-09T14:55:25+00:00

Last Scan

Scanned2025-10-10T14:55:25+00:00
URL https://yousayhrsay.com/robots.txt
Redirect https://www.yousayhrsay.com:443/robots.txt
Redirect Domain www.yousayhrsay.com
Redirect Base yousayhrsay.com
Domain IPs 13.33.88.27, 13.33.88.45, 13.33.88.57, 13.33.88.93
Redirect IPs 13.33.88.27, 13.33.88.45, 13.33.88.57, 13.33.88.93
Response IP 13.33.88.93
Found Yes
Hash 98212e32d70dd3677b65eabd74d94011d06189cae813e5d3eb0fd43ce723883a
SimHash 691c0812ea82

Groups

googlebot

Rule Path
Disallow /api/
Disallow /admin/
Disallow /member/
Disallow /profile/
Disallow /*?sort=*
Disallow /*?filter=*
Disallow /*?page=*
Disallow /*?tab=*
Disallow /*encryptedUserId
Disallow /*userValidationKey
Allow /

bingbot

Rule Path
Disallow /api/
Disallow /admin/
Disallow /member/
Allow /

gptbot

Rule Path
Allow /

claudebot

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

claude-web

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

perplexity

Rule Path
Allow /

google-extended

Rule Path
Allow /

googleother

Rule Path
Allow /

applebot-extended

Rule Path
Allow /

amazonbot

Rule Path
Allow /

ccbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

img2dataset

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

linkedinbot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

facebookbot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

whatsapp

Rule Path
Allow /

slackbot

Rule Path
Allow /

*

Rule Path
Disallow /api/
Disallow /admin/
Disallow /member/
Disallow /profile/
Disallow /private/
Disallow /tmp/
Disallow /_nuxt/
Disallow /cdn-cgi/
Disallow /*.json$
Disallow /*?sort=*
Disallow /*?filter=*
Disallow /*?tab=*
Disallow /*Ajax.htm
Disallow /ajax/
Disallow /track
Disallow /*encryptedUserId
Disallow /*userValidationKey
Disallow /*?page=6
Disallow /*?page=7
Disallow /*?page=8
Disallow /*?page=9
Disallow /*_P6
Disallow /*_P7
Disallow /*_P8
Disallow /*_P9
Allow /

Other Records

Field Value
sitemap https://www.yousayhrsay.com/sitemap.xml