me.ym.dev
robots.txt

Robots Exclusion Standard data for me.ym.dev

Resource Scan

Scan Details

Site Domain me.ym.dev
Base Domain ym.dev
Scan Status Ok
Last Scan2025-10-26T13:48:28+00:00
Next Scan 2025-11-02T13:48:28+00:00

Last Scan

Scanned2025-10-26T13:48:28+00:00
URL https://me.ym.dev/robots.txt
Redirect https://yakovmanshin.com/robots.txt
Redirect Domain yakovmanshin.com
Redirect Base yakovmanshin.com
Domain IPs 104.21.33.100, 172.67.189.203, 2606:4700:3030::6815:2164, 2606:4700:3033::ac43:bdcb
Redirect IPs 104.21.84.87, 172.67.190.145, 2606:4700:3034::ac43:be91, 2606:4700:3036::6815:5457
Response IP 104.21.84.87
Found Yes
Hash 985ab59c4015d035421fce6b7035275cc452ee4ef975775cea8e681146096004
SimHash 51117341e1d7

Groups

*

Rule Path
Allow /
Disallow /bird/page/
Disallow /tags/

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

applebot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

duckduckbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

facebot

Rule Path
Disallow /

google-cloudvertexbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexity-user

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://yakovmanshin.com/sitemap.xml