memes.yarn.co
robots.txt

Robots Exclusion Standard data for memes.yarn.co

Resource Scan

Scan Details

Site Domain memes.yarn.co
Base Domain yarn.co
Scan Status Ok
Last Scan2024-10-22T17:41:53+00:00
Next Scan 2024-11-21T17:41:53+00:00

Last Scan

Scanned2024-10-22T17:41:53+00:00
URL https://memes.yarn.co/robots.txt
Domain IPs 104.18.12.142, 104.18.13.142, 2606:4700::6812:c8e, 2606:4700::6812:d8e
Response IP 104.18.12.142
Found Yes
Hash 916b88da6667efeb3c39b9881ba48fab7a98ebd88110f9b514ea6056f8b01344
SimHash 09189506f991

Groups

*
*

Rule Path
Disallow /api/

gumgum-bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://getyarn.io/sitemap.xml

Comments

  • robotstxt.org