myarticles.io
robots.txt

Robots Exclusion Standard data for myarticles.io

Resource Scan

Scan Details

Site Domain myarticles.io
Base Domain myarticles.io
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-06-03T21:13:56+00:00
Next Scan 2025-06-10T21:13:56+00:00

Last Successful Scan

Scanned2025-05-26T21:13:14+00:00
URL https://myarticles.io/robots.txt
Domain IPs 2a02:4780:84:dbf1:8e89:a461:3c94:1f95, 93.127.187.43
Response IP 77.37.115.76
Found Yes
Hash 94cd8f3bcd9d629e5d6bd216ddd16b22c66e8c5b4fadbfd7e88d488517db8886
SimHash cb7fd810880b

Groups

*

Rule Path
Disallow /members/*/
Disallow /members/*/profile/
Disallow /members/*/activity/mentions/
Disallow /members/*/activity/favorites/
Disallow /members/*/activity/friends/
Disallow /members/*/activity/groups/
Disallow /members/*/activity/friends/
Disallow /members/*/friends/
Disallow /members/*/groups/
Disallow /members/*/forums/
Disallow /members/*/articles/
Disallow /members/*/media/
Disallow /members/*/forums/
Disallow /groups/*
Disallow /forums/*
Disallow */feed/
Disallow /*/feed/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/

chatgpt

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://myarticles.io/sitemap_index.xml

Comments

  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK