darius.com
robots.txt

Robots Exclusion Standard data for darius.com

Resource Scan

Scan Details

Site Domain darius.com
Base Domain darius.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2025-07-20T06:59:41+00:00
Next Scan 2025-08-19T06:59:41+00:00

Last Successful Scan

Scanned2025-06-13T18:49:22+00:00
URL https://darius.com/robots.txt
Redirect https://darius.com/robots.txt?gi=1a8d6162b989
Domain IPs 52.0.16.118, 52.1.119.170, 52.1.147.205, 52.1.173.203, 52.4.145.119, 52.4.175.111, 52.4.225.124, 52.4.240.221, 52.4.38.70, 52.5.181.79, 52.6.3.192, 52.6.46.142
Response IP 52.6.46.142
Found Yes
Hash 1c9ae0a4275ff7faa4460c2b60ac58ec25f7b48c4c2fcccc07b9529eab9bc85b
SimHash 69143b40c772

Groups

*

Rule Path
Disallow /m/
Disallow /me/
Disallow /%40me$
Disallow /%40me/
Disallow /*/edit$
Disallow /*/*/edit$
Disallow /media/
Disallow /p/*/share
Disallow /r/
Disallow /trending
Disallow /search?q$
Disallow /search?q=
Disallow /*/search?q=
Disallow /*/search/*?q=
Disallow /*/*source%3D
Allow /_/
Allow /_/api/users/*/meta
Allow /_/api/users/*/profile/stream
Allow /_/api/posts/*/responses
Allow /_/api/posts/*/responsesStream
Allow /_/api/posts/*/related

applebot-extended
bytespider
claudebot
facebookbot
googleother
google-extended
gptbot

Rule Path
Disallow /
Allow /about
Allow /business
Allow /earn
Allow /gift
Allow /membership
Allow /partner-program
Allow /verified-authors

Other Records

Field Value
sitemap https://darius.com/sitemap/sitemap.xml