humanpsychologyfacts.quora.com
robots.txt

Robots Exclusion Standard data for humanpsychologyfacts.quora.com

Resource Scan

Scan Details

Site Domain humanpsychologyfacts.quora.com
Base Domain quora.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-04-25T07:51:51+00:00
Next Scan 2024-05-25T07:51:51+00:00

Last Successful Scan

Scanned2024-03-20T07:27:53+00:00
URL https://humanpsychologyfacts.quora.com/robots.txt
Redirect https://darksecretsdaily.quora.com/robots.txt
Redirect Domain darksecretsdaily.quora.com
Redirect Base quora.com
Domain IPs 162.159.152.17, 162.159.153.247
Redirect IPs 162.159.152.17, 162.159.153.247
Response IP 162.159.152.17
Found Yes
Hash 13d8adad6afd3ff48647537fad50366a5c0318ecc714d0dadf9f363def5daa8f
SimHash 8c108318cdc0

Groups

googlebot

Rule Path
Disallow /*/admin_log
Disallow /*/stats
Disallow /*/settings
Disallow /*/queue
Disallow /*/suggestions
Disallow /*/submissions
Disallow /*/quality
Disallow /*/earnings

facebot

Rule Path
Disallow /*/admin_log
Disallow /*/stats
Disallow /*/settings
Disallow /*/queue
Disallow /*/suggestions
Disallow /*/submissions
Disallow /*/quality
Disallow /*/earnings

applebot

Rule Path
Disallow /*/admin_log
Disallow /*/stats
Disallow /*/settings
Disallow /*/queue
Disallow /*/suggestions
Disallow /*/submissions
Disallow /*/quality
Disallow /*/earnings

twitterbot

Rule Path
Disallow /*/admin_log
Disallow /*/stats
Disallow /*/settings
Disallow /*/queue
Disallow /*/suggestions
Disallow /*/submissions
Disallow /*/quality
Disallow /*/earnings

linkedinbot

Rule Path
Disallow /*/admin_log
Disallow /*/stats
Disallow /*/settings
Disallow /*/queue
Disallow /*/suggestions
Disallow /*/submissions
Disallow /*/quality
Disallow /*/earnings

feedly

Rule Path
Disallow /*/admin_log
Disallow /*/stats
Disallow /*/settings
Disallow /*/queue
Disallow /*/suggestions
Disallow /*/submissions
Disallow /*/quality
Disallow /*/earnings

pingdom

Rule Path
Disallow /*/admin_log
Disallow /*/stats
Disallow /*/settings
Disallow /*/queue
Disallow /*/suggestions
Disallow /*/submissions
Disallow /*/quality
Disallow /*/earnings

neevabot

Rule Path
Disallow /*/admin_log
Disallow /*/stats
Disallow /*/settings
Disallow /*/queue
Disallow /*/suggestions
Disallow /*/submissions
Disallow /*/quality
Disallow /*/earnings

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

jamesbot

Rule Path
Disallow /

*

Rule Path
Disallow /

Comments

  • If you operate a search engine and would like to crawl Quora,
  • you can email robotstxt@quora.com. Thanks.
  • To disallow GPTBot to access Quora.
  • People share a lot of sensitive material on Quora - controversial political
  • views, workplace gossip and compensation, and negative opinions held of
  • companies. Over many years, as they change jobs or change their views, it is
  • important that they can delete or anonymize their previously-written answers.
  • We opt out of the wayback machine because inclusion would allow people to
  • discover the identity of authors who had written sensitive answers publicly and
  • later had made them anonymous, and because it would prevent authors from being
  • able to remove their content from the internet if they change their mind about
  • publishing it. As far as we can tell, there is no way for sites to selectively
  • programmatically remove content from the archive and so this is the only way
  • for us to protect writers. If they open up an API where we can remove content
  • from the archive when authors remove it from Quora, but leave the rest of the
  • content archived, we would be happy to opt back in. See the page here:
  • https://archive.org/about/exclude.php
  • Meanwhile, if you are looking for an older version of any content on Quora, we
  • have full edit history tracked and accessible in product (with the exception of
  • content that has been removed by the author). You can generally access this by
  • clicking on timestamps, or by appending "/log" to the URL of any content page.
  • For any questions or feedback about this please visit our contact page
  • https://help.quora.com/hc/en-us/requests/new
  • Blocked since a lot of bad requests were made by this crawler.