qr.ae
robots.txt

Robots Exclusion Standard data for qr.ae

Resource Scan

Scan Details

Site Domain qr.ae
Base Domain qr.ae
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-13T03:33:54+00:00
Next Scan 2024-11-27T03:33:54+00:00

Last Successful Scan

Scanned2024-04-14T22:17:50+00:00
URL https://www.qr.ae/robots.txt
Redirect https://www.quora.com/robots.txt
Redirect Domain www.quora.com
Redirect Base quora.com
Domain IPs 52.2.236.164, 52.23.50.78, 52.3.162.157, 52.45.226.173, 52.45.72.125, 52.54.12.196, 52.71.163.88, 52.71.99.164
Redirect IPs 162.159.152.17, 162.159.153.247
Response IP 162.159.153.247
Found Yes
Hash 371091c1e31a4ecdd340a99aa99da6fc40cc416e1c6530d0a3a20c07e699ae9a
SimHash 2c1403384d81

Groups

googlebot

Rule Path
Disallow /ajax/
Disallow /%40async
Disallow /*/%40async
Disallow /log/
Disallow /*/log
Disallow /*/about
Disallow /*/action
Disallow /*/activity
Disallow /*/all_questions
Disallow /*/all_posts$
Disallow /*/all_posts/
Disallow /*/blogs$
Disallow /*/blogs/
Disallow /*/followers
Disallow /*/following
Disallow /*/link/
Disallow /*/manage
Disallow /*/mentions
Disallow /*/merged
Disallow /*/open_questions
Disallow /*/posts$
Disallow /*/posts/
Disallow /*/questions
Disallow /*/related
Disallow /*/reviews$
Disallow /*/reviews/
Disallow /*/share
Disallow /*/top_questions
Disallow /*/topic-bio
Disallow /*/topic_bio
Disallow /*/topics
Disallow /*/comment
Disallow /comment/
Disallow /*/comments
Disallow /*/all_comments
Disallow /*/answer_comments
Disallow /*/mobile_collapsed
Disallow /*/mobile_expanded_voter_list
Disallow /home/global_feed
Disallow /search?q=
Disallow /search/?q=
Disallow /digest/
Disallow /email_optout/
Disallow /qemail/
Disallow /invite/
Disallow /widgets/content_iframe/
Disallow /widgets/content_js/
Disallow /_/
Disallow /*_POST$
Disallow /*_POST/
Disallow /webnode2/
Disallow /anonymous/
Disallow /q/*/admin_log
Disallow /q/*/stats
Disallow /q/*/settings
Disallow /q/*/queue
Disallow /q/*/suggestions
Disallow /q/*/submissions
Disallow /q/*/quality
Disallow /q/*/earnings
Disallow /profile/*/Rss
Disallow /topic/*/Rss
Disallow */unanswered/*
Disallow */answer/
Disallow */answers/

facebot

Rule Path
Disallow /ajax/
Disallow /%40async
Disallow /*/%40async
Disallow /log/
Disallow /*/log
Disallow /*/about
Disallow /*/action
Disallow /*/activity
Disallow /*/all_questions
Disallow /*/all_posts$
Disallow /*/all_posts/
Disallow /*/blogs$
Disallow /*/blogs/
Disallow /*/followers
Disallow /*/following
Disallow /*/link/
Disallow /*/manage
Disallow /*/mentions
Disallow /*/merged
Disallow /*/open_questions
Disallow /*/posts$
Disallow /*/posts/
Disallow /*/questions
Disallow /*/related
Disallow /*/reviews$
Disallow /*/reviews/
Disallow /*/share
Disallow /*/top_questions
Disallow /*/topic-bio
Disallow /*/topic_bio
Disallow /*/topics
Disallow /*/comment
Disallow /comment/
Disallow /*/comments
Disallow /*/all_comments
Disallow /*/answer_comments
Disallow /*/mobile_collapsed
Disallow /*/mobile_expanded_voter_list
Disallow /home/global_feed
Disallow /search?q=
Disallow /search/?q=
Disallow /digest/
Disallow /email_optout/
Disallow /qemail/
Disallow /invite/
Disallow /widgets/content_iframe/
Disallow /widgets/content_js/
Disallow /_/
Disallow /*_POST$
Disallow /*_POST/
Disallow /webnode2/
Disallow /anonymous/
Disallow /q/*/admin_log
Disallow /q/*/stats
Disallow /q/*/settings
Disallow /q/*/queue
Disallow /q/*/suggestions
Disallow /q/*/submissions
Disallow /q/*/quality
Disallow /q/*/earnings
Disallow /profile/*/Rss
Disallow /topic/*/Rss

applebot

Rule Path
Disallow /ajax/
Disallow /%40async
Disallow /*/%40async
Disallow /log/
Disallow /*/log
Disallow /*/about
Disallow /*/action
Disallow /*/activity
Disallow /*/all_questions
Disallow /*/all_posts$
Disallow /*/all_posts/
Disallow /*/blogs$
Disallow /*/blogs/
Disallow /*/followers
Disallow /*/following
Disallow /*/link/
Disallow /*/manage
Disallow /*/mentions
Disallow /*/merged
Disallow /*/open_questions
Disallow /*/posts$
Disallow /*/posts/
Disallow /*/questions
Disallow /*/related
Disallow /*/reviews$
Disallow /*/reviews/
Disallow /*/share
Disallow /*/top_questions
Disallow /*/topic-bio
Disallow /*/topic_bio
Disallow /*/topics
Disallow /*/comment
Disallow /comment/
Disallow /*/comments
Disallow /*/all_comments
Disallow /*/answer_comments
Disallow /*/mobile_collapsed
Disallow /*/mobile_expanded_voter_list
Disallow /home/global_feed
Disallow /search?q=
Disallow /search/?q=
Disallow /digest/
Disallow /email_optout/
Disallow /qemail/
Disallow /invite/
Disallow /widgets/content_iframe/
Disallow /widgets/content_js/
Disallow /_/
Disallow /*_POST$
Disallow /*_POST/
Disallow /webnode2/
Disallow /anonymous/
Disallow /q/*/admin_log
Disallow /q/*/stats
Disallow /q/*/settings
Disallow /q/*/queue
Disallow /q/*/suggestions
Disallow /q/*/submissions
Disallow /q/*/quality
Disallow /q/*/earnings
Disallow /profile/*/Rss
Disallow /topic/*/Rss

twitterbot

Rule Path
Disallow /ajax/
Disallow /%40async
Disallow /*/%40async
Disallow /log/
Disallow /*/log
Disallow /*/about
Disallow /*/action
Disallow /*/activity
Disallow /*/all_questions
Disallow /*/all_posts$
Disallow /*/all_posts/
Disallow /*/blogs$
Disallow /*/blogs/
Disallow /*/followers
Disallow /*/following
Disallow /*/link/
Disallow /*/manage
Disallow /*/mentions
Disallow /*/merged
Disallow /*/open_questions
Disallow /*/posts$
Disallow /*/posts/
Disallow /*/questions
Disallow /*/related
Disallow /*/reviews$
Disallow /*/reviews/
Disallow /*/share
Disallow /*/top_questions
Disallow /*/topic-bio
Disallow /*/topic_bio
Disallow /*/topics
Disallow /*/comment
Disallow /comment/
Disallow /*/comments
Disallow /*/all_comments
Disallow /*/answer_comments
Disallow /*/mobile_collapsed
Disallow /*/mobile_expanded_voter_list
Disallow /home/global_feed
Disallow /search?q=
Disallow /search/?q=
Disallow /digest/
Disallow /email_optout/
Disallow /qemail/
Disallow /invite/
Disallow /widgets/content_iframe/
Disallow /widgets/content_js/
Disallow /_/
Disallow /*_POST$
Disallow /*_POST/
Disallow /webnode2/
Disallow /anonymous/
Disallow /q/*/admin_log
Disallow /q/*/stats
Disallow /q/*/settings
Disallow /q/*/queue
Disallow /q/*/suggestions
Disallow /q/*/submissions
Disallow /q/*/quality
Disallow /q/*/earnings
Disallow /profile/*/Rss
Disallow /topic/*/Rss

linkedinbot

Rule Path
Disallow /ajax/
Disallow /%40async
Disallow /*/%40async
Disallow /log/
Disallow /*/log
Disallow /*/about
Disallow /*/action
Disallow /*/activity
Disallow /*/all_questions
Disallow /*/all_posts$
Disallow /*/all_posts/
Disallow /*/blogs$
Disallow /*/blogs/
Disallow /*/followers
Disallow /*/following
Disallow /*/link/
Disallow /*/manage
Disallow /*/mentions
Disallow /*/merged
Disallow /*/open_questions
Disallow /*/posts$
Disallow /*/posts/
Disallow /*/questions
Disallow /*/related
Disallow /*/reviews$
Disallow /*/reviews/
Disallow /*/share
Disallow /*/top_questions
Disallow /*/topic-bio
Disallow /*/topic_bio
Disallow /*/topics
Disallow /*/comment
Disallow /comment/
Disallow /*/comments
Disallow /*/all_comments
Disallow /*/answer_comments
Disallow /*/mobile_collapsed
Disallow /*/mobile_expanded_voter_list
Disallow /home/global_feed
Disallow /search?q=
Disallow /search/?q=
Disallow /digest/
Disallow /email_optout/
Disallow /qemail/
Disallow /invite/
Disallow /widgets/content_iframe/
Disallow /widgets/content_js/
Disallow /_/
Disallow /*_POST$
Disallow /*_POST/
Disallow /webnode2/
Disallow /anonymous/
Disallow /q/*/admin_log
Disallow /q/*/stats
Disallow /q/*/settings
Disallow /q/*/queue
Disallow /q/*/suggestions
Disallow /q/*/submissions
Disallow /q/*/quality
Disallow /q/*/earnings
Disallow /profile/*/Rss
Disallow /topic/*/Rss

feedly

Rule Path
Disallow /ajax/
Disallow /%40async
Disallow /*/%40async
Disallow /log/
Disallow /*/log
Disallow /*/about
Disallow /*/action
Disallow /*/activity
Disallow /*/all_questions
Disallow /*/all_posts$
Disallow /*/all_posts/
Disallow /*/blogs$
Disallow /*/blogs/
Disallow /*/followers
Disallow /*/following
Disallow /*/link/
Disallow /*/manage
Disallow /*/mentions
Disallow /*/merged
Disallow /*/open_questions
Disallow /*/posts$
Disallow /*/posts/
Disallow /*/questions
Disallow /*/related
Disallow /*/reviews$
Disallow /*/reviews/
Disallow /*/share
Disallow /*/top_questions
Disallow /*/topic-bio
Disallow /*/topic_bio
Disallow /*/topics
Disallow /*/comment
Disallow /comment/
Disallow /*/comments
Disallow /*/all_comments
Disallow /*/answer_comments
Disallow /*/mobile_collapsed
Disallow /*/mobile_expanded_voter_list
Disallow /home/global_feed
Disallow /search?q=
Disallow /search/?q=
Disallow /digest/
Disallow /email_optout/
Disallow /qemail/
Disallow /invite/
Disallow /widgets/content_iframe/
Disallow /widgets/content_js/
Disallow /_/
Disallow /*_POST$
Disallow /*_POST/
Disallow /webnode2/
Disallow /anonymous/
Disallow /q/*/admin_log
Disallow /q/*/stats
Disallow /q/*/settings
Disallow /q/*/queue
Disallow /q/*/suggestions
Disallow /q/*/submissions
Disallow /q/*/quality
Disallow /q/*/earnings
Disallow /profile/*/Rss
Disallow /topic/*/Rss

pingdom

Rule Path
Disallow /ajax/
Disallow /%40async
Disallow /*/%40async
Disallow /log/
Disallow /*/log
Disallow /*/about
Disallow /*/action
Disallow /*/activity
Disallow /*/all_questions
Disallow /*/all_posts$
Disallow /*/all_posts/
Disallow /*/blogs$
Disallow /*/blogs/
Disallow /*/followers
Disallow /*/following
Disallow /*/link/
Disallow /*/manage
Disallow /*/mentions
Disallow /*/merged
Disallow /*/open_questions
Disallow /*/posts$
Disallow /*/posts/
Disallow /*/questions
Disallow /*/related
Disallow /*/reviews$
Disallow /*/reviews/
Disallow /*/share
Disallow /*/top_questions
Disallow /*/topic-bio
Disallow /*/topic_bio
Disallow /*/topics
Disallow /*/comment
Disallow /comment/
Disallow /*/comments
Disallow /*/all_comments
Disallow /*/answer_comments
Disallow /*/mobile_collapsed
Disallow /*/mobile_expanded_voter_list
Disallow /home/global_feed
Disallow /search?q=
Disallow /search/?q=
Disallow /digest/
Disallow /email_optout/
Disallow /qemail/
Disallow /invite/
Disallow /widgets/content_iframe/
Disallow /widgets/content_js/
Disallow /_/
Disallow /*_POST$
Disallow /*_POST/
Disallow /webnode2/
Disallow /anonymous/
Disallow /q/*/admin_log
Disallow /q/*/stats
Disallow /q/*/settings
Disallow /q/*/queue
Disallow /q/*/suggestions
Disallow /q/*/submissions
Disallow /q/*/quality
Disallow /q/*/earnings
Disallow /profile/*/Rss
Disallow /topic/*/Rss

neevabot

Rule Path
Disallow /ajax/
Disallow /%40async
Disallow /*/%40async
Disallow /log/
Disallow /*/log
Disallow /*/about
Disallow /*/action
Disallow /*/activity
Disallow /*/all_questions
Disallow /*/all_posts$
Disallow /*/all_posts/
Disallow /*/blogs$
Disallow /*/blogs/
Disallow /*/followers
Disallow /*/following
Disallow /*/link/
Disallow /*/manage
Disallow /*/mentions
Disallow /*/merged
Disallow /*/open_questions
Disallow /*/posts$
Disallow /*/posts/
Disallow /*/questions
Disallow /*/related
Disallow /*/reviews$
Disallow /*/reviews/
Disallow /*/share
Disallow /*/top_questions
Disallow /*/topic-bio
Disallow /*/topic_bio
Disallow /*/topics
Disallow /*/comment
Disallow /comment/
Disallow /*/comments
Disallow /*/all_comments
Disallow /*/answer_comments
Disallow /*/mobile_collapsed
Disallow /*/mobile_expanded_voter_list
Disallow /home/global_feed
Disallow /search?q=
Disallow /search/?q=
Disallow /digest/
Disallow /email_optout/
Disallow /qemail/
Disallow /invite/
Disallow /widgets/content_iframe/
Disallow /widgets/content_js/
Disallow /_/
Disallow /*_POST$
Disallow /*_POST/
Disallow /webnode2/
Disallow /anonymous/
Disallow /q/*/admin_log
Disallow /q/*/stats
Disallow /q/*/settings
Disallow /q/*/queue
Disallow /q/*/suggestions
Disallow /q/*/submissions
Disallow /q/*/quality
Disallow /q/*/earnings
Disallow /profile/*/Rss
Disallow /topic/*/Rss
Disallow */unanswered/*
Disallow */answer/
Disallow */answers/

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

jamesbot

Rule Path
Disallow /

*

Rule Path
Allow /$
Allow /about$
Allow /about/
Allow /challenges$
Allow /press$
Allow /login/
Allow /signup$
Disallow /
Disallow /ajax/
Disallow /%40async
Disallow /*/%40async
Disallow /log/
Disallow /*/log
Disallow /*/about
Disallow /*/action
Disallow /*/activity
Disallow /*/all_questions
Disallow /*/all_posts$
Disallow /*/all_posts/
Disallow /*/blogs$
Disallow /*/blogs/
Disallow /*/followers
Disallow /*/following
Disallow /*/link/
Disallow /*/manage
Disallow /*/mentions
Disallow /*/merged
Disallow /*/open_questions
Disallow /*/posts$
Disallow /*/posts/
Disallow /*/questions
Disallow /*/related
Disallow /*/reviews$
Disallow /*/reviews/
Disallow /*/share
Disallow /*/top_questions
Disallow /*/topic-bio
Disallow /*/topic_bio
Disallow /*/topics
Disallow /*/comment
Disallow /comment/
Disallow /*/comments
Disallow /*/all_comments
Disallow /*/answer_comments
Disallow /*/mobile_collapsed
Disallow /*/mobile_expanded_voter_list
Disallow /home/global_feed
Disallow /search?q=
Disallow /search/?q=
Disallow /digest/
Disallow /email_optout/
Disallow /qemail/
Disallow /invite/
Disallow /widgets/content_iframe/
Disallow /widgets/content_js/
Disallow /_/
Disallow /*_POST$
Disallow /*_POST/
Disallow /webnode2/
Disallow /anonymous/
Disallow /q/*/admin_log
Disallow /q/*/stats
Disallow /q/*/settings
Disallow /q/*/queue
Disallow /q/*/suggestions
Disallow /q/*/submissions
Disallow /q/*/quality
Disallow /q/*/earnings
Disallow /profile/*/Rss
Disallow /topic/*/Rss

Comments

  • If you operate a search engine and would like to crawl Quora,
  • you can email robotstxt@quora.com. Thanks.
  • To disallow GPTBot to access Quora.
  • People share a lot of sensitive material on Quora - controversial political
  • views, workplace gossip and compensation, and negative opinions held of
  • companies. Over many years, as they change jobs or change their views, it is
  • important that they can delete or anonymize their previously-written answers.
  • We opt out of the wayback machine because inclusion would allow people to
  • discover the identity of authors who had written sensitive answers publicly and
  • later had made them anonymous, and because it would prevent authors from being
  • able to remove their content from the internet if they change their mind about
  • publishing it. As far as we can tell, there is no way for sites to selectively
  • programmatically remove content from the archive and so this is the only way
  • for us to protect writers. If they open up an API where we can remove content
  • from the archive when authors remove it from Quora, but leave the rest of the
  • content archived, we would be happy to opt back in. See the page here:
  • https://archive.org/about/exclude.php
  • Meanwhile, if you are looking for an older version of any content on Quora, we
  • have full edit history tracked and accessible in product (with the exception of
  • content that has been removed by the author). You can generally access this by
  • clicking on timestamps, or by appending "/log" to the URL of any content page.
  • For any questions or feedback about this please visit our contact page
  • https://help.quora.com/hc/en-us/requests/new
  • Blocked since a lot of bad requests were made by this crawler.