linuxquestions.org
robots.txt

Robots Exclusion Standard data for linuxquestions.org

Resource Scan

Scan Details

Site Domain linuxquestions.org
Base Domain linuxquestions.org
Scan Status Ok
Last Scan2024-06-17T10:51:07+00:00
Next Scan 2024-06-24T10:51:07+00:00

Last Scan

Scanned2024-06-17T10:51:07+00:00
URL https://linuxquestions.org/robots.txt
Redirect https://www.linuxquestions.org/robots.txt
Redirect Domain www.linuxquestions.org
Redirect Base linuxquestions.org
Domain IPs 35.244.195.25
Redirect IPs 104.24.136.8, 104.24.137.8, 172.67.81.99, 2606:4700:20::6818:8808, 2606:4700:20::6818:8908, 2606:4700:20::ac43:5163
Response IP 104.24.137.8
Found Yes
Hash aeb759ddd037d4ff45b12af6185f5aa40fc11138ba025223190451ef8c53f7a4
SimHash 2416d48c4695

Groups

*

Rule Path
Disallow /questions/attachment.php
Disallow /questions/avatar.php
Disallow /questions/editpost.php
Disallow /questions/member.php
Disallow /questions/memberlist.php
Disallow /questions/misc.php
Disallow /questions/moderator.php
Disallow /questions/newreply.php
Disallow /questions/newthread.php
Disallow /questions/online.php
Disallow /questions/postings.php
Disallow /questions/printthread.php
Disallow /questions/private.php
Disallow /questions/private2.php
Disallow /questions/report.php
Disallow /questions/lqspy.php
Disallow /questions/search.php
Disallow /questions/sendtofriend.php
Disallow /questions/threadrate.php
Disallow /questions/usercp.php
Disallow /questions/social.php
Disallow /questions/images/
Disallow /questions/calendar.php
Disallow /questions/sendmessage.php
Disallow /questions/*nextnewest
Disallow /questions/*nextoldest
Disallow /bookmarks/
Disallow /user/
Disallow /questions/user/
Disallow /questions/tags/
Disallow /hcl/

mediapartners-google

Rule Path
Disallow

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

mozilla/5.0 (compatible;contxbot/1.0)

Rule Path
Disallow

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

Other Records

Field Value
sitemap https://www.linuxquestions.org/questions/sitemap/sitemap_index.xml.gz