coderanch.com
robots.txt

Robots Exclusion Standard data for coderanch.com

Resource Scan

Scan Details

Site Domain coderanch.com
Base Domain coderanch.com
Scan Status Ok
Last Scan2/25/2025, 11:50:17 AM
Next Scan 3/27/2025, 11:50:17 AM

Last Scan

Scanned2/25/2025, 11:50:17 AM
URL https://coderanch.com/robots.txt
Domain IPs 104.21.42.223, 172.67.167.32, 2606:4700:3032::ac43:a720, 2606:4700:3035::6815:2adf
Response IP 172.67.167.32
Found Yes
Hash 626c83e1f0686d5c0e9bdd2da95b643d9b599458b0d59adae1b7561dbeef4e16
SimHash 7006dc1b47c6

Groups

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

*

Rule Path
Disallow
Disallow /forums/bookmarks/
Disallow /forums/forums/readAll/
Disallow /forums/jforum
Disallow /forums/pm/sendTo/
Disallow /forums/pm/
Disallow /forums/posts/quote/
Disallow /forums/posts/reply/
Disallow /forums/posts/watch/
Disallow /forums/posts/unwatch/
Disallow /forums/posts/preList/
Disallow /forums/posts/prelist/
Disallow /forums/posts/like/
Disallow /forums/posts/unlike/
Disallow /forums/posts/reqResolved/
Disallow /forums/posts/listByUser/
Disallow /forums/posts/listTopicsByUser/
Disallow /forums/posts/downloadAttach/
Disallow /forums/rss/
Disallow /forums/search/
Disallow /forums/user/edit/
Disallow /forums/user/lostPassword
Disallow /forums/user/insert
Disallow /forums/user/login
Disallow /forums/user/activateManual
Disallow /captcha/
Disallow /how-to/search
Disallow /how-to/java/old/
Disallow /how-to/view?old%2F
Disallow /how-to/button
Disallow /how-to/diff
Disallow /how-to/edit
Disallow /pp/
Disallow /v/
Disallow /news/*/*

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

Other Records

Field Value
sitemap https://coderanch.com/sitemap/sitemapindex.xml

Comments

  • This file is read and obeyed by polite search engines
  • Please refer to the W3C notes:
  • http://www.w3.org/TR/html4/appendix/notes.html#h-B.4.1.1
  • or http://info.webcrawler.com/mak/projects/robots/exclusion.html
  • Slurp is Yahoo's crawler
  • Site map