daniweb.com
robots.txt

Robots Exclusion Standard data for daniweb.com

Resource Scan

Scan Details

Site Domain daniweb.com
Base Domain daniweb.com
Scan Status Ok
Last Scan2024-09-21T02:04:19+00:00
Next Scan 2024-09-28T02:04:19+00:00

Last Scan

Scanned2024-09-21T02:04:19+00:00
URL https://daniweb.com/robots.txt
Domain IPs 172.66.40.216, 172.66.43.40, 2606:4700:3108::ac42:28d8, 2606:4700:3108::ac42:2b28
Response IP 172.66.40.216
Found Yes
Hash 6e06e96e74be1e2d4517570a56b5d5648864a91d5616fad0604a17fc91a768d0
SimHash c18e2c51c259

Groups

*

Rule Path Comment
Disallow /*?uri= -
Disallow /*?sort= -
Allow /articles/latest/recommended$ -
Allow /articles/latest/recommended/ -
Disallow /articles/latest/recommended/*?sort= -
Disallow /api/ -
Disallow /articles/donate/ -
Disallow /articles/latest -
Disallow /articles/latest/ -
Disallow /attachments/ -
Disallow /charts/ -
Disallow /*/_/ -
Disallow /community/tags/ -
Disallow /community/contribute -
Disallow /*/contribute$ -
Disallow /*/contribute/ -
Disallow /cron/ -
Disallow /editorial/ -
Disallow /email/ -
Disallow /errors/ -
Disallow /infractions/ -
Disallow /js/ -
Disallow /posts/index/ -
Disallow /bookmarks -
Disallow /bookmarks/ -
Disallow /posts/history/ -
Disallow /members/*/*/* -
Disallow /profiles/ -
Disallow /reports/ -
Disallow /search/ -
Disallow /stats/activity_stream -
Disallow /stats/tags -
Disallow /stats/tags/ -
Disallow /stats/members -
Disallow /stats/members/ -
Disallow /stats/comments -
Disallow /stats/comments/ -
Disallow /stats/members -
Disallow /stats/moderators -
Disallow /stats/toplist -
Disallow /stats/toplist/ -
Disallow /tags/*%2B* -
Disallow /uptime/ -
Disallow /tools/js/ -
Disallow /tools/badges -
Disallow /tools/badges/stream -
Disallow /tools/backlinks/batch/ -
Disallow /tools/backlinks/batches -
Disallow /tools/backlinks/batches/ -
Disallow /connect/ajax/ -
Disallow /connect/api/ -
Disallow /connect/ads/ -
Disallow /connect/apps -
Disallow /connect/apps/ -
Disallow /connect/audiences/ -
Disallow /connect/conversations/ -
Disallow /connect/cron/ -
Disallow /connect/developers/audiences -
Disallow /connect/developers/audiences/ -
Disallow /connect/developers/apps/ -
Disallow /connect/developers/swagger -
Disallow /connect/developers/swagger/ -
Disallow /connect/groups -
Disallow /connect/groups/ -
Disallow /connect/matches -
Disallow /connect/matches/ -
Disallow /connect/oauth/ -
Disallow /connect/payments/ -
Disallow /connect/profile$ -
Disallow /connect/profile/login -
Disallow /connect/profile/login/signup -
Disallow /connect/profile/password -
Disallow /connect/profile/community -
Disallow /connect/profile/community/ -
Disallow /connect/profile/complete -
Disallow /connect/profile/complete/ -
Disallow /connect/profile/update -
Disallow /connect/profile/update/ -
Disallow /connect/profile/muted -
Disallow /connect/profile/muted/ -
Disallow /connect/profile/password/ -
Disallow /connect/profile/logout -
Disallow /connect/profile/auth/ -
Disallow /connect/profile/apps -
Disallow /connect/profile/apps/ -
Disallow /connect/profile/delete -
Disallow /connect/profile/settings -
Disallow /connect/profile/settings/ -
Disallow /connect/profile/onboard -
Disallow /connect/profile/onboard/ -
Disallow /connect/profile/donations -
Disallow /connect/profile/donations/* -
Disallow /connect/profile/payments -
Disallow /connect/profile/payments/* -
Disallow /connect/urls/ -
Disallow /connect/users -
Disallow /connect/users/ -
Disallow /connect/welcome/onboard -
Disallow /connect/welcome/onboard/ -
Disallow /connect/welcome/invite/ -
Disallow /connect/welcome/unsubscribe/ -
Disallow /connect/welcome/verify/ -
Disallow /community/js_*_tagcloud/ Start of htaccess Legacy
Disallow /code/coder*.html -
Disallow /news/writer*.html -
Disallow /forums/finduser*.html -
Disallow /forums/member*.html -
Disallow /forums/search*.html -
Disallow /forums/unanswered*.html -
Disallow /certificates/stats.php Start of vB Legacy
Disallow /forums/attachment.php -
Disallow /forums/calendar.php -
Disallow /forums/data.php -
Disallow /forums/group.php -
Disallow /forums/showgroups.php -
Disallow /forums/member.php -
Disallow /forums/memberlist.php -
Disallow /forums/search.php -
Disallow /forums/stats.php -
Disallow /forums/tagcloud.php -

Other Records

Field Value
sitemap https://www.daniweb.com/latest-sitemap.xml
sitemap https://www.daniweb.com/sitemap.xml

Comments

  • Disallow: /community-center/say-hello/
  • Disallow: /community-center/geeks-lounge/
  • Disallow: /community-center/meta-daniweb/
  • Disallow: /tags/
  • Disallow: /forums/tag-*.html