Log In Sign Up

hackread.com
robots.txt

Robots Exclusion Standard data for hackread.com

Resource Scan

Scan Details

Site Domain hackread.com
Base Domain hackread.com
Scan Status Ok
Last Scan2026-02-05T07:58:28+00:00
Next Scan 2026-02-12T07:58:28+00:00

Last Scan

Scanned2026-02-05T07:58:28+00:00
URL https://hackread.com/robots.txt
Domain IPs 104.21.36.114, 172.67.192.211, 2606:4700:3030::6815:2472, 2606:4700:3031::ac43:c0d3
Response IP 172.67.192.211
Found Yes
Hash 324bc6d7e7267794ffa40053723a84c1a41ae07d7fa76b6dffe58f64ea569fa0
SimHash 49e251504499

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/cache/
Disallow /cgi-bin/
Disallow /trackback
Disallow /*trackback*
Disallow /?s=
Disallow /*feed*
Disallow /*.html/$
Allow /wp-admin/admin-ajax.php
Allow /wp-content/uploads/

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Disallow

googlebot-news

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

twitterbot

Rule Path
Disallow
Allow /images
Allow /archives

*

Rule Path
Disallow /wp-content/uploads/2011/
Disallow /wp-content/uploads/2012/
Disallow /wp-content/uploads/2013/
Disallow /wp-content/uploads/2014/
Disallow /wp-content/uploads/2015/
Disallow /wp-content/uploads/2016/
Disallow /wp-content/uploads/2017/
Disallow /wp-content/uploads/2018/
Disallow /wp-content/uploads/2019/
Disallow /wp-content/uploads/2020/

*

Rule Path
Disallow /webdisk/
Disallow /cpanel/
Disallow /webmail/
Disallow /whm/
Disallow /wwww/
Disallow /wwwww/
Disallow /wwwwww/
Back to top

Other Records

Field Value
sitemap https://www.hackread.com/sitemap.xml
Back to top

Comments

  • General rules for all bots
  • Ensure full access for Google crawlers (Search, News, Ads)
  • Allow specific content for Twitterbot
  • Sitemap location
  • Block old uploads folders from indexing
  • Prevent indexing of invalid or service subdomains
Back to top