Log In Sign Up

drkevinho.com.au
robots.txt

Robots Exclusion Standard data for drkevinho.com.au

Resource Scan

Scan Details

Site Domain drkevinho.com.au
Base Domain drkevinho.com.au
Scan Status Ok
Last Scan2025-11-09T05:15:55+00:00
Next Scan 2025-12-09T05:15:55+00:00

Last Scan

Scanned2025-11-09T05:15:55+00:00
URL https://drkevinho.com.au/robots.txt
Domain IPs 104.21.82.239, 172.67.209.74, 2606:4700:3032::6815:52ef, 2606:4700:3035::ac43:d14a
Response IP 172.67.209.74
Found Yes
Hash f0aff41db8df5ede905d076c40415eff604718b57f113fd7c1229e3a3ce8103d
SimHash 7444c9596533

Groups

*

Rule Path
Allow /wp-content/uploads/
Disallow /wp-admin/
Disallow /readme.html
Disallow /refer/
Disallow /cgi-bin/
Disallow /?s=
Disallow /blog/?s=
Disallow /trackback/
Disallow /comments/
Disallow /tag/
Disallow /date/
Disallow /category/
Disallow /archives/
Disallow /amp/
Disallow /feed/
Disallow /*?noamp=mobile
Disallow /?amp
Allow /*.js$
Allow /*.css$
Back to top

Other Records

Field Value
sitemap https://www.drkevinho.com.au/sitemap_index.xml
Back to top

Comments

  • Allow all bots
  • Allow important directories
  • Disallow unnecessary or sensitive areas
  • Disallow: /wp-includes/
  • Disallow: /author/
  • Prevent indexing of redundant archives (if not needed)
  • Allow CSS and JS files for proper rendering
  • Reference your sitemap for better crawling
Back to top