Log In Sign Up

defconlevel.com
robots.txt

Robots Exclusion Standard data for defconlevel.com

Resource Scan

Scan Details

Site Domain defconlevel.com
Base Domain defconlevel.com
Scan Status Ok
Last Scan2026-02-23T17:00:57+00:00
Next Scan 2026-03-02T17:00:57+00:00

Last Scan

Scanned2026-02-23T17:00:57+00:00
URL https://defconlevel.com/robots.txt
Domain IPs 104.21.26.161, 172.67.137.49, 2606:4700:3033::6815:1aa1, 2606:4700:3036::ac43:8931
Response IP 172.67.137.49
Found Yes
Hash 1fb769417cd433e8e3316321d7bf8d81f4dbe968fbc76be3030c920d151901dc
SimHash 2546de11e473

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /setup/
Disallow /config/
Disallow /cache/
Disallow /storage/
Disallow /src/
Disallow /debug/
Disallow /public/debug/
Disallow /api/
Disallow /*debug*
Disallow /*test*
Disallow /*.sql$
Disallow /*.log$
Disallow /uploads/
Allow /alert/
Allow /alerts/
Allow /css/
Allow /js/
Allow /images/

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://www.defconlevel.com/sitemap.xml
sitemap https://www.defconlevel.com/sitemap.php
sitemap https://www.defconlevel.com/news-sitemap.xml
Back to top

Comments

  • DEFCON Level Website - robots.txt
  • Last Updated: 2026-01-03
  • Allow all search engines
  • Allow crawling of all public pages
  • Block admin and sensitive directories
  • Block API endpoints (not for public indexing)
  • Block debug and test files
  • Block upload directories (but allow images via Allow below)
  • Explicitly ALLOW alert pages (do NOT block)
  • Allow CSS, JS, and images
  • Crawl delay (be nice to server)
  • Sitemaps
  • News sitemap for Google News
Back to top