activedevelopment.co.nz
robots.txt

Robots Exclusion Standard data for activedevelopment.co.nz

Resource Scan

Scan Details

Site Domain activedevelopment.co.nz
Base Domain activedevelopment.co.nz
Scan Status Ok
Last Scan2025-08-19T21:50:40+00:00
Next Scan 2025-09-18T21:50:40+00:00

Last Scan

Scanned2025-08-19T21:50:40+00:00
URL https://activedevelopment.co.nz/robots.txt
Domain IPs 75.2.60.5, 99.83.231.61
Response IP 75.2.60.5
Found Yes
Hash 9bb06542ec57bf8956a9cb48ffa8be2145d92b6810a00a5c213aad87b6780904
SimHash 415dce106517

Groups

*

Rule Path
Allow /

Other Records

Field Value
sitemap https://activedevelopment.co.nz/sitemap.xml

Comments

  • Robots.txt for Active Development
  • Allow all bots
  • Disallow admin or private paths (if any exist)
  • Disallow: /admin/
  • Disallow: /api/
  • Sitemap location
  • Crawl delay (optional - uncomment if needed)
  • Crawl-delay: 1