goodleafdev.com
robots.txt

Robots Exclusion Standard data for goodleafdev.com

Resource Scan

Scan Details

Site Domain goodleafdev.com
Base Domain goodleafdev.com
Scan Status Ok
Last Scan2025-12-05T07:14:23+00:00
Next Scan 2026-01-04T07:14:23+00:00

Last Scan

Scanned2025-12-05T07:14:23+00:00
URL https://goodleafdev.com/robots.txt
Domain IPs 172.66.40.158, 172.66.43.98, 2606:4700:3108::ac42:289e, 2606:4700:3108::ac42:2b62
Response IP 172.66.40.158
Found Yes
Hash 28d889edc60374a0464bf5430ea7208f1005226fe9542d2abb98b803d2dcbf31
SimHash 6700c872e7a7

Groups

*

Rule Path
Allow /
Allow /vps/
Allow /dedicated-servers/
Allow /game/
Allow /colocation/
Allow /network/
Allow /about/
Allow /datacenters/
Disallow /admin/
Disallow /api/
Disallow /_next/
Disallow /private/
Allow /_next/static/
Allow /icons/
Allow /img/
Allow /bg/

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

Other Records

Field Value
sitemap https://goodleaf.host/sitemap.xml

Comments

  • Important pages
  • Disallow admin/sensitive areas
  • Allow search engines to access important resources
  • Sitemap location
  • Crawl delay to be respectful
  • Popular search engines