windsor-aughtry.com
robots.txt

Robots Exclusion Standard data for windsor-aughtry.com

Resource Scan

Scan Details

Site Domain windsor-aughtry.com
Base Domain windsor-aughtry.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-18T03:29:15+00:00
Next Scan 2025-10-02T03:29:15+00:00

Last Successful Scan

Scanned2025-08-11T03:28:01+00:00
URL https://windsor-aughtry.com/robots.txt
Domain IPs 104.21.89.204, 172.67.164.225, 2606:4700:3032::6815:59cc, 2606:4700:3037::ac43:a4e1
Response IP 172.67.164.225
Found Yes
Hash 866f90bd678ba6295f8af443787c2f3010489108235e60d352f3b7dd1a5af5bf
SimHash 32261b04ebc1

Groups

*

Rule Path
Disallow /admin/
Disallow /includes/
Disallow /private/
Disallow /tmp/
Disallow /config/
Disallow /logs/
Disallow /*?ref=
Disallow /*?src=
Disallow /*?utm_*
Disallow /*%26*
Disallow /*.php$
Disallow /*.cgi$
Disallow /*.pl$
Disallow /login/
Disallow /register/
Disallow /account/
Disallow /cart/
Disallow /checkout/

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://windsor-aughtry.com/sitemap.xml

Comments

  • Blok direktori teknis atau tidak relevan
  • Blok parameter URL yang dapat menyebabkan duplikasi konten
  • Blok akses ke file script langsung (opsional jika ada routing .php)
  • Hindari crawl halaman akun atau checkout jika ada sistem login/transaksi
  • (Opsional) kurangi intensitas crawl
  • Sitemap lokasi