horsetalk.co.nz
robots.txt

Robots Exclusion Standard data for horsetalk.co.nz

Resource Scan

Scan Details

Site Domain horsetalk.co.nz
Base Domain horsetalk.co.nz
Scan Status Ok
Last Scan2024-09-18T14:50:13+00:00
Next Scan 2024-09-25T14:50:13+00:00

Last Scan

Scanned2024-09-18T14:50:13+00:00
URL https://horsetalk.co.nz/robots.txt
Domain IPs 114.23.97.33
Response IP 114.23.97.33
Found Yes
Hash cb90b6aafb6495525d2ca92426467fa087bf050d739eb9811154eb9df06c5e2a
SimHash c85648028537

Groups

ccbot

Rule Path
Disallow /

chatgtp-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

antrhopic-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

commoncrawl

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

imagesift

Rule Path
Disallow /

cohere

Rule Path
Disallow /

browseai

Rule Path
Disallow /

Warnings

  • 1 invalid line.