horsetalk.co.nz
robots.txt

Robots Exclusion Standard data for horsetalk.co.nz

Resource Scan

Scan Details

Site Domain horsetalk.co.nz
Base Domain horsetalk.co.nz
Scan Status Ok
Last Scan2024-11-15T18:18:35+00:00
Next Scan 2024-11-22T18:18:35+00:00

Last Scan

Scanned2024-11-15T18:18:35+00:00
URL https://horsetalk.co.nz/robots.txt
Domain IPs 114.23.97.33
Response IP 114.23.97.33
Found Yes
Hash 5166be8e96f521ae1de073bbe37571ad6f4b1976b34879ed1b69665822ed3e29
SimHash c05648068575

Groups

ccbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

chatgtp-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

antrhopic-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

commoncrawl

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

imagesift

Rule Path
Disallow /

cohere

Rule Path
Disallow /

browseai

Rule Path
Disallow /

Warnings

  • 1 invalid line.