haystack.tv
robots.txt

Robots Exclusion Standard data for haystack.tv

Resource Scan

Scan Details

Site Domain haystack.tv
Base Domain haystack.tv
Scan Status Ok
Last Scan2024-11-09T08:50:53+00:00
Next Scan 2024-11-16T08:50:53+00:00

Last Scan

Scanned2024-11-09T08:50:53+00:00
URL https://haystack.tv/robots.txt
Redirect https://www.haystack.tv/robots.txt
Redirect Domain www.haystack.tv
Redirect Base haystack.tv
Domain IPs 13.33.88.100, 13.33.88.15, 13.33.88.38, 13.33.88.76
Redirect IPs 13.33.88.100, 13.33.88.15, 13.33.88.38, 13.33.88.76, 2600:9000:223b:2600:8:263e:3c80:93a1, 2600:9000:223b:5a00:8:263e:3c80:93a1, 2600:9000:223b:6c00:8:263e:3c80:93a1, 2600:9000:223b:9600:8:263e:3c80:93a1, 2600:9000:223b:c600:8:263e:3c80:93a1, 2600:9000:223b:da00:8:263e:3c80:93a1, 2600:9000:223b:f200:8:263e:3c80:93a1, 2600:9000:223b:f400:8:263e:3c80:93a1
Response IP 13.33.88.15
Found Yes
Hash 6eb418aa45378be58b6c6616cea436bedad8344da4c29af2789c5a624583608d
SimHash 400c49438115

Groups

yandex

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

seekr

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

youbot

Rule Path
Disallow /

*

Rule Path
Allow /
Disallow /activate*
Disallow /signin*

Other Records

Field Value
sitemap http://www.haystack.tv/sitemap.xml