thestreamable.com
robots.txt

Robots Exclusion Standard data for thestreamable.com

Resource Scan

Scan Details

Site Domain thestreamable.com
Base Domain thestreamable.com
Scan Status Ok
Last Scan2025-10-07T18:13:37+00:00
Next Scan 2025-10-14T18:13:37+00:00

Last Scan

Scanned2025-10-07T18:13:37+00:00
URL https://thestreamable.com/robots.txt
Domain IPs 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91
Response IP 151.101.65.91
Found Yes
Hash 458f08a144f825c4aaab7742f840dfeaf080a2a1e3300899d5d59bd35fbd75ce
SimHash 721811520478

Groups

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

cohere-training-data-crawler

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

quillbot.com

Rule Path
Disallow /

quora-bot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

aliyunsecbot

Rule Path
Disallow /

audigentadbot

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

echoboxbot

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

jetslide

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mycentralaiscraperbot

Rule Path
Disallow /

newsnow

Rule Path
Disallow /

news-please

Rule Path
Disallow /

peer39_crawler

Rule Path
Disallow /

peer39_crawler/1.0

Rule Path
Disallow /

poseidon research crawler

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

seekrbot

Rule Path
Disallow /

seznamhomepagecrawler

Rule Path
Disallow /

taragroup intelligent bot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

viennatinybot

Rule Path
Disallow /

*

Rule Path
Disallow /*?url=
Disallow /*?teams=
Disallow /*?matchmaker=
Disallow /*?pageid=
Disallow /*?market=
Disallow /*%26url%3D
Disallow /*%26teams%3D
Disallow /*%26matchmaker%3D
Disallow /*%26pageid%3D
Disallow /*%26market%3D
Disallow /panel
Disallow /k-api
Disallow /cdn-cgi/
Allow /cdn-cgi/image/
Allow /ads.txt
Disallow /*.csv$
Disallow /*.txt$
Disallow /*.json$
Disallow */api/analytics/
Disallow */doctrine/ajax/

Other Records

Field Value
sitemap https://thestreamable.com/sitemap.xml
sitemap https://thestreamable.com/news.xml
sitemap https://thestreamable.com/evergreen-deals/sitemap.xml
sitemap https://thestreamable.com/evergreen-how-to-watch/sitemap.xml
sitemap https://thestreamable.com/feature-editorial/sitemap.xml
sitemap https://thestreamable.com/review/sitemap.xml
sitemap https://thestreamable.com/help-how-to/sitemap.xml
sitemap https://thestreamable.com/best-list/sitemap.xml
sitemap https://thestreamable.com/versus/sitemap.xml