Log In Sign Up

terminals.io
robots.txt

Robots Exclusion Standard data for terminals.io

Resource Scan

Scan Details

Site Domain terminals.io
Base Domain terminals.io
Scan Status Ok
Last Scan2025-09-29T15:47:46+00:00
Next Scan 2025-10-13T15:47:46+00:00

Last Scan

Scanned2025-09-29T15:47:46+00:00
URL https://terminals.io/robots.txt
Redirect https://www.terminals.io/robots.txt
Redirect Domain www.terminals.io
Redirect Base terminals.io
Domain IPs 104.26.12.233, 104.26.13.233, 172.67.68.129
Redirect IPs 104.26.12.233, 104.26.13.233, 172.67.68.129
Response IP 172.67.68.129
Found Yes
Hash fafce85c26e546566bf3f63522e9622dd291c5663b5ca1b8a4fbfa71b2fc8a22
SimHash 34914840c0b7

Groups

*

Rule Path
Disallow /admin
Disallow /user/

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

google-cloudvertexbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

kangaroo bot

Rule Path
Disallow /

pangubot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

cohere-training-data-crawler

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /

youbot

Rule Path
Disallow /

flyriverbot

No rules defined. All paths allowed.

Back to top