Log In Sign Up

t.co.il
robots.txt

Robots Exclusion Standard data for t.co.il

Resource Scan

Scan Details

Site Domain t.co.il
Base Domain t.co.il
Scan Status Ok
Last Scan2026-02-09T09:00:08+00:00
Next Scan 2026-02-16T09:00:08+00:00

Last Scan

Scanned2026-02-09T09:00:08+00:00
URL https://t.co.il/robots.txt
Redirect https://www.t.co.il/robots.txt
Redirect Domain www.t.co.il
Redirect Base t.co.il
Domain IPs 104.26.8.235, 104.26.9.235, 172.67.71.72, 2606:4700:20::681a:8eb, 2606:4700:20::681a:9eb, 2606:4700:20::ac43:4748
Redirect IPs 104.26.8.235, 104.26.9.235, 172.67.71.72, 2606:4700:20::681a:8eb, 2606:4700:20::681a:9eb, 2606:4700:20::ac43:4748
Response IP 104.26.8.235
Found Yes
Hash 1545401a5d938ca6f0759386b32a23dbd46071fd62a2ecb87eed31cd950a2ebf
SimHash 01578ec88aaa

Groups

*

Rule Path
Disallow /dashboard/
Disallow /lp/active-business/
Disallow /admin/
Disallow /onboarding

mj12bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baiduspider+(+http://www.baidu.com/search/spider.htm)

Rule Path
Disallow /

baiduspider/2.0;+http://www.baidu.com/search/spider.html

Rule Path
Disallow /

baiduspider/2.0

Rule Path
Disallow /

mozilla/5.0(compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /
Back to top

Comments

  • Disallow: /
  • Baiduspider
Back to top

Warnings

  • 6 invalid lines.
Back to top