Log In Sign Up

jetsotoday.com
robots.txt

Robots Exclusion Standard data for jetsotoday.com

Resource Scan

Scan Details

Site Domain jetsotoday.com
Base Domain jetsotoday.com
Scan Status Ok
Last Scan2025-05-29T18:43:51+00:00
Next Scan 2025-06-05T18:43:51+00:00

Last Scan

Scanned2025-05-29T18:43:51+00:00
URL https://jetsotoday.com/robots.txt
Domain IPs 104.26.2.40, 104.26.3.40, 172.67.71.95, 2606:4700:20::681a:228, 2606:4700:20::681a:328, 2606:4700:20::ac43:475f
Response IP 104.26.2.40
Found Yes
Hash 5a62272b0146f695e261ef27aec6ec71eb33ef0f356546b0990d34d362eae215
SimHash f29e4370a800

Groups

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /
Disallow /

imagesiftbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sougou web spider

Rule Path
Disallow /

linespider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

crowdtanglebot

Rule Path
Disallow /

sitelockspider

Rule Path
Disallow /

proximic

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

leikibot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgili/0.5 +https://omgili.com

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

yak

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

crowdtanglebot

Rule Path
Disallow /

leikibot

Rule Path
Disallow /

sitelockspider

Rule Path
Disallow /

criteobot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /
Disallow /*?*wordfence_lh
Back to top

Warnings

  • 2 invalid lines.
  • `user -agent` is not a known field.
Back to top