kyodemo.net
robots.txt

Robots Exclusion Standard data for kyodemo.net

Resource Scan

Scan Details

Site Domain kyodemo.net
Base Domain kyodemo.net
Scan Status Ok
Last Scan2024-06-24T16:29:14+00:00
Next Scan 2024-07-01T16:29:14+00:00

Last Scan

Scanned2024-06-24T16:29:14+00:00
URL https://kyodemo.net/robots.txt
Domain IPs 104.21.38.85, 172.67.220.199, 2606:4700:3031::ac43:dcc7, 2606:4700:3036::6815:2655
Response IP 172.67.220.199
Found Yes
Hash 422b0fea59a8ea6bc7a64c82f88ce8cfff045a58c49212956efe8851bb1f8bbe
SimHash 5fdfdb71cfb1

Groups

mj12bot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

steeler

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

slurp

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

seekport

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

clark-crawler2

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow

*

Rule Path
Allow /sdemo/*.html
Allow /sdemo/*.json
Allow /sdemo/com/
Disallow /idemo/
Disallow /demo/
Disallow /sdemo/a
Disallow /sdemo/c
Disallow /sdemo/h
Disallow /sdemo/j
Disallow /sdemo/m
Disallow /sdemo/t
Disallow /sdemo/w
Disallow /sdemo/r/f_
Disallow /sdemo/r/o_
Disallow /sdemo/r/n_
Disallow /sdemo/r/i_
Disallow /sdemo/r/e_
Disallow /sdemo/r/*-
Disallow /sdemo/r/*?
Disallow /sdemo/r/*/l
Disallow /sdemo/b/*?
Disallow /sdemo/b/n_
Disallow /sdemo/b/i_

Warnings

  • 2 invalid lines.