donel.su
robots.txt

Robots Exclusion Standard data for donel.su

Resource Scan

Scan Details

Site Domain donel.su
Base Domain donel.su
Scan Status Ok
Last Scan2024-11-03T11:32:12+00:00
Next Scan 2024-12-03T11:32:12+00:00

Last Scan

Scanned2024-11-03T11:32:12+00:00
URL https://donel.su/robots.txt
Domain IPs 87.249.41.112
Response IP 87.249.41.112
Found Yes
Hash 2049dc3221b85a665af12f4e603bde2c45c3afda6ed9f39208bfd1acc61c5962
SimHash c61b59620627

Groups

yandex

Rule Path
Disallow /Admin
Disallow /include
Disallow /privacy
Disallow /compare
Disallow /oferta
Disallow /cart
Disallow /order_complete
Disallow /favorites
Disallow *?*
Disallow /search*
Allow /files/imagecache
Disallow /upload*
Disallow /login*
Disallow /cabinet*
Disallow /configure*

googlebot

Rule Path
Disallow /Admin
Disallow /include
Disallow /privacy
Allow /Admin/js*
Allow /css*
Disallow /compare
Disallow /oferta
Disallow /cart
Disallow /order_complete
Disallow /favorites
Disallow *?*
Disallow /search*
Disallow /upload*
Disallow /login*
Disallow /cabinet*
Disallow /configure*
Disallow /files/imagecache*

googlebot-image

Rule Path
Allow /

*

Rule Path
Disallow /Admin
Disallow /include
Disallow /compare
Disallow /privacy
Disallow /oferta
Disallow /cart
Disallow /order_complete
Disallow /favorites
Disallow *?*
Disallow /search*
Disallow /upload*
Disallow /login*
Disallow /cabinet*
Disallow /configure*

dataforseobot/1.0

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

semrushbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

flipboardproxy

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

trovitbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

detectify

Rule Path
Disallow /

riddler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

disco pump

Rule Path
Disallow /

wget

Rule Path
Disallow /

teleport pro

Rule Path
Disallow /

psbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

netinfobot

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

omniexplorer_bot

Rule Path
Disallow /

w3c_validator

Rule Path
Disallow /

accoona

Rule Path
Disallow /

ask jeeves

Rule Path
Disallow /

webalta

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

tourlentabot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

nigma.ru

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

statsbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

findlinks

Rule Path
Disallow /

proximic

Rule Path
Disallow /

openindexspider

Rule Path
Disallow /

statdom.ru

Rule Path
Disallow /

exabot

Rule Path
Disallow /

spider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

obot

Rule Path
Disallow /

c-t bot

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

snoopy

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

domainvader

Rule Path
Disallow /

yeti

Rule Path
Disallow /

dcpbot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

megaindex.ru/2.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

Other Records

Field Value
sitemap https://donel.su/sitemap.xml

Warnings

  • `host` is not a known field.