Log In Sign Up

ilanekle.net
robots.txt

Robots Exclusion Standard data for ilanekle.net

Resource Scan

Scan Details

Site Domain ilanekle.net
Base Domain ilanekle.net
Scan Status Ok
Last Scan2024-11-13T13:12:35+00:00
Next Scan 2024-11-20T13:12:35+00:00

Last Scan

Scanned2024-11-13T13:12:35+00:00
URL https://ilanekle.net/robots.txt
Domain IPs 193.38.34.5
Response IP 193.38.34.5
Found Yes
Hash 1562d603db2a97317d140457985a1c8280c36dda36bdde391dd6c2f2c90cdae6
SimHash 0815474247b9

Groups

*

Rule Path
Allow /
Allow *?page
Disallow *?p*
Disallow *?q*
Disallow /KWPanel
Disallow /ajax*

mj12bot

Rule Path
Allow /

yandex

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

bingbot

Rule Path
Allow /

scrappy

Rule Path
Allow /

scrappy bot

Rule Path
Allow /

discobot

Rule Path
Allow /

dotbot

Rule Path
Allow /

yacybot

Rule Path
Allow /

googlebot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

slurp

Rule Path
Allow /

gulliver

Rule Path
Allow /

fast

Rule Path
Allow /

scooter

Rule Path
Allow /

ia_archiver

Rule Path
Allow /

nutch

Rule Path
Allow /

fast-webcrawler

Rule Path
Allow /

teoma

Rule Path
Allow /

lycos_spider_(t-rex)

Rule Path
Allow /

coccoc

Rule Path
Allow /

googlemediabot*

Rule Path
Allow /

vscooter

Rule Path
Allow /

msnptc

Rule Path
Allow /

msnbot-media/1.0

Rule Path
Allow /

architextspider

Rule Path
Allow /

zyborg

Rule Path
Allow /

surveybot

Rule Path
Allow /

alexabot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

jikespider

Rule Path
Allow /

naverbot

Rule Path
Allow /

yeti

Rule Path
Allow /

linespider

Rule Path
Allow /

baiduspider-image

Rule Path
Allow /

youdaobot

Rule Path
Allow /

sogou web spider

Rule Path
Allow /

sogou inst spider

Rule Path
Allow /

sogou spider2

Rule Path
Allow /

sogou blog

Rule Path
Allow /

sogou news spider

Rule Path
Allow /

sogou orion spider

Rule Path
Allow /

chinasospider

Rule Path
Allow /

sosospider

Rule Path
Allow /

yisouspider

Rule Path
Allow /

easouspider

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

yaanibot

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

semrushbot-sa

Rule Path
Allow /

semrushbot-si

Rule Path
Allow /

yahoo pipes 2.0

Rule Path
Allow /

adidxbot

Rule Path
Allow /

applebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

seznambot

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

daumoa

Rule Path
Allow /

orangebot

Rule Path
Allow /

orangebot-collector

Rule Path
Allow /

stackrambler

Rule Path
Allow /

mail.ru_bot

Rule Path
Allow /

vebidoobot

Rule Path
Allow /

rogerbot

Rule Path
Allow /

deepcrawl

Rule Path
Allow /

bytespider

Rule Path
Allow /

cincobot

Rule Path
Allow /

jooblebot

Rule Path
Allow /

flipboard

Rule Path
Allow /

yandexnews

Rule Path
Allow /

proximic

Rule Path
Allow /

voltron

Rule Path
Allow /

bender

Rule Path
Allow /

piplbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://ilanekle.net/sitemap.xml
Back to top

Comments

  • www.ilanekle.net
Back to top

Warnings

  • 2 invalid lines.
Back to top