Log In Sign Up

ilanekle.net
robots.txt

Robots Exclusion Standard data for ilanekle.net

Resource Scan

Scan Details

Site Domain ilanekle.net
Base Domain ilanekle.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-03-27T19:20:41+00:00
Next Scan 2024-06-25T19:20:41+00:00

Last Successful Scan

Scanned2023-11-29T19:18:55+00:00
URL https://ilanekle.net/robots.txt
Domain IPs 46.8.194.20
Response IP 46.8.194.20
Found Yes
Hash 1562d603db2a97317d140457985a1c8280c36dda36bdde391dd6c2f2c90cdae6
SimHash 0815474247b9

Groups

*

Rule Path
Allow /
Allow *?page
Disallow *?p*
Disallow *?q*
Disallow /KWPanel
Disallow /ajax*

mj12bot

Rule Path
Allow /

yandex

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

bingbot

Rule Path
Allow /

scrappy

Rule Path
Allow /

scrappy bot

Rule Path
Allow /

discobot

Rule Path
Allow /

dotbot

Rule Path
Allow /

yacybot

Rule Path
Allow /

googlebot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

slurp

Rule Path
Allow /

gulliver

Rule Path
Allow /

fast

Rule Path
Allow /

scooter

Rule Path
Allow /

ia_archiver

Rule Path
Allow /

nutch

Rule Path
Allow /

fast-webcrawler

Rule Path
Allow /

teoma

Rule Path
Allow /

lycos_spider_(t-rex)

Rule Path
Allow /

coccoc

Rule Path
Allow /

googlemediabot*

Rule Path
Allow /

vscooter

Rule Path
Allow /

msnptc

Rule Path
Allow /

msnbot-media/1.0

Rule Path
Allow /

architextspider

Rule Path
Allow /

zyborg

Rule Path
Allow /

surveybot

Rule Path
Allow /

alexabot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

jikespider

Rule Path
Allow /

naverbot

Rule Path
Allow /

yeti

Rule Path
Allow /

linespider

Rule Path
Allow /

baiduspider-image

Rule Path
Allow /

youdaobot

Rule Path
Allow /

sogou web spider

Rule Path
Allow /

sogou inst spider

Rule Path
Allow /

sogou spider2

Rule Path
Allow /

sogou blog

Rule Path
Allow /

sogou news spider

Rule Path
Allow /

sogou orion spider

Rule Path
Allow /

chinasospider

Rule Path
Allow /

sosospider

Rule Path
Allow /

yisouspider

Rule Path
Allow /

easouspider

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

yaanibot

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

semrushbot-sa

Rule Path
Allow /

semrushbot-si

Rule Path
Allow /

yahoo pipes 2.0

Rule Path
Allow /

adidxbot

Rule Path
Allow /

applebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

seznambot

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

daumoa

Rule Path
Allow /

orangebot

Rule Path
Allow /

orangebot-collector

Rule Path
Allow /

stackrambler

Rule Path
Allow /

mail.ru_bot

Rule Path
Allow /

vebidoobot

Rule Path
Allow /

rogerbot

Rule Path
Allow /

deepcrawl

Rule Path
Allow /

bytespider

Rule Path
Allow /

cincobot

Rule Path
Allow /

jooblebot

Rule Path
Allow /

flipboard

Rule Path
Allow /

yandexnews

Rule Path
Allow /

proximic

Rule Path
Allow /

voltron

Rule Path
Allow /

bender

Rule Path
Allow /

piplbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://ilanekle.net/sitemap.xml
Back to top

Comments

  • www.ilanekle.net
Back to top

Warnings

  • 2 invalid lines.
Back to top