malikshaw.com
robots.txt

Robots Exclusion Standard data for malikshaw.com

Resource Scan

Scan Details

Site Domain malikshaw.com
Base Domain malikshaw.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-10-24T12:33:37+00:00
Next Scan 2024-11-23T12:33:37+00:00

Last Successful Scan

Scanned2024-09-02T12:32:14+00:00
URL https://www.malikshaw.com/robots.txt
Domain IPs 94.237.53.143
Response IP 94.237.53.143
Found Yes
Hash 288844709aa22daa5c31a26d1e0d8e175dffb73a60de54c5a9a322fecb019487
SimHash 4b16d1e04833

Groups

*

Rule Path
Disallow /administrator/
Allow /cache/
Disallow /cli/
Allow /components/
Allow /images/
Disallow /includes/
Disallow /installation/
Disallow /language/
Allow /libraries/
Disallow /logs/
Allow /media/
Allow /modules/
Allow /plugins/
Allow /templates/
Disallow /tmp/

rogerbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexcatalog

Rule Path
Disallow /

yandexfavicons

Rule Path
Disallow /

yandexdirect

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

jakarta commons

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

powermarks

Rule Path
Disallow /

offbyone

Rule Path
Disallow /

nutraspace

Rule Path
Disallow /

nutch

Rule Path
Disallow /

jobcrawlerbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

sightupbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

wise-guys

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

java

Rule Path
Disallow /

www-mechanize

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

spbot

Rule Path
Disallow /

docomo

Rule Path
Disallow /

pingdom.com_bot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

findlinks

Rule Path
Disallow /

butterfly

Rule Path
Disallow /

obot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

innosense

Rule Path
Disallow /

aghaven

Rule Path
Disallow /

jobrapido

Rule Path
Disallow /

mozilla/5.0 (compatible; jobrapido/1.1; +http://www.jobrapido.com)

Rule Path
Disallow /

mozilla/5.0 (jobrapido webpump)

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Warnings

  • 2 invalid lines.