Log In Sign Up

hdiy.net
robots.txt

Robots Exclusion Standard data for hdiy.net

Resource Scan

Scan Details

Site Domain hdiy.net
Base Domain hdiy.net
Scan Status Ok
Last Scan2024-11-14T15:52:41+00:00
Next Scan 2024-11-21T15:52:41+00:00

Last Scan

Scanned2024-11-14T15:52:41+00:00
URL http://hdiy.net/robots.txt
Domain IPs 61.61.96.158
Response IP 61.61.96.158
Found Yes
Hash b9ea3435ea3083904787c03d5b8b7a7c168c76ccdb4bda1afc131bfdbacf9fcc
SimHash f094676206e3

Groups

*

Rule Path
Disallow /*.jpg$
Disallow /*.gif$
Disallow /ctrl_center
Disallow /gb2big5
Disallow /hdiyeditor
Disallow /hdiyeditor1

baiduspider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou pic spider

Rule Path
Disallow /

sogou head spider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou orion spider

Rule Path
Disallow /

sogou-test-spider

Rule Path
Disallow /

sogou pic agent

Rule Path
Disallow /

sitelockspider

Rule Path
Disallow /

sitelockspider [en] (winnt; i ;nav)

Rule Path
Disallow /

sitelock

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

lmspider

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

wesee

Rule Path
Disallow /

wasalive-bot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

genieo

Rule Path
Disallow /

kraken

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

claritybot

Rule Path
Disallow /

musobot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

slackbot-linkexpanding

Rule Path
Disallow /

calculon spider

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

zitebot

Rule Path
Disallow /

tinfoil

Rule Path
Disallow /

tinfoilsecurity

Rule Path
Disallow /

spider-pig

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

openlinkprofiler

Rule Path
Disallow /

spbot

Rule Path
Disallow /

vebidoobot

Rule Path
Disallow /

universalfeedparser

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

jorgee

Rule Path
Disallow /

psbot

Rule Path
Disallow /

psbot-page

Rule Path
Disallow /

java

Rule Path
Disallow /

rukicrawler

Rule Path
Disallow /

pycurl

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

grouphigh

Rule Path
Disallow /

idg/uk

Rule Path
Disallow /

extlinksbot

Rule Path
Disallow /

daum

Rule Path
Disallow /

webcrawler

Rule Path
Disallow /

jamesbot

Rule Path
Disallow /

phpcrawl

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

dataprovider

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

apple-pubsub

Rule Path
Disallow /

ips-agent

Rule Path
Disallow /

ucrawler

Rule Path
Disallow /

zovika

Rule Path
Disallow /

bubing

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

panscient.companscient

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

lua-resty

Rule Path
Disallow /

ccbot/2.0

Rule Path
Disallow /

seokicks

Rule Path
Disallow /
Back to top

Comments

  • Bad Bots
Back to top

Warnings

  • 2 invalid lines.
Back to top