Log In Sign Up

dinghanlvjian.com
robots.txt

Robots Exclusion Standard data for dinghanlvjian.com

Resource Scan

Scan Details

Site Domain dinghanlvjian.com
Base Domain dinghanlvjian.com
Scan Status Ok
Last Scan2024-09-19T01:19:57+00:00
Next Scan 2024-10-19T01:19:57+00:00

Last Scan

Scanned2024-09-19T01:19:57+00:00
URL https://dinghanlvjian.com/robots.txt
Domain IPs 23.235.180.46
Response IP 23.235.180.46
Found Yes
Hash 1aeae96e368545a300633129b7ecca049d6d8caaa39883e72b64677c924bed6e
SimHash 5132d5c8f4b2

Groups

baiduspider

Rule Path
Allow /

sogou spider

Rule Path
Allow /

sosospider

Rule Path
Allow /

googlebot

Rule Path
Allow /

mediapartners-google

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

zmeu phpmyadmin

Rule Path
Disallow /

winhttp

Rule Path
Disallow /

httpclient

Rule Path
Disallow /

microsoft url control

Rule Path
Disallow /

yyspider

Rule Path
Disallow /

jaunty

Rule Path
Disallow /

obot

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

indy library

Rule Path
Disallow /

flightdeckreports bot

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ezooms.bot

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

crawldaddy

Rule Path
Disallow /

asktbfxtv

Rule Path
Disallow /

alexa toolbar

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

coolpadwebkit

Rule Path
Disallow /

java

Rule Path
Disallow /

feedly

Rule Path
Disallow /

universalfeedparser

Rule Path
Disallow /

apachebench

Rule Path
Disallow /

microsoft url control

Rule Path
Disallow /

zmeu

Rule Path
Disallow /

lightdeckreports bot

Rule Path
Disallow /

digext

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

headlesschrome

Rule Path
Disallow /

feeddemon

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

opensiteexplorer

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

hubspot

Rule Path
Disallow /

opensiteexplorer

Rule Path
Disallow /

python

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /
Back to top

Warnings

  • 2 invalid lines.
Back to top