Log In Sign Up

panelook.com
robots.txt

Robots Exclusion Standard data for panelook.com

Resource Scan

Scan Details

Site Domain panelook.com
Base Domain panelook.com
Scan Status Ok
Last Scan2024-06-21T11:16:10+00:00
Next Scan 2024-07-21T11:16:10+00:00

Last Scan

Scanned2024-06-21T11:16:10+00:00
URL https://panelook.com/robots.txt
Domain IPs 120.132.118.50
Response IP 120.132.118.50
Found Yes
Hash 7fceb9418bae9e651c074d3cdbd23492f24187c6ec25390b98f1751358fb4b08
SimHash 6f515d435e9a

Groups

baiduspider

Rule Path
Allow

baiduspider-image

Rule Path
Allow

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 100

bytespider

Rule Path
Allow

duckduckbot

Rule Path
Allow

googlebot

Rule Path
Allow

haosouspider

Rule Path
Allow

sogou web spider

Rule Path
Allow

sogou spider2

Rule Path
Allow

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 100

toutiaospider

Rule Path
Allow

yandexbot

Rule Path
Allow

robozilla

Rule Path
Allow

addsugarspiderbot

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

augustbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chinasospider

Rule Path
Disallow /

docomo

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

foxspider

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

lycos

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

psbot

Rule Path
Disallow /

safesearch microdata crawler

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sinaweibobot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

slackbot-linkexpanding

Rule Path
Disallow /

sogou blog

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

sogou news spider

Rule Path
Disallow /

sogou orion spider

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

spbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

umbot-ln

Rule Path
Disallow /

wangidspider

Rule Path
Disallow /

y!j-asr

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

yoozbot

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

yrspider

Rule Path
Disallow /

zumbot

Rule Path
Disallow /
Back to top

Comments

  • Updated on Dec. 13, 2023
Back to top

Warnings

  • 2 invalid lines.
Back to top