Log In Sign Up

gateau.jp
robots.txt

Robots Exclusion Standard data for gateau.jp

Resource Scan

Scan Details

Site Domain gateau.jp
Base Domain gateau.jp
Scan Status Ok
Last Scan2024-11-04T02:22:52+00:00
Next Scan 2024-12-04T02:22:52+00:00

Last Scan

Scanned2024-11-04T02:22:52+00:00
URL http://gateau.jp/robots.txt
Domain IPs 210.189.77.127
Response IP 210.189.77.127
Found Yes
Hash b7525e4d00d1787262876491a5136a6784e2aadb7b5c0c3b69fccc58d42dd110
SimHash 1a9774c224b8

Groups

bingbot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

nabot

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti/1.0

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

zoomspider

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

dalvik

Rule Path
Disallow /

exabot

Rule Path
Disallow /

tsolcrawler

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

teoma

Rule Path
Disallow /

e-societyrobot

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

becomejpbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

exb language crawler

Rule Path
Disallow /

kyoto-crawler

Rule Path
Disallow /

r6_feedfetcher

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

digg

Rule Path
Disallow /

abonti

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

blogramcrawler

Rule Path
Disallow /

feedburner

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

obot

Rule Path
Disallow /

memonewsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

showyoubot

Rule Path
Disallow /

butterfly

Rule Path
Disallow /

socialradarbot

Rule Path
Disallow /

feedbot

Rule Path
Disallow /

tweetedtimes bot

Rule Path
Disallow /

pockey-gethtml

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /
Back to top

Warnings

  • 6 invalid lines.
Back to top