Log In Sign Up

e-sheet.co.jp
robots.txt

Robots Exclusion Standard data for e-sheet.co.jp

Resource Scan

Scan Details

Site Domain e-sheet.co.jp
Base Domain e-sheet.co.jp
Scan Status Ok
Last Scan2024-10-28T07:24:12+00:00
Next Scan 2024-11-27T07:24:12+00:00

Last Scan

Scanned2024-10-28T07:24:12+00:00
URL https://e-sheet.co.jp/robots.txt
Redirect https://www.e-sheet.co.jp/robots.txt
Redirect Domain www.e-sheet.co.jp
Redirect Base e-sheet.co.jp
Domain IPs 183.90.183.12
Redirect IPs 183.90.183.12
Response IP 183.90.183.12
Found Yes
Hash a81867270b0a6122672ba6f5a82e3404ae169e73f976f1a5a7dc597e7e4e38b1
SimHash 3d146743c4b5

Groups

megalodon
ia_archiver
rogerbot

Rule Path
Disallow *

baiduspider
baiduspider-ads
baiduspider-cpro
baiduspider-favo
baiduspider-image
baiduspider-news
baiduspider-video
yeti
daum
yandex
yandexbot
mail.ru_bot
mappy
seekport
mojeek
exabot
qwantify
netestate
seznambot
mappy
feedly
cliqzbot
pocketimagecache
startmebot
qqbrowser
proximic
blogmurabot
semrushbot
ahrefsbot
blexbot
mj12bot
smtbot
barkrowler
blexbot
sistrix crawler
woorankreview
proximic
ccbot
rogerbot
yasserg/crawler4j
go-resty/resty
python-requests
tomnomnom/meg
python-urllib/2.7
go-http-client/1.1
curl/7.58.0
vert.x-webclient/3.8.4

Rule Path
Disallow *

hatenabookmark

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

bingbot
msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

duckduckbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

twitterbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

facebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

pinterest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

crowsnest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

gunosy

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

sbooksnet
serpstatbot

Rule Path
Disallow /www.e-sheet.co.jp/
Disallow /e-sheet.co.jp/
Disallow /cgiFolder/
Disallow /cgi-bin/
Disallow /log/
Disallow /logs/
Disallow /.fast-cgi-bin/
Disallow /_cms/
Disallow /_backup/
Disallow /html/_json
Disallow /html/_mytag
Disallow /html/_preset
Disallow /html/_setting
Disallow /html/_template
Disallow /tmp/
Disallow /dat/
Disallow /test/
Disallow /system/
Disallow /rssFolder/
Disallow /inquiry_test/
Disallow /error/
Disallow /certify/
Disallow /.htaccess
Disallow /.htpasswd
Disallow /inquiry/lib/
Disallow /html/feed
Disallow /html/php
Disallow /html/svg
Disallow /html/_cms_preview.html
Disallow *.bak
Disallow /faq/
Disallow sitemap.html
Disallow /catalogue/company/company_information.html
Allow index.html

Other Records

Field Value
crawl-delay 600
Back to top

Other Records

Field Value
sitemap https://www.e-sheet.co.jp/sitemap.xml
Back to top

Comments

  • ======================================== init default == block ==
  • User-Agent:*
  • Disallow:*
  • ======================================== Customize setting == block == web site archiver ==
  • ======================================== Customize setting == block == web site crawler ==
  • User-agent: AhrefsBot
  • ======================================== Customize setting == GOOGLE ==
  • User-agent: Googlebot
  • Disallow: *
  • ==================================================
  • User-agent: Googlebot-Image
  • Disallow: *
  • Allow: /
  • Allow: /img
  • Allow: /image
  • ==================================================
  • User-agent: Googlebot
  • Allow: *
  • ==================================================
  • User-agent: APIs-Google
  • Disallow: *
  • ==================================================
  • User-agent: Mediapartners-Google
  • User-agent: AdsBot-Google
  • User-agent: AdsBot-Google-Mobile
  • User-agent: AdsBot-Google-Mobile-Apps
  • Disallow: *
  • ==================================================
  • User-agent: Googlebot-News
  • User-agent: Googlebot-Video
  • User-agent: Google Favicon
  • User-agent: FeedFetcher-Google
  • User-agent: Google-Read-Aloud
  • Disallow:
  • ==================================================
  • DuplexWeb-Google
  • Disallow:
  • Allow: /
  • Allow: /img
  • Allow: /image
  • ==================================================
  • == Customize setting == Crawl-delay ： 数値（分） ======================================================================
  • ======================================== Customize setting == HatenaBookmark ==
  • ======================================== Customize setting == Microsoft ==
  • ======================================== Customize setting == yahoo ==
  • ======================================== Customize setting == DuckDuckGo ==
  • ======================================== Customize setting == Twitter ==
  • ======================================== Customize setting == Facebook ==
  • ======================================== Customize setting == Pinterest ==
  • ======================================== Customize setting == apple ==
  • ======================================== Customize setting == smartnews ==
  • ======================================== Customize setting == Gunosy ==
  • ======================================== Customize setting == others ==
  • ====================================================================== Customize setting == Crawl-delay ： 数値（分） ==
  • ======================================== Customize setting == web site : folder / page ==
  • preset
  • cms
  • customized/workspace
  • customized
  • added : noindex parameter on page header
  • Disallow: /inquiry/check.html
  • Disallow: /inquiry/error.html
  • Disallow: /inquiry/completion.html
  • not in service
  • do not brocking html file for connect serchconsole # Disallow: googlebd80130d5677692a.html
  • do not brocking html file for connect serchconsole # Disallow: google3280615ef87a2116.html
  • do not brocking html file for connect serchconsole # Disallow: google6a10f8eee2eeb7a5.html
  • not existing Allow: index.php
  • ======================================== sitemap ==
Back to top

Warnings

  • 1 invalid line.
Back to top