kentei.cc
robots.txt
Robots Exclusion Standard data for kentei.cc
Resource Scan
Scan Details
| Site Domain | kentei.cc |
| Base Domain | kentei.cc |
| Scan Status | Ok |
| Last Scan | 2024-09-26T17:21:41+00:00 |
| Next Scan | 2024-10-03T17:21:41+00:00 |
Last Scan
| Scanned | 2024-09-26T17:21:41+00:00 |
| URL | https://www.kentei.cc/robots.txt |
| Domain IPs | 108.157.254.108, 108.157.254.34, 108.157.254.57, 108.157.254.73 |
| Response IP | 108.157.254.34 |
| Found | Yes |
| Hash | 2ff9ddf252a9d82ae6fa3d481f65babee845c697b9bbd8e2925f8a6c6147b257 |
| SimHash | 335c53706bb8 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /kentei/create |
| Disallow | /json |
semrushbot
proximic
grapeshotcrawler
blexbot
linguee bot
qwantify
yandexbot
ltx71
icc-crawler
ccbot
baidumobaider
baiduspider
petalbot
ahrefsbot
criteobot
cincraw
domainsproject
dotbot
admantx.com
paloaltonetworks.com
dataforseobot
microadbot
python-requests
scalaj-http
uipbot
hypefactors.com
barkrowler
megaindex.ru
ttd-content
java
scrapy
peer39_crawler
blexbot
grapeshotcrawler
dataforseobot
seznambot
bidswitchbot
sogou web spider
vivaldi
curl
yandex browser
cfnetwork
help@dataminr.com
mj12bot
leikibot
gumgum-bot
quantcastbot
pear http_request
cortex
crowsnest
react.org
mail.ru_bot
amazonbot
seekportbot
integralads
admantx
ias-or
ias-va
ias-
amazonadbot
gptbot
pixalate.com
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://kentei.cc/smart/sitemap/all.php |