levtech.jp
robots.txt

Robots Exclusion Standard data for levtech.jp

Resource Scan

Scan Details

Site Domain levtech.jp
Base Domain levtech.jp
Scan Status Ok
Last Scan2024-09-25T22:24:35+00:00
Next Scan 2024-10-25T22:24:35+00:00

Last Scan

Scanned2024-09-25T22:24:35+00:00
URL https://levtech.jp/robots.txt
Domain IPs 13.33.30.119, 13.33.30.124, 13.33.30.21, 13.33.30.82
Response IP 13.33.30.119
Found Yes
Hash 5bdd700cce76c5266910b695bc2500e99c5c2bf2b6be67e46262fd5063253a58
SimHash 006e7e00c28a

Groups

*

Rule Path
Disallow /admin/
Disallow /api/
Disallow /contact/business/
Disallow /contact/private/
Disallow /entry/
Disallow /member/
Disallow /lp/
Disallow /skill-regist/

ia_archiver

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

ichiro

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baidumobaider

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2