raillab.jp
robots.txt

Robots Exclusion Standard data for raillab.jp

Resource Scan

Scan Details

Site Domain raillab.jp
Base Domain raillab.jp
Scan Status Ok
Last Scan2024-06-26T03:03:28+00:00
Next Scan 2024-07-03T03:03:28+00:00

Last Scan

Scanned2024-06-26T03:03:28+00:00
URL https://raillab.jp/robots.txt
Domain IPs 18.155.68.104, 18.155.68.22, 18.155.68.55, 18.155.68.78
Response IP 18.155.68.78
Found Yes
Hash 2610e07b91e65f2012b8afed64c2cf6ad4a7c520bbff25f339b3091091667588
SimHash 42399e3af591

Groups

*

Rule Path
Disallow /auth/
Disallow /api/
Disallow /utility/
Disallow /news/list?f_cat=
Disallow /car/*/photos/list?photo_type=
Disallow /carset/*/photos/list?photo_type=
Disallow /company/*/photos/list?photo_type=
Disallow /member/*/photo?photo_type=
Disallow /transport/*/photos/list?photo_type=
Disallow /station/*/photos/list?photo_type=
Disallow /terminal/*/photos/list?photo_type=
Disallow /train/*/photos/list?photo_type=
Disallow /car/*/records/list?sort=
Disallow /carset/*/records/list?sort=
Disallow /member/*/record/list?sort=
Disallow /transport/*/records/list?sort=
Disallow /station/*/records/list?sort=
Disallow /terminal/*/records/list?sort=
Disallow /train/*/records/list?sort=

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

grapeshot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

googlebot-news

Rule Path
Disallow /
Allow /news*