rugby-history.com
robots.txt

Robots Exclusion Standard data for rugby-history.com

Resource Scan

Scan Details

Site Domain rugby-history.com
Base Domain rugby-history.com
Scan Status Ok
Last Scan2024-11-08T22:54:10+00:00
Next Scan 2024-11-15T22:54:10+00:00

Last Scan

Scanned2024-11-08T22:54:10+00:00
URL https://rugby-history.com/robots.txt
Domain IPs 3.114.71.155, 52.198.5.0
Response IP 52.198.5.0
Found Yes
Hash 45934815418c3a9c4b7717aa488fa57db46c9f09803e9a9c58e7e1fc0bab0466
SimHash 012bad944306

Groups

*

Rule Path
Disallow /koko/team/
Disallow /syakai/team/
Disallow /tyugaku/team/
Disallow /blog/
Disallow /draft/koko/
Disallow /draft/tyugaku/
Disallow /blog/player/
Disallow /blog/univ/
Disallow /blog/syakai/
Disallow /crowd/
Disallow /calendar/map.php
Disallow /add/
Disallow /home.php
Disallow /post/entry.php
Disallow /player/favorite/
Disallow /player/update/
Disallow /player/update_kyureki/
Disallow /score/favorite/
Disallow /score/update/
Disallow /profile/
Disallow /privacy/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

Comments

  • 全てのクローラーを対象