smrj.go.jp
robots.txt
Robots Exclusion Standard data for smrj.go.jp
Resource Scan
Scan Details
| Site Domain | smrj.go.jp |
| Base Domain | smrj.go.jp |
| Scan Status | Ok |
| Last Scan | 2024-10-20T13:22:21+00:00 |
| Next Scan | 2024-11-19T13:22:21+00:00 |
Last Scan
| Scanned | 2024-10-20T13:22:21+00:00 |
| URL | https://www.smrj.go.jp/robots.txt |
| Domain IPs | 202.238.147.90 |
| Response IP | 202.238.147.90 |
| Found | Yes |
| Hash | 7f2211f25f03d38c463c31123f6cf97dc6fff801f40db7dd8d0bda95e4f45b06 |
| SimHash | e25ddcc441b3 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /fsw/ |
| Disallow | /fsw-sy-navi/ |
| Disallow | /fsw-kyosai/ |
| Disallow | /*/exclusive/ |
| Disallow | /*/exclusive.html |
| Disallow | /*/*_ex/ |
| Disallow | /jinzai001/ |
gsa-crawler
| Rule | Path |
|---|---|
| Disallow | /fsw/ |
| Disallow | /fsw-sy-navi/ |
| Disallow | /fsw-kyosai/ |
| Disallow | /*/exclusive/ |
| Disallow | /*/exclusive.html |
| Disallow | /*/*_ex/ |
| Disallow | /jinzai001/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
msnbot
| Rule | Path |
|---|---|
| Disallow | /fsw/ |
| Disallow | /fsw-sy-navi/ |
| Disallow | /fsw-kyosai/ |
| Disallow | /*/exclusive/ |
| Disallow | /*/exclusive.html |
| Disallow | /*/*_ex/ |
| Disallow | /jinzai001/ |
| Disallow | /skyosai1/cgi-bin/ |
| Disallow | /sy-navi1/cgi-bin/ |
| Disallow | /pickup |
| Disallow | /all |
| Disallow | /movie |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.smrj.go.jp/sitemap.xml |
Warnings
- 2 invalid lines.