niigataakamichi-housedo.com
robots.txt
Robots Exclusion Standard data for niigataakamichi-housedo.com
Resource Scan
Scan Details
| Site Domain | niigataakamichi-housedo.com |
| Base Domain | niigataakamichi-housedo.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2024-05-30T22:12:20+00:00 |
| Next Scan | 2024-08-28T22:12:20+00:00 |
Last Successful Scan
| Scanned | 2024-01-09T21:38:50+00:00 |
| URL | https://niigataakamichi-housedo.com/robots.txt |
| Domain IPs | 54.192.150.22, 54.192.150.36, 54.192.150.46, 54.192.150.64 |
| Response IP | 108.157.194.109 |
| Found | Yes |
| Hash | 19eaf320f75de0464b439a8b2ee87dc04b28f97fea5043101249d13f82697617 |
| SimHash | 1a7f7442dfbc |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /php-comps/ |
| Disallow | /php-partials/ |
| Disallow | /node_modules/ |
| Disallow | /test/ |
bingbot
| Rule | Path |
|---|---|
| Disallow | /php-comps/ |
| Disallow | /php-partials/ |
| Disallow | /node_modules/ |
| Disallow | /test/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 150 |
googlebot
| Rule | Path |
|---|---|
| Disallow | /php-comps/ |
| Disallow | /php-partials/ |
| Disallow | /node_modules/ |
| Disallow | /test/ |
slurp
| Rule | Path |
|---|---|
| Disallow | /php-comps/ |
| Disallow | /php-partials/ |
| Disallow | /node_modules/ |
| Disallow | /test/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
msnbot
| Rule | Path |
|---|---|
| Disallow | /php-comps/ |
| Disallow | /php-partials/ |
| Disallow | /node_modules/ |
| Disallow | /test/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 150 |
msnbot-media
| Rule | Path |
|---|---|
| Disallow | /php-comps/ |
| Disallow | /php-partials/ |
| Disallow | /node_modules/ |
| Disallow | /test/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 150 |
teoma
| Rule | Path |
|---|---|
| Disallow | /php-comps/ |
| Disallow | /php-partials/ |
| Disallow | /node_modules/ |
| Disallow | /test/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
yandex
| Rule | Path |
|---|---|
| Disallow | /php-comps/ |
| Disallow | /php-partials/ |
| Disallow | /node_modules/ |
| Disallow | /test/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
Warnings
- 4 invalid lines.