provincia.bergamo.it
robots.txt
Robots Exclusion Standard data for provincia.bergamo.it
Resource Scan
Scan Details
| Site Domain | provincia.bergamo.it |
| Base Domain | provincia.bergamo.it |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2024-08-08T05:35:29+00:00 |
| Next Scan | 2024-11-06T05:35:29+00:00 |
Last Successful Scan
| Scanned | 2023-12-20T05:30:56+00:00 |
| URL | https://www.provincia.bergamo.it/robots.txt |
| Domain IPs | 195.78.210.236 |
| Response IP | 195.78.210.236 |
| Found | Yes |
| Hash | e49ce53bfca4b6371a13b0928de8a5098a813d7b926c4f638d57449ba24bef8a |
| SimHash | 2800cbc356f7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | / |
googlebot
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
googlebot-image
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
mediapartners-google
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
bingbot
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
msnbot
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
msnbot-media
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
Comments