weather.us
robots.txt
Robots Exclusion Standard data for weather.us
Resource Scan
Scan Details
| Site Domain | weather.us |
| Base Domain | weather.us |
| Scan Status | Ok |
| Last Scan | 2024-09-19T17:22:35+00:00 |
| Next Scan | 2024-09-26T17:22:35+00:00 |
Last Scan
| Scanned | 2024-09-19T17:22:35+00:00 |
| URL | https://weather.us/robots.txt |
| Domain IPs | 2600:1413:b000:6::17d5:2bd3, 2600:1413:b000:6::17d5:2bdf, 96.17.96.24, 96.17.96.30 |
| Response IP | 23.44.4.145 |
| Found | Yes |
| Hash | 714baa8b8d716da44793d0bc39e66e31a3f59a111df1a5bc6e568892e1b6fb1a |
| SimHash | 620090646f99 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /images/overlay |
| Disallow | /images/scale |
| Disallow | /ajax |
| Disallow | /ajax_pub |
| Disallow | /*/ajax |
| Disallow | /*/ajax_pub |
ahrefsbot
| Rule | Path |
|---|---|
| Disallow | /images/overlay |
| Disallow | /images/scale |
| Disallow | /ajax |
| Disallow | /ajax_pub |
| Disallow | /*/ajax |
| Disallow | /*/ajax_pub |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
grapeshot
| Rule | Path |
|---|---|
| Disallow | /images/overlay |
| Disallow | /images/scale |
| Disallow | /ajax |
| Disallow | /ajax_pub |
| Disallow | /*/ajax |
| Disallow | /*/ajax_pub |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
bingbot
| Rule | Path |
|---|---|
| Disallow | /images/overlay |
| Disallow | /images/scale |
| Disallow | /ajax |
| Disallow | /ajax_pub |
| Disallow | /*/ajax |
| Disallow | /*/ajax_pub |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |