meteologix.com
robots.txt
Robots Exclusion Standard data for meteologix.com
Resource Scan
Scan Details
| Site Domain | meteologix.com |
| Base Domain | meteologix.com |
| Scan Status | Ok |
| Last Scan | 2024-05-22T14:53:56+00:00 |
| Next Scan | 2024-05-29T14:53:56+00:00 |
Last Scan
| Scanned | 2024-05-22T14:53:56+00:00 |
| URL | https://meteologix.com/robots.txt |
| Domain IPs | 23.44.5.40, 23.44.5.98, 2600:1417:3f::1732:e8eb, 2600:1417:3f::6858:469b |
| Response IP | 23.44.5.40 |
| Found | Yes |
| Hash | 714baa8b8d716da44793d0bc39e66e31a3f59a111df1a5bc6e568892e1b6fb1a |
| SimHash | 620090646f99 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /images/overlay |
| Disallow | /images/scale |
| Disallow | /ajax |
| Disallow | /ajax_pub |
| Disallow | /*/ajax |
| Disallow | /*/ajax_pub |
ahrefsbot
| Rule | Path |
|---|---|
| Disallow | /images/overlay |
| Disallow | /images/scale |
| Disallow | /ajax |
| Disallow | /ajax_pub |
| Disallow | /*/ajax |
| Disallow | /*/ajax_pub |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
grapeshot
| Rule | Path |
|---|---|
| Disallow | /images/overlay |
| Disallow | /images/scale |
| Disallow | /ajax |
| Disallow | /ajax_pub |
| Disallow | /*/ajax |
| Disallow | /*/ajax_pub |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
bingbot
| Rule | Path |
|---|---|
| Disallow | /images/overlay |
| Disallow | /images/scale |
| Disallow | /ajax |
| Disallow | /ajax_pub |
| Disallow | /*/ajax |
| Disallow | /*/ajax_pub |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |