halleyweb.com
robots.txt
Robots Exclusion Standard data for halleyweb.com
Resource Scan
Scan Details
| Site Domain | halleyweb.com |
| Base Domain | halleyweb.com |
| Scan Status | Ok |
| Last Scan | 2024-06-02T02:23:41+00:00 |
| Next Scan | 2024-07-02T02:23:41+00:00 |
Last Scan
| Scanned | 2024-06-02T02:23:41+00:00 |
| URL | https://halleyweb.com/robots.txt |
| Domain IPs | 195.78.210.236 |
| Response IP | 195.78.210.236 |
| Found | Yes |
| Hash | d1129be5d1fa013fe5e326cf107dc10532c5d08a60a5144e8ca1ccbb3556923d |
| SimHash | 2801d5c156f7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | / |
googlebot
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
googlebot-image
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
mediapartners-google
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
bingbot
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
msnbot
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
msnbot-media
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
yahoo-blogs
| Rule | Path |
|---|---|
| Disallow | /de/ |
| Disallow | /mc/ |
| Disallow | /images/ |
| Disallow | /*/de/* |
| Disallow | /*/mc/* |
| Disallow | /*/images/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 600 |
Comments