adserver8.com
robots.txt
Robots Exclusion Standard data for adserver8.com
Resource Scan
Scan Details
| Site Domain | adserver8.com |
| Base Domain | adserver8.com |
| Scan Status | Ok |
| Last Scan | 2024-05-07T23:03:46+00:00 |
| Next Scan | 2024-06-06T23:03:46+00:00 |
Last Scan
| Scanned | 2024-05-07T23:03:46+00:00 |
| URL | https://adserver8.com/robots.txt |
| Domain IPs | 216.38.59.116 |
| Response IP | 216.38.59.116 |
| Found | Yes |
| Hash | a7f740cde2a71370a4b91a4d3693355e18d1fd1f75c74bcc0f8eed529833bfff |
| SimHash | bc2372146f40 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /images/ |
| Disallow | /js/ |
| Disallow | /ssl/ |
| Disallow | /Click/ |
| Disallow | /css/ |
| Disallow | /Emails/ |
| Disallow | /Nav/ |
| Disallow | /Hotels/ |
| Disallow | /includes/ |
| Disallow | /Stats/ |
| Disallow | /Tracka/ |
| Disallow | /Ads/ |
| Disallow | /swf/ |
| Disallow | /Data/ |
| Disallow | /Calendar/ |
| Disallow | /Forum/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
googlebot allow: /
googlebot-mobile allow: /
msnbot allow: /
baidu disallow: /
baiduspider disallow: /
baiduspider+ disallow: /
baiduspider+(+http://www.baidu.com/search/spider.htm) disallow: /
baiduspider-image disallow: /
baiduspider-video disallow: /
baiduspider/2.0 disallow: /
baiduspider/2.0;+http://www.baidu.com/search/spider.html disallow: /
easouspider disallow: /
ichiro disallow: /
jikespider disallow: /
majestic-12 disallow: /
moget disallow: /
mozilla/5.0(compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html) disallow: /
naverbot disallow: /
sogou head spider/3.0 disallow: /
sogou orion spider/3.0 disallow: /
sogou pic agent disallow: /
sogou pic spider/3.0 disallow: /
sogou spider disallow: /
sogou web spider disallow: /
sogou web spider/4.0 disallow: /
sogou-test-spider/4.0 disallow: /
sosospider disallow: /
sosospider+ disallow: /
sosospider/2.0 disallow: /
yandex disallow: /
yeti disallow: /
yisouspider disallow: /
youdaobot disallow: /
youdaobot/1.0 disallow: /
No rules defined. All paths allowed.
Warnings
- 4 invalid lines.