annonces-caravaning.com
robots.txt
Robots Exclusion Standard data for annonces-caravaning.com
Resource Scan
Scan Details
| Site Domain | annonces-caravaning.com |
| Base Domain | annonces-caravaning.com |
| Scan Status | Ok |
| Last Scan | 2024-09-27T02:05:11+00:00 |
| Next Scan | 2024-10-04T02:05:11+00:00 |
Last Scan
| Scanned | 2024-09-27T02:05:11+00:00 |
| URL | https://www.annonces-caravaning.com/robots.txt |
| Domain IPs | 178.33.93.92 |
| Response IP | 178.33.93.92 |
| Found | Yes |
| Hash | 925967d6196fe7778c58553d12241845c36645fadea6d5899a3578baacdf3869 |
| SimHash | fa637682d233 |
Groups
turnitinbot
catchbot
iccrawler - icjobs
r6_feedfetcher
r6_commentreader
youdaobot
sogou
sosospider
naverbot
yeti
ichiro
spinn3r
spbot
trendictionbot
yandex
blexbot
nerdybot
proximic
mauibot
mauibot (crawler.feedback+wc@gmail.com)
megaindex.ru/2.0
megaindex.ru
megaindex.com
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /*.inc$ |
| Disallow | /*.gz$ |
| Disallow | /*.swf$ |
| Disallow | /*.wmv$ |
| Disallow | /*.cgi$ |
| Disallow | /*.xhtml$ |
| Disallow | /*.pdf$ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.annonces-caravaning.com/sitemap.xml |
Warnings
- 1 invalid line.
Comments