dslpreise24.de
robots.txt
Robots Exclusion Standard data for dslpreise24.de
Resource Scan
Scan Details
| Site Domain | dslpreise24.de |
| Base Domain | dslpreise24.de |
| Scan Status | Ok |
| Last Scan | 2024-09-09T16:03:29+00:00 |
| Next Scan | 2024-10-09T16:03:29+00:00 |
Last Scan
| Scanned | 2024-09-09T16:03:29+00:00 |
| URL | http://dslpreise24.de/robots.txt |
| Domain IPs | 62.102.223.254 |
| Response IP | 62.102.223.254 |
| Found | Yes |
| Hash | d626b18c5026ae1d97e9cb0450c9d13f841a90bfb7ddec396870d4b599ac1ee5 |
| SimHash | 6b757f9dd290 |
Groups
googlebot
googlebot-image
mediapartners-google
feedfetcher-google
googlebot-mobile
adsbot-google
bingbot
msnbot
slurp
teoma
is_archiver
| Rule | Path |
|---|---|
| Disallow | /gsuche/ |
| Disallow | /goto/ |
| Disallow | /shop/ |
| Disallow | /go/ |
| Disallow | /g/ |
| Disallow | /l/ |
| Disallow | /gutschein-anzeigen/ |
| Disallow | /gutschein/anzeigen/ |
| Disallow | /gutschein-einloesen/ |
| Disallow | /abo-anzeigen/ |
| Disallow | /zum-shop/ |
| Disallow | /internal/ |
| Disallow | /anbieter/ |
| Disallow | /zum-gutschein/ |
| Disallow | /*.pdf$ |
| Disallow | /*.php$ |
| Disallow | /trackback/ |
| Disallow | */trackback/ |
| Disallow | /feed/ |
| Disallow | */feed/ |
| Disallow | /wp-admin/ |
| Disallow | /*?* |
| Disallow | /mirror/apache/*/ |
| Disallow | /mirror/archlinux/*/ |
| Disallow | /mirror/cygwin/*/ |
| Disallow | /mirror/gnu/*/ |
| Disallow | /mirror/raspbian/*/ |
| Disallow | /mirror/xubuntu/*/ |
sistrix
strucr
mj12bot
ahrefsbot
seokicks-robot
xovi
searchmetricsbot
backlinkcrawler
| Rule | Path |
|---|---|
| Disallow | /goto/ |
| Disallow | /shop/ |
| Disallow | /go/ |
| Disallow | /g/ |
| Disallow | /l/ |
| Disallow | /gutschein-anzeigen/ |
| Disallow | /gutschein/anzeigen/ |
| Disallow | /gutschein-einloesen/ |
| Disallow | /abo-anzeigen/ |
| Disallow | /zum-shop/ |
| Disallow | /anbieter/ |
| Disallow | /zum-gutschein/ |
| Disallow | /*.pdf$ |
| Disallow | /*.php$ |
| Disallow | /trackback/ |
| Disallow | */trackback/ |
| Disallow | /feed/ |
| Disallow | */feed/ |
| Disallow | /wp-admin/ |
| Disallow | /*?* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
Other Records
| Field | Value |
|---|---|
| sitemap | http://dslpreise24.de/sitemap.xml |
Comments