pferde.de
robots.txt
Robots Exclusion Standard data for pferde.de
Resource Scan
Scan Details
| Site Domain | pferde.de |
| Base Domain | pferde.de |
| Scan Status | Ok |
| Last Scan | 2024-06-21T14:04:25+00:00 |
| Next Scan | 2024-06-28T14:04:25+00:00 |
Last Scan
| Scanned | 2024-06-21T14:04:25+00:00 |
| URL | https://pferde.de/robots.txt |
| Redirect | https://www.pferde.de/robots.txt |
| Redirect Domain | www.pferde.de |
| Redirect Base | pferde.de |
| Domain IPs | 18.64.67.47, 18.64.67.59, 18.64.67.64, 18.64.67.75 |
| Redirect IPs | 18.64.67.47, 18.64.67.59, 18.64.67.64, 18.64.67.75 |
| Response IP | 3.160.246.53 |
| Found | Yes |
| Hash | ab134b59c645dd542015ae8b75e671d5db6b835052d7243676701ed4a41c36d8 |
| SimHash | 4f63d137d0e5 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /ads.txt |
| Disallow | / |
googlebot
googlebot-image
googlebot-mobile
googlebot-news
googlebot-video
mediapartners-google
mediapartners (googlebot)
adsbot-google
feedfetcher-google
ia_archiver
bingbot
heritrix
msnbot
slurp
gumgum
teoma
ebay relevance ad crawler
ebay relevance ad crawler powered by contentdetection (www.mindup.de)
browsershots
twitterbot
termlabs
facebookexternalhit
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
facebookexternalhit/1.1
facebookcatalog/1.0
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /ajax |
| Disallow | /category_error.php |
| Disallow | /crossdomain |
| Disallow | /error.php |
| Disallow | /fb_login.php |
| Disallow | /feed/ |
| Disallow | /flvplayer |
| Disallow | /googlemaps |
| Disallow | /infopoint |
| Disallow | /merkzettel.php |
| Disallow | /session_transfer.php |
| Disallow | /unsub |
| Disallow | /verwaltung |
Warnings
- `mozilla/5.0 (compatible; facebookbot/1.0; +https` is not a known field.
Comments