bento.de
robots.txt
Robots Exclusion Standard data for bento.de
Resource Scan
Scan Details
Site Domain | bento.de |
Base Domain | bento.de |
Scan Status | Ok |
Last Scan | 2024-05-07T22:40:52+00:00 |
Next Scan | 2024-05-14T22:40:52+00:00 |
Last Scan
Scanned | 2024-05-07T22:40:52+00:00 |
URL | https://bento.de/robots.txt |
Redirect | https://www.spiegel.de/robots.txt |
Redirect Domain | www.spiegel.de |
Redirect Base | spiegel.de |
Domain IPs | 213.95.204.189 |
Redirect IPs | 128.65.210.180, 128.65.210.181, 128.65.210.182, 128.65.210.183, 128.65.210.184, 128.65.210.185 |
Response IP | 128.65.210.185 |
Found | Yes |
Hash | 99f95f3a36b8f6e07a09efcb28161251e47f4e4272352661d9de1cd7168b5cba |
SimHash | b2200d1aeda9 |
Groups
*
Rule | Path |
---|---|
Allow | / |
Disallow | /*CR-Dokumentation.pdf$ |
Disallow | /gutscheine/suche? |
Disallow | /gutscheine/*?code=* |
Disallow | /gutscheine/*%26code%3D* |
Other Records
Field | Value |
---|---|
sitemap | https://www.spiegel.de/sitemaps/news-de.xml |
sitemap | https://www.spiegel.de/sitemaps/videos/sitemap.xml |
sitemap | https://www.spiegel.de/plus/sitemap.xml |
sitemap | https://www.spiegel.de/sitemap.xml |
sitemap | https://gutscheine.spiegel.de/sitemap.xml |
Comments