bsi.bund.de
robots.txt

Robots Exclusion Standard data for bsi.bund.de

Resource Scan

Scan Details

Site Domain bsi.bund.de
Base Domain bund.de
Scan Status Ok
Last Scan2025-09-20T07:51:36+00:00
Next Scan 2025-10-20T07:51:36+00:00

Last Scan

Scanned2025-09-20T07:51:36+00:00
URL http://bsi.bund.de/robots.txt
Redirect https://www.bsi.bund.de/robots.txt
Redirect Domain www.bsi.bund.de
Redirect Base bund.de
Domain IPs 80.245.144.218
Redirect IPs 80.245.144.218
Response IP 80.245.144.218
Found Yes
Hash b1dd4c510e379a38198aaec4ed6f2cb92ede6c839b9cba749ce3fd9fe181ecf5
SimHash 652905449753

Groups

d115crawler

Rule Path
Allow /DE/Service/D115/

*

Rule Path
Disallow /SiteGlobals/
Disallow /DE/Service/Warenkorb
Disallow /EN/Service/
Disallow /SharedDocs/Downloads/DE/BSI/Zertifizierung/
Allow /SiteGlobals/Functions/JavaScript/
Allow /SiteGlobals/StyleBundles/
Allow /SiteGlobals/Frontend/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.bsi.bund.de/_config/xml-sitemap/Sitemap_Index.xml