busch-jaeger.nl
robots.txt
Robots Exclusion Standard data for busch-jaeger.nl
Resource Scan
Scan Details
| Site Domain | busch-jaeger.nl |
| Base Domain | busch-jaeger.nl |
| Scan Status | Failed |
| Failure Reason | Scan timed out. |
| Last Scan | 2024-06-06T23:23:38+00:00 |
| Next Scan | 2024-09-04T23:23:38+00:00 |
Last Successful Scan
| Scanned | 2024-01-16T23:02:48+00:00 |
| URL | https://busch-jaeger.nl/robots.txt |
| Redirect | https://www.busch-jaeger.nl/robots.txt |
| Redirect Domain | www.busch-jaeger.nl |
| Redirect Base | busch-jaeger.nl |
| Domain IPs | 18.156.180.252 |
| Redirect IPs | 96.17.96.31, 96.17.96.7 |
| Response IP | 23.59.168.137 |
| Found | Yes |
| Hash | ec07334b7f4de2519bc99f3ace5d0d5b14a88ec92c0cc7b0050ba9368c17cbf2 |
| SimHash | 0e014a044b82 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*?id=* |
| Disallow | /*%26id%3D* |
| Disallow | /*?L=0* |
| Disallow | /*%26L%3D0* |
| Disallow | /*?tx_solr* |
| Disallow | /*%26tx_solr* |
| Disallow | /*/Private/* |
| Disallow | /*/Configuration/* |
| Disallow | /typo3temp/* |
| Disallow | /media/bje/* |
| Allow | /typo3temp/*.css |
| Allow | /typo3temp/*.css.*.gzip |
| Allow | /typo3temp/*.js |
| Allow | /typo3temp/*.js.*.gzip |
| Allow | /typo3temp/*.jpg |
| Allow | /typo3temp/*.gif |
| Allow | /typo3temp/*.png |
| Disallow | *.sql |
| Disallow | *.sql.gz |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |