erland.isaksson.info
robots.txt

Robots Exclusion Standard data for erland.isaksson.info

Resource Scan

Scan Details

Site Domain erland.isaksson.info
Base Domain isaksson.info
Scan Status Ok
Last Scan2024-10-24T03:11:38+00:00
Next Scan 2024-11-23T03:11:38+00:00

Last Scan

Scanned2024-10-24T03:11:38+00:00
URL http://erland.isaksson.info/robots.txt
Domain IPs 212.71.232.78
Response IP 212.71.232.78
Found Yes
Hash 6e3038ea56030371112cbb692fd81889b0a96eac81be7b4da4cfbdf99e44c08d
SimHash 2a74e3b1ed56

Groups

*

Rule Path
Disallow /download/do/downloadapplication*
Disallow /arkiv/vboxsbsimages/sbssvn-0.1.ova
Disallow /arkiv/vboxsbsimages/lmssvn-0.1.ova

amazon-kendra-web-crawler-*

Product Comment
amazon-kendra-web-crawler-* all customers of Amazon Kendra's web crawler
Rule Path Comment
Disallow / disallow everything

amazonbot

Rule Path
Disallow /