kali.org
robots.txt

Robots Exclusion Standard data for kali.org

Resource Scan

Scan Details

Site Domain kali.org
Base Domain kali.org
Scan Status Ok
Last Scan2024-09-09T10:19:40+00:00
Next Scan 2024-09-23T10:19:40+00:00

Last Scan

Scanned2024-09-09T10:19:40+00:00
URL https://kali.org/robots.txt
Redirect https://www.kali.org/robots.txt
Redirect Domain www.kali.org
Redirect Base kali.org
Domain IPs 104.18.4.159, 104.18.5.159, 2606:4700::6812:49f, 2606:4700::6812:59f
Redirect IPs 104.18.4.159, 104.18.5.159, 2606:4700::6812:49f, 2606:4700::6812:59f
Response IP 104.18.4.159
Found Yes
Hash 30c23c911a541e23bd9b19b2d51b5d5b9f0295c353d0406a07cb705bbde4c84b
SimHash 34b8d64845b7

Groups

*

Rule Path
Disallow
Disallow /docs/virtualization/import-premade-virtualbox-legacy/
Disallow /docs/virtualization/install-vmware-guest-tools-legacy/
Disallow /docs/virtualization/install-virtualbox-guest-additions-legacy/
Disallow /docs/installation/mini-iso/
Disallow /docs/arm/efikamx/
Disallow /docs/arm/imx-6ull-evk/
Disallow /docs/arm/raspberry-pi-with-luks-full-disk-encryption/
Disallow /docs/arm/ss808-mk808/
Disallow /tools/gdb/
Disallow /tools/rtlsdr-scanner/
Disallow /tools/android-sdk/
Disallow /tools/plecost/
Disallow /tools/p7zip/
Disallow /tools/pev/
Disallow /tools/abootimg/
Disallow /tools/bluez-firmware/
Disallow /tools/fake-hwclock/
Disallow /tools/faraday-client/
Disallow /tools/firmware-sof/
Disallow /tools/initramfs-tools/
Disallow /tools/ipv6-toolkit/
Disallow /tools/king-phisher/
Disallow /tools/netkit-ftp/
Disallow /tools/netkit-telnet/
Disallow /tools/netkit-tftp/
Disallow /tools/radare2-cutter/
Disallow /tools/smali/
Disallow /tools/starkiller/
Disallow /tools/u-boot/
Disallow /tools/usbutils/
Disallow /tools/xprobe/
Disallow /tools/zd1211-firmware/
Disallow /docs/wsl/mesa-21/
Disallow /docs/arm/gemini-pda/
Disallow /docs/introduction/kali-linux-default-passwords/
Disallow /docs/policy/kali-linux-root-user-policy/
Disallow /docs/development/public-packaging/
Disallow /docs/arm/galaxy-note-10.1/

Other Records

Field Value
sitemap https://www.kali.org/sitemap.xml

Comments

  • All robots
  • Allow everything...
  • ...expect for archived pages (kali-docs/kali-tools)
  • Point out where sitemap is