penetration-testing.com
robots.txt

Robots Exclusion Standard data for penetration-testing.com

Resource Scan

Scan Details

Site Domain penetration-testing.com
Base Domain penetration-testing.com
Scan Status Ok
Last Scan2025-09-07T18:22:28+00:00
Next Scan 2025-10-07T18:22:28+00:00

Last Scan

Scanned2025-09-07T18:22:28+00:00
URL https://penetration-testing.com/robots.txt
Redirect https://www.penetration-testing.com/robots.txt
Redirect Domain www.penetration-testing.com
Redirect Base penetration-testing.com
Domain IPs 13.35.202.128, 13.35.202.27, 13.35.202.63, 13.35.202.76, 2600:9000:2078:2e00:b:a9aa:fac0:93a1, 2600:9000:2078:4000:b:a9aa:fac0:93a1, 2600:9000:2078:6600:b:a9aa:fac0:93a1, 2600:9000:2078:7200:b:a9aa:fac0:93a1, 2600:9000:2078:800:b:a9aa:fac0:93a1, 2600:9000:2078:9600:b:a9aa:fac0:93a1, 2600:9000:2078:9c00:b:a9aa:fac0:93a1, 2600:9000:2078:da00:b:a9aa:fac0:93a1
Redirect IPs 104.18.40.47, 172.64.147.209, 2606:4700:4402::ac40:93d1, 2606:4700:4407::6812:282f
Response IP 104.18.40.47
Found Yes
Hash 0fd77a8a3b6b70f72990abb4f3594053e5a2d57eefe7277eb1a18ff34ba22850
SimHash c901d940c553

Groups

*

Rule Path
Disallow /*?
Allow /~gitbook/image?*
Allow /~gitbook/icon?*
Allow /

Other Records

Field Value
sitemap https://www.penetration-testing.com/sitemap.xml