freebsd.org
robots.txt
Robots Exclusion Standard data for freebsd.org
Resource Scan
Scan Details
| Site Domain | freebsd.org |
| Base Domain | freebsd.org |
| Scan Status | Ok |
| Last Scan | 2024-06-12T08:59:49+00:00 |
| Next Scan | 2024-07-12T08:59:49+00:00 |
Last Scan
| Scanned | 2024-06-12T08:59:49+00:00 |
| URL | https://freebsd.org/robots.txt |
| Redirect | https://www.freebsd.org/robots.txt |
| Redirect Domain | www.freebsd.org |
| Redirect Base | freebsd.org |
| Domain IPs | 2610:1c1:1:606c::50:15, 96.47.72.84 |
| Redirect IPs | 203.80.16.154, 2404:a8:3ff::50:3 |
| Response IP | 203.80.16.154 |
| Found | Yes |
| Hash | 6d84161e16dfba34aebee11474ef59b2a5a7574e6ab6769001e00843d45f81eb |
| SimHash | 103ddc72e893 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi/man.cgi?apropos=2& |
| Disallow | /cgi/ports.cgi? |
| Disallow | /cgi/man-autocomplete.cgi |
| Disallow | /cgi/ports-autocomplete.cgi |
| Disallow | /statistic |
Comments