insectimages.org
robots.txt
Robots Exclusion Standard data for insectimages.org
Resource Scan
Scan Details
| Site Domain | insectimages.org |
| Base Domain | insectimages.org |
| Scan Status | Ok |
| Last Scan | 2024-05-28T14:24:30+00:00 |
| Next Scan | 2024-06-04T14:24:30+00:00 |
Last Scan
| Scanned | 2024-05-28T14:24:30+00:00 |
| URL | https://insectimages.org/robots.txt |
| Redirect | https://www.insectimages.org/robots.txt |
| Redirect Domain | www.insectimages.org |
| Redirect Base | insectimages.org |
| Domain IPs | 52.84.162.101, 52.84.162.109, 52.84.162.29, 52.84.162.66 |
| Redirect IPs | 108.139.10.49, 108.139.10.51, 108.139.10.55, 108.139.10.82 |
| Response IP | 108.157.52.83 |
| Found | Yes |
| Hash | 14f3322b824fa1315e08254cc45c251be3ddd225595f564b63ee56376b93dae7 |
| SimHash | 0a585af0c013 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /spdn/ |
| Disallow | /admin/ |
| Disallow | /browse/cart.cfm |
| Disallow | /browse/imgdownJoe.cfm |
| Disallow | /requests/Lightboxadd.cfm |
| Disallow | /requests/member/ |
| Disallow | /requests/Lightbox.cfm |
| Disallow | /requests/ |
| Disallow | /member/ |
| Disallow | /log.cfm |
| Disallow | /search/index.cfm |
| Disallow | /search/action.cfm |
| Disallow | /pwreset/ |
| Disallow | /browse/highslide-imageservice.cfm |
| Disallow | /*?*forcelogin* |
| Disallow | /support/techreq.cfm* |
| Disallow | /browse/imgdownjoe.cfm |
| Disallow | /*?*CFTOKEN* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |