Log In Sign Up

forestpests.org
robots.txt

Robots Exclusion Standard data for forestpests.org

Resource Scan

Scan Details

Site Domain forestpests.org
Base Domain forestpests.org
Scan Status Ok
Last Scan2024-09-27T10:49:05+00:00
Next Scan 2024-10-04T10:49:05+00:00

Last Scan

Scanned2024-09-27T10:49:05+00:00
URL https://www.forestpests.org/robots.txt
Domain IPs 18.238.192.102, 18.238.192.75, 18.238.192.79, 18.238.192.99
Response IP 3.164.206.12
Found Yes
Hash 6c556ee7636168d28a001690960ef42a2948950342d0a20a9ca63057eb7c4358
SimHash 2b2b3e6240d5

Groups

*

Rule Path
Disallow /spdn/
Disallow /addins/
Disallow /Addins/
Disallow /images/
Disallow /admin/
Disallow /browse/imgdownJoe.cfm
Disallow /requests/
Disallow /member/
Disallow /log.cfm
Disallow /search/
Disallow /contribute/
Disallow /css/
Disallow /imgs/
Disallow /pwreset/
Disallow /sidebar/
Disallow /tables/

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top

Comments

  • Generated by RoboGen Standard Edition 1.1 -- http://www.rietta.com/robogen
  • ------------------------------------------------
  • The University of Georgia -- http://www.ipmimages.org/
  • Robot Exclusion File -- robots.txt
  • Author: Chuck Bargeron
  • Last Updated: 10/04/02 13:39:30
Back to top

Warnings

  • 1 invalid line.
Back to top