Log In Sign Up

grahamengineering.com
robots.txt

Robots Exclusion Standard data for grahamengineering.com

Resource Scan

Scan Details

Site Domain grahamengineering.com
Base Domain grahamengineering.com
Scan Status Ok
Last Scan2025-10-22T04:46:40+00:00
Next Scan 2025-11-21T04:46:40+00:00

Last Scan

Scanned2025-10-22T04:46:40+00:00
URL https://grahamengineering.com/robots.txt
Domain IPs 192.124.249.40
Response IP 192.124.249.40
Found Yes
Hash 7d58c1d24f4644562b7d96bf71ff5aba6c5678361d8c7eeca4300451fa4bb95b
SimHash 635608016697

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /scripts
Disallow /wp-admin
Disallow /wp-content/cache
Allow /wp-content/cache/minify
Disallow /trackback
Disallow /comments
Disallow */trackback
Disallow */comments
Disallow /?s=*
Allow /wp-content/uploads

Other Records

Field Value
crawl-delay 3

googlebot-image

Rule Path
Disallow
Allow /*

mediapartners-google*

Rule Path
Disallow
Allow /*

siteauditbot

Rule Path
Allow /*

*

Rule Path
Disallow /wp-content/cache/
Back to top

Comments

  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
  • Designate site map, if one exists.
  • Sitemap: /sitemap.xml
  • RULES: General Search Website Rules.
  • Disallow: /wp-includes
  • Disallow: /wp-content/plugins
  • Disallow: /category/*/*
  • Disallow: /tag/
  • Set crawl speed for search engines.
  • (Most search engines use this.)
  • Google Image
  • Google AdSense
  • SEMRush
  • BEGIN W3TC ROBOTS
  • END W3TC ROBOTS
Back to top