hornetapp.com
robots.txt

Robots Exclusion Standard data for hornetapp.com

Resource Scan

Scan Details

Site Domain hornetapp.com
Base Domain hornetapp.com
Scan Status Ok
Last Scan2024-06-22T14:24:36+00:00
Next Scan 2024-06-29T14:24:36+00:00

Last Scan

Scanned2024-06-22T14:24:36+00:00
URL https://hornetapp.com/robots.txt
Domain IPs 54.192.18.101, 54.192.18.124, 54.192.18.51, 54.192.18.70
Response IP 52.84.229.124
Found Yes
Hash 149cc8bc36bc2948f2bfc20a1d96a2521a2586fe8f95292a6a7b8f2b37646146
SimHash 68168713a7f1

Groups

*

Rule Path
Disallow /reset-password
Disallow /stories/*/comments
Disallow /stories/*/feed
Disallow /stories/*/search
Disallow /stories/*/trackback
Disallow /stories/*?
Disallow /stories/*?*
Disallow /stories/blog
Disallow /stories/cgi-bin
Disallow /stories/comment
Disallow /stories/comment/reply/304/
Disallow /stories/comments
Disallow /stories/feed
Disallow /stories/hornet-best-practices-guide
Disallow /stories/hornet-style-guide
Disallow /stories/trackback
Disallow /stories/wp-admin
Disallow /stories/wp-content/cache
Disallow /stories/wp-content/plugins
Disallow /stories/wp-content/themes
Disallow /stories/wp-includes
Disallow /stories/write-for-us
Disallow /stories-ping
Disallow /about/wp-login.php
Allow /stories/wp-content/uploads
Disallow /stories/wp-content/uploads/cache
Disallow /community
Disallow /contribute
Allow /community/knowledge-base
Allow /contribute/knowledge-base
Allow /community/programs
Allow /contribute/programs

googlebot

Rule Path
Disallow /stories/*.cgi$
Disallow /stories/*.gz$
Disallow /stories/*.inc$
Disallow /stories/*.wmv$
Disallow /stories/*.xhtml$
Disallow /stories/*/comments
Disallow /stories/*/feed
Disallow /stories/*/page/
Disallow /stories/*/search
Disallow /stories/blog
Disallow /stories/calendar
Disallow /stories/cmnty-translation-queue*
Disallow /stories/comment
Disallow /stories/comments
Disallow /stories/hornet-best-practices-guide
Disallow /stories/hornet-style-guide
Disallow /stories/omgwtflol
Disallow /stories/wp-content/cache
Disallow /stories/wp-content/uploads/cache
Disallow /stories/wp-includes
Disallow /stories/write-for-us
Allow /stories/*.css?*
Allow /stories/*.js?*
Allow /stories/*.ttf?*
Allow /stories/*.woff?*
Allow /stories/*bunyad_custom_css*
Allow /stories/*sharify-style.php$

duggmirror

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow
Allow /stories/*

mediapartners-google*

Rule Path
Disallow
Allow /stories/*

Other Records

Field Value
sitemap https://hornet.com/sitemap.xml

Comments

  • disable duggmirror
  • allow google image bot to search all images
  • allow adsense bot on entire site