healthprofs.com
robots.txt

Robots Exclusion Standard data for healthprofs.com

Resource Scan

Scan Details

Site Domain healthprofs.com
Base Domain healthprofs.com
Scan Status Ok
Last Scan2024-06-15T08:16:22+00:00
Next Scan 2024-07-15T08:16:22+00:00

Last Scan

Scanned2024-06-15T08:16:22+00:00
URL https://healthprofs.com/robots.txt
Redirect https://www.healthprofs.com/robots.txt
Redirect Domain www.healthprofs.com
Redirect Base healthprofs.com
Domain IPs 54.221.232.151, 54.243.89.234
Redirect IPs 54.221.232.151, 54.243.89.234
Response IP 54.243.89.234
Found Yes
Hash bdf80743d7632acbf0150fb297aaca96e6e2ec1dfd874e229b6b600863161409
SimHash 290473f8daba

Groups

sosospider

Rule Path
Disallow /

yandex bot

Rule Path
Disallow /

larbin

Rule Path
Disallow /

bizwikibot

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

riddlerbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

*

Rule Path
Disallow /includes/
Disallow /misc/
Disallow /modules/
Disallow /profiles/
Disallow /scripts/
Disallow /themes/
Disallow /CHANGELOG.txt
Disallow /cron.php
Disallow /INSTALL.mysql.txt
Disallow /INSTALL.pgsql.txt
Disallow /INSTALL.sqlite.txt
Disallow /install.php
Disallow /INSTALL.txt
Disallow /LICENSE.txt
Disallow /MAINTAINERS.txt
Disallow /update.php
Disallow /UPGRADE.txt
Disallow /xmlrpc.php
Allow /misc/*.css$
Allow /misc/*.css?
Allow /misc/*.js$
Allow /misc/*.js?
Allow /misc/*.gif
Allow /misc/*.jpg
Allow /misc/*.jpeg
Allow /misc/*.png
Allow /modules/*.css$
Allow /modules/*.css?
Allow /modules/*.js$
Allow /modules/*.js?
Allow /modules/*.gif
Allow /modules/*.jpg
Allow /modules/*.jpeg
Allow /modules/*.png
Allow /profiles/*.css$
Allow /profiles/*.css?
Allow /profiles/*.js$
Allow /profiles/*.js?
Allow /profiles/*.gif
Allow /profiles/*.jpg
Allow /profiles/*.jpeg
Allow /profiles/*.png
Allow /themes/*.css$
Allow /themes/*.css?
Allow /themes/*.js$
Allow /themes/*.js?
Allow /themes/*.gif
Allow /themes/*.jpg
Allow /themes/*.jpeg
Allow /themes/*.png
Disallow /admin$
Disallow /admin/
Disallow /comment/reply/
Disallow /filter/tips/
Disallow /node/add
Disallow /search
Disallow /user$
Disallow /user/
Disallow /*/admin$
Disallow /*/admin/
Disallow /*/comment/reply/
Disallow /*/filter/tips/
Disallow /*/node/add/
Disallow /*/search/
Disallow /*/user$
Disallow /*/user/
Disallow /au/admin$
Disallow /au/admin/
Disallow /au/comment/reply/
Disallow /au/filter/tips/
Disallow /au/node/add/
Disallow /au/search/
Disallow /au/user$
Disallow /au/user/
Disallow /ca/admin$
Disallow /ca/admin/
Disallow /ca/comment/reply/
Disallow /ca/filter/tips/
Disallow /ca/node/add/
Disallow /ca/search/
Disallow /ca/user$
Disallow /ca/user/
Disallow /gb/admin$
Disallow /gb/admin/
Disallow /gb/comment/reply/
Disallow /gb/filter/tips/
Disallow /gb/node/add/
Disallow /gb/search/
Disallow /gb/user$
Disallow /gb/user/
Disallow /intl/admin$
Disallow /intl/admin/
Disallow /intl/comment/reply/
Disallow /intl/filter/tips/
Disallow /intl/node/add/
Disallow /intl/search/
Disallow /intl/user$
Disallow /intl/user/
Disallow /us/admin$
Disallow /us/admin/
Disallow /us/comment/reply/
Disallow /us/filter/tips/
Disallow /us/node/add/
Disallow /us/search/
Disallow /us/user$
Disallow /us/user/
Disallow /?q=
Disallow /*?q=

Other Records

Field Value
crawl-delay 10

Comments

  • Resource Directories
  • Static Files
  • Static Drupal resources explicitly allowed
  • Drupal Paths
  • Drupal Paths, wildcard prefix
  • Drupal Paths, au prefix
  • Drupal Paths, ca prefix
  • Drupal Paths, gb prefix
  • Drupal Paths, intl prefix
  • Drupal Paths, us prefix
  • Paths (no unclean URLs)