Log In Sign Up

bsavalibrary.com
robots.txt

Robots Exclusion Standard data for bsavalibrary.com

Resource Scan

Scan Details

Site Domain bsavalibrary.com
Base Domain bsavalibrary.com
Scan Status Ok
Last Scan2026-02-14T14:11:07+00:00
Next Scan 2026-03-16T14:11:07+00:00

Last Scan

Scanned2026-02-14T14:11:07+00:00
URL https://bsavalibrary.com/robots.txt
Redirect https://www.bsavalibrary.com/robots.txt
Redirect Domain www.bsavalibrary.com
Redirect Base bsavalibrary.com
Domain IPs 104.26.4.139, 104.26.5.139, 172.67.72.83
Redirect IPs 104.26.4.139, 104.26.5.139, 172.67.72.83
Response IP 104.26.5.139
Found Yes
Hash 30b5c05f492dfc9098383ec55371a59872951eeab48e4e54b6b6fd7f51c5c02c
SimHash 25b5cf3aa117

Groups

googlebot

Rule Path
Disallow /admin/
Disallow /alerts/
Disallow /cache/
Disallow /carnet/
Disallow /librarypass/
Disallow /cart
Disallow /cart/
Disallow /cartregistration/
Disallow /cc/
Disallow /commenting/
Disallow /counterstats/
Disallow /deliver/
Disallow /editcontent/
Disallow /email
Disallow /email/
Disallow /error/
Disallow /export/
Disallow /librarians/uploadbranding
Disallow /licence-admin/
Disallow /managesavedsearches
Disallow /markedlist/
Disallow /mostviewed/
Disallow /offer/
Disallow /openurl/profile
Disallow /orders
Disallow /orders/
Disallow /pricing/
Disallow /profile
Disallow /profile/
Disallow /pubstats/
Disallow /recommendto
Disallow /reference/
Disallow /regactivation
Disallow /regactivation/
Disallow /registration
Disallow /registration/
Disallow /resourcelookup
Disallow /runsavedsearch
Disallow /savedsearches
Disallow /search
Disallow /search/
Disallow /selfservice/
Disallow /session
Disallow /session/
Disallow /statslogredirect
Disallow /statslogwrapper
Disallow /subscribedtitles
Disallow /uploadfile
Disallow /user/
Disallow /workflow/
Disallow /content/contributor
Disallow /content/contributor/
Disallow /content/institution
Disallow /content/institution/
Disallow /content/video-library?*
Disallow /content/video-library/*?*
Disallow /*.gif$
Disallow /*.tif$
Disallow /*.png$
Disallow /*.jpg$
Disallow /*.svg$
Disallow /*.JPG$

yandexbot

Rule Path
Disallow /admin/
Disallow /alerts/
Disallow /cache/
Disallow /carnet/
Disallow /librarypass/
Disallow /cart
Disallow /cart/
Disallow /cartregistration/
Disallow /cc/
Disallow /commenting/
Disallow /counterstats/
Disallow /deliver/
Disallow /editcontent/
Disallow /email
Disallow /email/
Disallow /error/
Disallow /export/
Disallow /librarians/uploadbranding
Disallow /licence-admin/
Disallow /managesavedsearches
Disallow /markedlist/
Disallow /mostviewed/
Disallow /offer/
Disallow /openurl/profile
Disallow /orders
Disallow /orders/
Disallow /pricing/
Disallow /profile
Disallow /profile/
Disallow /pubstats/
Disallow /recommendto
Disallow /reference/
Disallow /regactivation
Disallow /regactivation/
Disallow /registration
Disallow /registration/
Disallow /resourcelookup
Disallow /runsavedsearch
Disallow /savedsearches
Disallow /search
Disallow /search/
Disallow /selfservice/
Disallow /session
Disallow /session/
Disallow /statslogredirect
Disallow /statslogwrapper
Disallow /subscribedtitles
Disallow /uploadfile
Disallow /user/
Disallow /workflow/
Disallow /content/contributor
Disallow /content/contributor/
Disallow /content/institution
Disallow /content/institution/
Disallow /content/video-library?*
Disallow /content/video-library/*?*
Disallow /*.gif$
Disallow /*.tif$
Disallow /*.png$
Disallow /*.jpg$
Disallow /*.svg$
Disallow /*.JPG$

Other Records

Field Value
crawl-delay 4

yandeximages

Rule Path
Disallow /admin/
Disallow /alerts/
Disallow /cache/
Disallow /carnet/
Disallow /librarypass/
Disallow /cart
Disallow /cart/
Disallow /cartregistration/
Disallow /cc/
Disallow /commenting/
Disallow /counterstats/
Disallow /deliver/
Disallow /editcontent/
Disallow /email
Disallow /email/
Disallow /error/
Disallow /export/
Disallow /librarians/uploadbranding
Disallow /licence-admin/
Disallow /managesavedsearches
Disallow /markedlist/
Disallow /mostviewed/
Disallow /offer/
Disallow /openurl/profile
Disallow /orders
Disallow /orders/
Disallow /pricing/
Disallow /profile
Disallow /profile/
Disallow /pubstats/
Disallow /recommendto
Disallow /reference/
Disallow /regactivation
Disallow /regactivation/
Disallow /registration
Disallow /registration/
Disallow /resourcelookup
Disallow /runsavedsearch
Disallow /savedsearches
Disallow /search
Disallow /search/
Disallow /selfservice/
Disallow /session
Disallow /session/
Disallow /statslogredirect
Disallow /statslogwrapper
Disallow /subscribedtitles
Disallow /uploadfile
Disallow /user/
Disallow /workflow/
Disallow /content/contributor
Disallow /content/contributor/
Disallow /content/institution
Disallow /content/institution/
Disallow /content/video-library?*
Disallow /content/video-library/*?*
Disallow /*.gif$
Disallow /*.tif$
Disallow /*.png$
Disallow /*.jpg$
Disallow /*.svg$
Disallow /*.JPG$

Other Records

Field Value
crawl-delay 4

bingbot

Rule Path
Disallow /admin/
Disallow /alerts/
Disallow /cache/
Disallow /carnet/
Disallow /librarypass/
Disallow /cart
Disallow /cart/
Disallow /cartregistration/
Disallow /cc/
Disallow /commenting/
Disallow /counterstats/
Disallow /deliver/
Disallow /editcontent/
Disallow /email
Disallow /email/
Disallow /error/
Disallow /export/
Disallow /librarians/uploadbranding
Disallow /licence-admin/
Disallow /managesavedsearches
Disallow /markedlist/
Disallow /mostviewed/
Disallow /offer/
Disallow /openurl/profile
Disallow /orders
Disallow /orders/
Disallow /pricing/
Disallow /profile
Disallow /profile/
Disallow /pubstats/
Disallow /recommendto
Disallow /reference/
Disallow /regactivation
Disallow /regactivation/
Disallow /registration
Disallow /registration/
Disallow /resourcelookup
Disallow /runsavedsearch
Disallow /savedsearches
Disallow /search
Disallow /search/
Disallow /selfservice/
Disallow /session
Disallow /session/
Disallow /statslogredirect
Disallow /statslogwrapper
Disallow /subscribedtitles
Disallow /uploadfile
Disallow /user/
Disallow /workflow/
Disallow /content/contributor
Disallow /content/contributor/
Disallow /content/institution
Disallow /content/institution/
Disallow /content/video-library?*
Disallow /content/video-library/*?*
Disallow /*.gif$
Disallow /*.tif$
Disallow /*.png$
Disallow /*.jpg$
Disallow /*.svg$
Disallow /*.JPG$

Other Records

Field Value
crawl-delay 2

baiduspider

Rule Path
Disallow /admin/
Disallow /alerts/
Disallow /cache/
Disallow /carnet/
Disallow /librarypass/
Disallow /cart
Disallow /cart/
Disallow /cartregistration/
Disallow /cc/
Disallow /commenting/
Disallow /counterstats/
Disallow /deliver/
Disallow /editcontent/
Disallow /email
Disallow /email/
Disallow /error/
Disallow /export/
Disallow /librarians/uploadbranding
Disallow /licence-admin/
Disallow /managesavedsearches
Disallow /markedlist/
Disallow /mostviewed/
Disallow /offer/
Disallow /openurl/profile
Disallow /orders
Disallow /orders/
Disallow /pricing/
Disallow /profile
Disallow /profile/
Disallow /pubstats/
Disallow /recommendto
Disallow /reference/
Disallow /regactivation
Disallow /regactivation/
Disallow /registration
Disallow /registration/
Disallow /resourcelookup
Disallow /runsavedsearch
Disallow /savedsearches
Disallow /search
Disallow /search/
Disallow /selfservice/
Disallow /session
Disallow /session/
Disallow /statslogredirect
Disallow /statslogwrapper
Disallow /subscribedtitles
Disallow /uploadfile
Disallow /user/
Disallow /workflow/
Disallow /content/contributor
Disallow /content/contributor/
Disallow /content/institution
Disallow /content/institution/
Disallow /content/video-library?*
Disallow /content/video-library/*?*
Disallow /*.gif$
Disallow /*.tif$
Disallow /*.png$
Disallow /*.jpg$
Disallow /*.svg$
Disallow /*.JPG$

Other Records

Field Value
crawl-delay 2

*

Rule Path
Disallow /admin/
Disallow /alerts/
Disallow /cache/
Disallow /carnet/
Disallow /librarypass/
Disallow /cart
Disallow /cart/
Disallow /cartregistration/
Disallow /cc/
Disallow /commenting/
Disallow /counterstats/
Disallow /deliver/
Disallow /editcontent/
Disallow /email
Disallow /email/
Disallow /error/
Disallow /export/
Disallow /librarians/uploadbranding
Disallow /licence-admin/
Disallow /managesavedsearches
Disallow /markedlist/
Disallow /mostviewed/
Disallow /offer/
Disallow /openurl/profile
Disallow /orders
Disallow /orders/
Disallow /pricing/
Disallow /profile
Disallow /profile/
Disallow /pubstats/
Disallow /recommendto
Disallow /reference/
Disallow /regactivation
Disallow /regactivation/
Disallow /registration
Disallow /registration/
Disallow /resourcelookup
Disallow /runsavedsearch
Disallow /savedsearches
Disallow /search
Disallow /search/
Disallow /selfservice/
Disallow /session
Disallow /session/
Disallow /statslogredirect
Disallow /statslogwrapper
Disallow /subscribedtitles
Disallow /uploadfile
Disallow /user/
Disallow /workflow/
Disallow /content/contributor
Disallow /content/contributor/
Disallow /content/institution
Disallow /content/institution/
Disallow /content/video-library?*
Disallow /content/video-library/*?*
Disallow /*.gif$
Disallow /*.tif$
Disallow /*.png$
Disallow /*.jpg$
Disallow /*.svg$
Disallow /*.JPG$

Other Records

Field Value
crawl-delay 2
Back to top

Other Records

Field Value
sitemap http://bsavalibrary.com/sitemap.xml
sitemap http://www.bsavalibrary.com/sitemap.xml
Back to top

Comments

  • Agent-specific settings for indexing the site.
  • Allow Google for full text crawling
  • Google ignores Crawl-delay
  • Crawl-delay: 2
  • URL prefixes
  • URL wildcards
  • Yandex have two bots so they can work twice as slowly
  • Yandex have two bots so they can work twice as slowly
  • Bing specific configuration
  • Baidu specific configuration
  • Generic settings for indexing the site.
Back to top