Log In Sign Up

jbe-platform.com
robots.txt

Robots Exclusion Standard data for jbe-platform.com

Resource Scan

Scan Details

Site Domain jbe-platform.com
Base Domain jbe-platform.com
Scan Status Ok
Last Scan2025-11-02T18:52:49+00:00
Next Scan 2025-12-02T18:52:49+00:00

Last Scan

Scanned2025-11-02T18:52:49+00:00
URL https://jbe-platform.com/robots.txt
Redirect https://www.jbe-platform.com/robots.txt
Redirect Domain www.jbe-platform.com
Redirect Base jbe-platform.com
Domain IPs 104.26.8.249, 104.26.9.249, 172.67.70.228
Redirect IPs 104.26.8.249, 104.26.9.249, 172.67.70.228
Response IP 104.26.8.249
Found Yes
Hash 215491e7154a26e43656dcca8b3bfbf1752d1f4bfa078437b87791224a70d3d8
SimHash 2dbdcd3aa397

Groups

googlebot

Rule Path
Disallow /admin/
Disallow /alerts/
Disallow /cache/
Disallow /carnet/
Disallow /cart
Disallow /cart/
Disallow /cartregistration/
Disallow /cc/
Disallow /commenting/
Disallow /counterstats/
Disallow /deliver/
Disallow /editcontent/
Disallow /email
Disallow /email/
Disallow /error/
Disallow /export/
Disallow /librarians/uploadbranding
Disallow /licence-admin/
Disallow /managesavedsearches
Disallow /markedlist/
Disallow /mostviewed/
Disallow /offer/
Disallow /openurl/profile
Disallow /orders
Disallow /orders/
Disallow /pricing/
Disallow /profile
Disallow /profile/
Disallow /pubstats/
Disallow /recommendto
Disallow /reference/
Disallow /regactivation
Disallow /regactivation/
Disallow /registration
Disallow /registration/
Disallow /resourcelookup
Disallow /runsavedsearch
Disallow /savedsearches
Disallow /search
Disallow /search/
Disallow /selfservice/
Disallow /session
Disallow /session/
Disallow /statslogredirect
Disallow /statslogwrapper
Disallow /subscribedtitles
Disallow /uploadfile
Disallow /user/
Disallow /workflow/
Disallow /content/contributor
Disallow /content/contributor/
Disallow /content/institution
Disallow /content/institution/

yandexbot

Rule Path
Disallow /admin/
Disallow /alerts/
Disallow /cache/
Disallow /carnet/
Disallow /cart
Disallow /cart/
Disallow /cartregistration/
Disallow /cc/
Disallow /commenting/
Disallow /counterstats/
Disallow /deliver/
Disallow /editcontent/
Disallow /email
Disallow /email/
Disallow /error/
Disallow /export/
Disallow /librarians/uploadbranding
Disallow /licence-admin/
Disallow /managesavedsearches
Disallow /markedlist/
Disallow /mostviewed/
Disallow /offer/
Disallow /openurl/profile
Disallow /orders
Disallow /orders/
Disallow /pricing/
Disallow /profile
Disallow /profile/
Disallow /pubstats/
Disallow /recommendto
Disallow /reference/
Disallow /regactivation
Disallow /regactivation/
Disallow /registration
Disallow /registration/
Disallow /resourcelookup
Disallow /runsavedsearch
Disallow /savedsearches
Disallow /search
Disallow /search/
Disallow /selfservice/
Disallow /session
Disallow /session/
Disallow /statslogredirect
Disallow /statslogwrapper
Disallow /subscribedtitles
Disallow /uploadfile
Disallow /user/
Disallow /workflow/
Disallow /content/contributor
Disallow /content/contributor/
Disallow /content/institution
Disallow /content/institution/

Other Records

Field Value
crawl-delay 4

yandeximages

Rule Path
Disallow /admin/
Disallow /alerts/
Disallow /cache/
Disallow /carnet/
Disallow /cart
Disallow /cart/
Disallow /cartregistration/
Disallow /cc/
Disallow /commenting/
Disallow /counterstats/
Disallow /deliver/
Disallow /editcontent/
Disallow /email
Disallow /email/
Disallow /error/
Disallow /export/
Disallow /librarians/uploadbranding
Disallow /licence-admin/
Disallow /managesavedsearches
Disallow /markedlist/
Disallow /mostviewed/
Disallow /offer/
Disallow /openurl/profile
Disallow /orders
Disallow /orders/
Disallow /pricing/
Disallow /profile
Disallow /profile/
Disallow /pubstats/
Disallow /recommendto
Disallow /reference/
Disallow /regactivation
Disallow /regactivation/
Disallow /registration
Disallow /registration/
Disallow /resourcelookup
Disallow /runsavedsearch
Disallow /savedsearches
Disallow /search
Disallow /search/
Disallow /selfservice/
Disallow /session
Disallow /session/
Disallow /statslogredirect
Disallow /statslogwrapper
Disallow /subscribedtitles
Disallow /uploadfile
Disallow /user/
Disallow /workflow/
Disallow /content/contributor
Disallow /content/contributor/
Disallow /content/institution
Disallow /content/institution/

Other Records

Field Value
crawl-delay 4

bingbot

Rule Path
Disallow /admin/
Disallow /alerts/
Disallow /cache/
Disallow /carnet/
Disallow /cart
Disallow /cart/
Disallow /cartregistration/
Disallow /cc/
Disallow /commenting/
Disallow /counterstats/
Disallow /deliver/
Disallow /editcontent/
Disallow /email
Disallow /email/
Disallow /error/
Disallow /export/
Disallow /librarians/uploadbranding
Disallow /licence-admin/
Disallow /managesavedsearches
Disallow /markedlist/
Disallow /mostviewed/
Disallow /offer/
Disallow /openurl/profile
Disallow /orders
Disallow /orders/
Disallow /pricing/
Disallow /profile
Disallow /profile/
Disallow /pubstats/
Disallow /recommendto
Disallow /reference/
Disallow /regactivation
Disallow /regactivation/
Disallow /registration
Disallow /registration/
Disallow /resourcelookup
Disallow /runsavedsearch
Disallow /savedsearches
Disallow /search
Disallow /search/
Disallow /selfservice/
Disallow /session
Disallow /session/
Disallow /statslogredirect
Disallow /statslogwrapper
Disallow /subscribedtitles
Disallow /uploadfile
Disallow /user/
Disallow /workflow/
Disallow /content/contributor
Disallow /content/contributor/
Disallow /content/institution
Disallow /content/institution/

Other Records

Field Value
crawl-delay 2

baiduspider

Rule Path
Disallow /admin/
Disallow /alerts/
Disallow /cache/
Disallow /carnet/
Disallow /cart
Disallow /cart/
Disallow /cartregistration/
Disallow /cc/
Disallow /commenting/
Disallow /counterstats/
Disallow /deliver/
Disallow /editcontent/
Disallow /email
Disallow /email/
Disallow /error/
Disallow /export/
Disallow /librarians/uploadbranding
Disallow /licence-admin/
Disallow /managesavedsearches
Disallow /markedlist/
Disallow /mostviewed/
Disallow /offer/
Disallow /openurl/profile
Disallow /orders
Disallow /orders/
Disallow /pricing/
Disallow /profile
Disallow /profile/
Disallow /pubstats/
Disallow /recommendto
Disallow /reference/
Disallow /regactivation
Disallow /regactivation/
Disallow /registration
Disallow /registration/
Disallow /resourcelookup
Disallow /runsavedsearch
Disallow /savedsearches
Disallow /search
Disallow /search/
Disallow /selfservice/
Disallow /session
Disallow /session/
Disallow /statslogredirect
Disallow /statslogwrapper
Disallow /subscribedtitles
Disallow /uploadfile
Disallow /user/
Disallow /workflow/
Disallow /content/contributor
Disallow /content/contributor/
Disallow /content/institution
Disallow /content/institution/

Other Records

Field Value
crawl-delay 2

*

Rule Path
Disallow /admin/
Disallow /alerts/
Disallow /cache/
Disallow /carnet/
Disallow /cart
Disallow /cart/
Disallow /cartregistration/
Disallow /cc/
Disallow /commenting/
Disallow /counterstats/
Disallow /deliver/
Disallow /editcontent/
Disallow /email
Disallow /email/
Disallow /error/
Disallow /export/
Disallow /librarians/uploadbranding
Disallow /licence-admin/
Disallow /managesavedsearches
Disallow /markedlist/
Disallow /mostviewed/
Disallow /offer/
Disallow /openurl/profile
Disallow /orders
Disallow /orders/
Disallow /pricing/
Disallow /profile
Disallow /profile/
Disallow /pubstats/
Disallow /recommendto
Disallow /reference/
Disallow /regactivation
Disallow /regactivation/
Disallow /registration
Disallow /registration/
Disallow /resourcelookup
Disallow /runsavedsearch
Disallow /savedsearches
Disallow /search
Disallow /search/
Disallow /selfservice/
Disallow /session
Disallow /session/
Disallow /statslogredirect
Disallow /statslogwrapper
Disallow /subscribedtitles
Disallow /uploadfile
Disallow /user/
Disallow /workflow/
Disallow /content/contributor
Disallow /content/contributor/
Disallow /content/institution
Disallow /content/institution/

Other Records

Field Value
crawl-delay 2
Back to top

Other Records

Field Value
sitemap https://www.jbe-platform.com/sitemap.xml
sitemap https://www.jbe-platform.com/sitemap.xml
Back to top

Comments

  • Agent-specific settings for indexing the site.
  • Allow Google for full text crawling
  • Google ignores Crawl-delay
  • Crawl-delay: 2
  • URL prefixes
  • URL wildcards
  • Yandex have two bots so they can work twice as slowly
  • Yandex have two bots so they can work twice as slowly
  • Bing specific configuration
  • Baidu specific configuration
  • Generic settings for indexing the site.
Back to top