Log In Sign Up

genesdev.cshlp.org
robots.txt

Robots Exclusion Standard data for genesdev.cshlp.org

Resource Scan

Scan Details

Site Domain genesdev.cshlp.org
Base Domain cshlp.org
Scan Status Ok
Last Scan2025-08-08T14:44:36+00:00
Next Scan 2025-09-07T14:44:36+00:00

Last Scan

Scanned2025-08-08T14:44:36+00:00
URL https://genesdev.cshlp.org/robots.txt
Domain IPs 104.232.28.199
Response IP 104.232.28.199
Found Yes
Hash 57847fbe553613200ffb108dda68bdd4bfc3f01e57b8c049454e7d91b456ac85
SimHash a693fd10c1d1

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Disallow /accesslogs
Disallow /adsystem
Disallow /apps/
Disallow /backtocs/
Disallow /browse/
Disallow /careerfocus/
Disallow /cgi/adclick
Disallow /cgi/alerts
Disallow /cgi/authordata
Disallow /cgi/changeuserinfo
Disallow /cgi/citemap
Disallow /citemap
Disallow /cgi/citmgr
Disallow /citmgr
Disallow /cgi/cookietest
Disallow /cgi/crossref-forward-links
Disallow /cgi/ctalert
Disallow /cgi/ctmain
Disallow /cgi/eletter-submit
Disallow /cgi/etoc
Disallow /cgi/external_ref
Disallow /external_ref
Disallow /cgi/flagsearch
Disallow /cgi/folders
Disallow /cgi/issue_pdf
Disallow /cgi/login
Disallow /login
Disallow /cgi/mailafriend
Disallow /email
Disallow /cgi/markedcitation
Disallow /cgi/myjs
Disallow /cgi/pdf_extract
Disallow /cgi/powerpoint
Disallow /powerpoint
Disallow /cgi/register
Disallow /cgi/reprintsidebar
Disallow /cgi/savedsearch
Disallow /cgi/scopus
Disallow /cgi/search
Disallow /search
Disallow /cgi/searchhistory
Disallow /cgi/searchresults
Disallow /cgi/topics
Disallow /classifieds/
Disallow /conf/
Disallow /guides/
Disallow /help
Disallow /honeypot/
Disallow /math/
Disallow /misc/press/
Disallow /press
Disallow /searchall
Disallow /lens
Disallow /lensxml

Other Records

Field Value
crawl-delay 10

fasterfox

Rule Path
Disallow /

msnbot

Rule Path
Disallow /accesslogs
Disallow /adsystem
Disallow /apps/
Disallow /backtocs/
Disallow /browse/
Disallow /careerfocus/
Disallow /cgi/adclick
Disallow /cgi/alerts
Disallow /cgi/authordata
Disallow /cgi/changeuserinfo
Disallow /cgi/citemap
Disallow /citemap
Disallow /cgi/citmgr
Disallow /citmgr
Disallow /cgi/cookietest
Disallow /cgi/crossref-forward-links
Disallow /cgi/ctalert
Disallow /cgi/ctmain
Disallow /cgi/eletter-submit
Disallow /cgi/etoc
Disallow /cgi/external_ref
Disallow /external_ref
Disallow /cgi/flagsearch
Disallow /cgi/folders
Disallow /cgi/issue_pdf
Disallow /cgi/login
Disallow /login
Disallow /cgi/mailafriend
Disallow /email
Disallow /cgi/markedcitation
Disallow /cgi/myjs
Disallow /cgi/pdf_extract
Disallow /cgi/powerpoint
Disallow /powerpoint
Disallow /cgi/register
Disallow /cgi/reprintsidebar
Disallow /cgi/savedsearch
Disallow /cgi/scopus
Disallow /cgi/search
Disallow /search
Disallow /cgi/searchhistory
Disallow /cgi/searchresults
Disallow /cgi/topics
Disallow /classifieds/
Disallow /conf/
Disallow /guides/
Disallow /help
Disallow /honeypot/
Disallow /math/
Disallow /misc/press/
Disallow /press
Disallow /searchall
Disallow /lens
Disallow /lensxml

Other Records

Field Value
crawl-delay 10

ia_archiver

Rule Path
Disallow /accesslogs
Disallow /adsystem
Disallow /apps/
Disallow /backtocs/
Disallow /browse/
Disallow /careerfocus/
Disallow /cgi/adclick
Disallow /cgi/alerts
Disallow /cgi/authordata
Disallow /cgi/changeuserinfo
Disallow /cgi/citemap
Disallow /citemap
Disallow /cgi/citmgr
Disallow /citmgr
Disallow /cgi/cookietest
Disallow /cgi/crossref-forward-links
Disallow /cgi/ctalert
Disallow /cgi/ctmain
Disallow /cgi/eletter-submit
Disallow /cgi/etoc
Disallow /cgi/external_ref
Disallow /external_ref
Disallow /cgi/flagsearch
Disallow /cgi/folders
Disallow /cgi/issue_pdf
Disallow /cgi/login
Disallow /login
Disallow /cgi/mailafriend
Disallow /email
Disallow /cgi/markedcitation
Disallow /cgi/myjs
Disallow /cgi/pdf_extract
Disallow /cgi/powerpoint
Disallow /powerpoint
Disallow /cgi/register
Disallow /cgi/reprintsidebar
Disallow /cgi/savedsearch
Disallow /cgi/scopus
Disallow /cgi/search
Disallow /search
Disallow /cgi/searchhistory
Disallow /cgi/searchresults
Disallow /cgi/topics
Disallow /classifieds/
Disallow /conf/
Disallow /guides/
Disallow /help
Disallow /honeypot/
Disallow /math/
Disallow /misc/press/
Disallow /press
Disallow /searchall
Disallow /lens
Disallow /lensxml

Other Records

Field Value
crawl-delay 10

slurp

Rule Path
Disallow /accesslogs
Disallow /adsystem
Disallow /apps/
Disallow /backtocs/
Disallow /browse/
Disallow /careerfocus/
Disallow /cgi/adclick
Disallow /cgi/alerts
Disallow /cgi/authordata
Disallow /cgi/changeuserinfo
Disallow /cgi/citemap
Disallow /citemap
Disallow /cgi/citmgr
Disallow /citmgr
Disallow /cgi/cookietest
Disallow /cgi/crossref-forward-links
Disallow /cgi/ctalert
Disallow /cgi/ctmain
Disallow /cgi/eletter-submit
Disallow /cgi/etoc
Disallow /cgi/external_ref
Disallow /external_ref
Disallow /cgi/flagsearch
Disallow /cgi/folders
Disallow /cgi/issue_pdf
Disallow /cgi/login
Disallow /login
Disallow /cgi/mailafriend
Disallow /email
Disallow /cgi/markedcitation
Disallow /cgi/myjs
Disallow /cgi/pdf_extract
Disallow /cgi/powerpoint
Disallow /powerpoint
Disallow /cgi/register
Disallow /cgi/reprintsidebar
Disallow /cgi/savedsearch
Disallow /cgi/scopus
Disallow /cgi/search
Disallow /search
Disallow /cgi/searchhistory
Disallow /cgi/searchresults
Disallow /cgi/topics
Disallow /classifieds/
Disallow /conf/
Disallow /guides/
Disallow /help
Disallow /honeypot/
Disallow /math/
Disallow /misc/press/
Disallow /press
Disallow /searchall
Disallow /lens
Disallow /lensxml

Other Records

Field Value
crawl-delay 10

turnitinbot

Rule Path
Disallow /accesslogs
Disallow /adsystem
Disallow /apps/
Disallow /backtocs/
Disallow /browse/
Disallow /careerfocus/
Disallow /cgi/adclick
Disallow /cgi/alerts
Disallow /cgi/authordata
Disallow /cgi/changeuserinfo
Disallow /cgi/citemap
Disallow /citemap
Disallow /cgi/citmgr
Disallow /citmgr
Disallow /cgi/cookietest
Disallow /cgi/crossref-forward-links
Disallow /cgi/ctalert
Disallow /cgi/ctmain
Disallow /cgi/eletter-submit
Disallow /cgi/etoc
Disallow /cgi/external_ref
Disallow /external_ref
Disallow /cgi/flagsearch
Disallow /cgi/folders
Disallow /cgi/issue_pdf
Disallow /cgi/login
Disallow /login
Disallow /cgi/mailafriend
Disallow /email
Disallow /cgi/markedcitation
Disallow /cgi/myjs
Disallow /cgi/pdf_extract
Disallow /cgi/powerpoint
Disallow /powerpoint
Disallow /cgi/register
Disallow /cgi/reprintsidebar
Disallow /cgi/savedsearch
Disallow /cgi/scopus
Disallow /cgi/search
Disallow /search
Disallow /cgi/searchhistory
Disallow /cgi/searchresults
Disallow /cgi/topics
Disallow /classifieds/
Disallow /conf/
Disallow /guides/
Disallow /help
Disallow /honeypot/
Disallow /math/
Disallow /misc/press/
Disallow /press
Disallow /searchall
Disallow /lens
Disallow /lensxml

Other Records

Field Value
crawl-delay 10

baiduspider

Rule Path
Disallow /accesslogs
Disallow /adsystem
Disallow /apps/
Disallow /backtocs/
Disallow /browse/
Disallow /careerfocus/
Disallow /cgi/adclick
Disallow /cgi/alerts
Disallow /cgi/authordata
Disallow /cgi/changeuserinfo
Disallow /cgi/citemap
Disallow /citemap
Disallow /cgi/citmgr
Disallow /citmgr
Disallow /cgi/cookietest
Disallow /cgi/crossref-forward-links
Disallow /cgi/ctalert
Disallow /cgi/ctmain
Disallow /cgi/eletter-submit
Disallow /cgi/etoc
Disallow /cgi/external_ref
Disallow /external_ref
Disallow /cgi/flagsearch
Disallow /cgi/folders
Disallow /cgi/issue_pdf
Disallow /cgi/login
Disallow /login
Disallow /cgi/mailafriend
Disallow /email
Disallow /cgi/markedcitation
Disallow /cgi/myjs
Disallow /cgi/pdf_extract
Disallow /cgi/powerpoint
Disallow /powerpoint
Disallow /cgi/register
Disallow /cgi/reprintsidebar
Disallow /cgi/savedsearch
Disallow /cgi/scopus
Disallow /cgi/search
Disallow /search
Disallow /cgi/searchhistory
Disallow /cgi/searchresults
Disallow /cgi/topics
Disallow /classifieds/
Disallow /conf/
Disallow /guides/
Disallow /help
Disallow /honeypot/
Disallow /math/
Disallow /misc/press/
Disallow /press
Disallow /searchall
Disallow /lens
Disallow /lensxml

Other Records

Field Value
crawl-delay 10

infovell

Rule Path
Disallow /accesslogs
Disallow /adsystem
Disallow /apps/
Disallow /backtocs/
Disallow /browse/
Disallow /careerfocus/
Disallow /cgi/adclick
Disallow /cgi/alerts
Disallow /cgi/authordata
Disallow /cgi/changeuserinfo
Disallow /cgi/citemap
Disallow /citemap
Disallow /cgi/citmgr
Disallow /citmgr
Disallow /cgi/cookietest
Disallow /cgi/crossref-forward-links
Disallow /cgi/ctalert
Disallow /cgi/ctmain
Disallow /cgi/eletter-submit
Disallow /cgi/etoc
Disallow /cgi/external_ref
Disallow /external_ref
Disallow /cgi/flagsearch
Disallow /cgi/folders
Disallow /cgi/issue_pdf
Disallow /cgi/login
Disallow /login
Disallow /cgi/mailafriend
Disallow /email
Disallow /cgi/markedcitation
Disallow /cgi/myjs
Disallow /cgi/pdf_extract
Disallow /cgi/powerpoint
Disallow /powerpoint
Disallow /cgi/register
Disallow /cgi/reprintsidebar
Disallow /cgi/savedsearch
Disallow /cgi/scopus
Disallow /cgi/search
Disallow /search
Disallow /cgi/searchhistory
Disallow /cgi/searchresults
Disallow /cgi/topics
Disallow /classifieds/
Disallow /conf/
Disallow /guides/
Disallow /help
Disallow /honeypot/
Disallow /math/
Disallow /misc/press/
Disallow /press
Disallow /searchall
Disallow /lens
Disallow /lensxml

Other Records

Field Value
crawl-delay 10
Back to top

Comments

  • THIS IS A GENERATED FILE.
  • ANY CHANGES YOU MAKE DIRECTLY TO THIS FILE WILL BE OVERWRITTEN
  • AND LOST THE NEXT TIME THIS FILE IS GENERATED.
  • (This file last generated: May 20, 2025, 6:39 A.M.)
  • BELOW added by: file:/hwpnfs3/mayshare02/shared/webapp/modules/genesdev/resources/shared/build/robots-txt.base.xml
  • BELOW added by: file:/hwpnfs3/mayshare02/shared/webapp/modules/genesdev/resources/publisher/build/robots-txt.mix-in.xml
  • END OF ADDED BLOCK
  • END OF ADDED BLOCK
Back to top