sciencepubs.org
robots.txt

Robots Exclusion Standard data for sciencepubs.org

Resource Scan

Scan Details

Site Domain sciencepubs.org
Base Domain sciencepubs.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2024-09-25T18:23:34+00:00
Next Scan 2024-10-09T18:23:34+00:00

Last Successful Scan

Scanned2024-09-10T18:22:57+00:00
URL http://sciencepubs.org/robots.txt
Redirect https://www.science.org/robots.txt
Redirect Domain www.science.org
Redirect Base science.org
Domain IPs 52.8.62.150, 54.151.119.244
Redirect IPs 104.18.34.21, 172.64.153.235
Response IP 172.64.153.235
Found Yes
Hash b424573b525d9cf186f0a2db0f5508e5a9d02efb30de1148457dea6795e3f6fb
SimHash 793c58208d53

Groups

*

Rule Path
Disallow /action
Disallow /help
Disallow /search
Disallow /feedback
Disallow /page/account-confirmation-thanks
Disallow /media
Disallow /medical-research
Disallow /servlet/linkout
Disallow /na101/
Disallow /na101v1/
Disallow /na102/
Disallow /doi/mlt/
Disallow /pb/widgets
Disallow /author
Allow /action/showFeed
Allow /action/showJournal
Allow /action/showPublications
Allow /action/showXml
Allow /action/showTopic
Allow /action/showBook
Allow /action/showCoverImage
Allow /action/downloadSupplement

facebookexternalhit
linkedinbot
twitterbot

Rule Path
Allow /

gptbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.science.org/sitemap_index.xml
sitemap https://www.science.org/sitemap-index-1.txt