natureindex.com
robots.txt

Robots Exclusion Standard data for natureindex.com

Resource Scan

Scan Details

Site Domain natureindex.com
Base Domain natureindex.com
Scan Status Ok
Last Scan2024-06-19T22:45:38+00:00
Next Scan 2024-06-26T22:45:38+00:00

Last Scan

Scanned2024-06-19T22:45:38+00:00
URL https://natureindex.com/robots.txt
Redirect https://www.nature.com/nature-index/robots.txt
Redirect Domain www.nature.com
Redirect Base nature.com
Domain IPs 199.232.44.95
Redirect IPs 151.101.0.95, 151.101.128.95, 151.101.192.95, 151.101.64.95
Response IP 199.232.44.95
Found Yes
Hash 30f578140c039e91645ee54da35a8d2a2908cd669d8a888000dea1fe12ed4f84
SimHash 4b394408c205

Groups

petalbot

Rule Path
Disallow /nature-index/

*

Rule Path
Disallow /nature-index/article-api/
Disallow /nature-index/city-maps/
Disallow /nature-index/country-outputs-api/
Disallow /nature-index/country-suggestion
Disallow /nature-index/country-suggestion/news-archive
Disallow /nature-index/global-city-map/data
Disallow /nature-index/institution-outputs-api/
Disallow /nature-index/institution-suggestion
Disallow /nature-index/institution-suggestion/news-archive
Disallow /nature-index/research-leaders/export/
Disallow /nature-index/country-outputs/export/
Disallow /nature-index/country-territory-research-output/export
Disallow /nature-index/institution-outputs/export/
Disallow /nature-index/institution-research-output/export
Disallow /nature-index/news/archive/search/by-country
Disallow /nature-index/news/archive/search/by-institution
Disallow /nature-index/research-leaders/2021/academic-normalized

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.nature.com/nature-index/sitemap.xml