Log In Sign Up

nrc.nl
robots.txt

Robots Exclusion Standard data for nrc.nl

Resource Scan

Scan Details

Site Domain nrc.nl
Base Domain nrc.nl
Scan Status Ok
Last Scan2024-09-28T09:48:17+00:00
Next Scan 2024-10-05T09:48:17+00:00

Last Scan

Scanned2024-09-28T09:48:17+00:00
URL https://nrc.nl/robots.txt
Redirect https://www.nrc.nl/robots.txt
Redirect Domain www.nrc.nl
Redirect Base nrc.nl
Domain IPs 46.22.183.139
Redirect IPs 46.22.183.139
Response IP 46.22.183.139
Found Yes
Hash 045b9a4469a3e25c76b1d4aea881b8e0b351f6f05a62419d4635debeeaa2e3d3
SimHash e8389f518e77

Groups

*

Rule Path
Disallow /*/data/related$
Disallow /api/wordproof/
Disallow /data/
Disallow /de/data/s3/
Disallow /login/
Disallow /nieuwsbrieven/preview/
Disallow /paywall-api/
Disallow /search/

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.nrc.nl/sitemap/index.xml
Back to top

Comments

  • All copyrights, neighbouring rights and database rights in the content and
  • layout of this website/app are explicitly reserved and are for personal,
  • non-commercial use only. In accordance with Article 4 of the Directive on
  • Copyright in the Digital Single Market (CDSM) and its transposition into
  • the law of the applicable Member State, all content of this website on
  • which it is made available is not to be used for the purposes of text and
  • data mining, extraction, scraping and/or the use of programs or robots for
  • automatic data collection and/or extraction of digital data, whether for
  • machine learning or artificial intelligence purposes or otherwise. See
  • also the Terms and Conditions of this website.
Back to top