Log In Sign Up

eindhovensdagblad.nl
robots.txt

Robots Exclusion Standard data for eindhovensdagblad.nl

Resource Scan

Scan Details

Site Domain eindhovensdagblad.nl
Base Domain eindhovensdagblad.nl
Scan Status Ok
Last Scan2024-11-09T16:21:52+00:00
Next Scan 2024-11-23T16:21:52+00:00

Last Scan

Scanned2024-11-09T16:21:52+00:00
URL https://eindhovensdagblad.nl/robots.txt
Redirect https://www.ed.nl/robots.txt
Redirect Domain www.ed.nl
Redirect Base ed.nl
Domain IPs 2600:1413:b000:6::17d5:2bd3, 2600:1413:b000:6::17d5:2bd8, 96.17.96.21, 96.17.96.32
Redirect IPs 2600:1413:b000:6::17d5:2bd0, 2600:1413:b000:6::17d5:2bda, 96.17.96.6, 96.17.96.9
Response IP 23.44.5.58
Found Yes
Hash eb2744de66b767b360f69af2833cb1ada1e7a02ea59d056bd4ded08021c8ba6d
SimHash 61198b58dd75

Groups

*

Rule Path
Disallow /*webview
Disallow /auth
Disallow /*widget*
Disallow /*?*otag=
Disallow /*?*abo_type=
Disallow /*?*utm_source=
Disallow /*?*currentArticleId=
Disallow /*?*articleUrl=
Disallow /zoeken?query=
Disallow /inloggen?*
Disallow /login?*
Disallow *~ab9e5892*
Disallow *~af7ac112*
Disallow *~aaf8b8d8*
Disallow *~ae9fae67*
Disallow *~af8b8d8*
Disallow *~a2575106*
Disallow *?*redirect_url=*

twitterbot

Rule Path
Allow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.ed.nl/sitemap.xml
sitemap https://www.ed.nl/sitemap-news.xml
Back to top

Comments

  • Alle auteurs-, naburige en databankrechten die op de inhoud en opmaak van de DPG Media websites
  • en DPG Media apps rusten, worden door DPG Media BV uitdrukkelijk voorbehouden. De inhoud van de
  • DPG Media websites en apps is uitsluitend voor persoonlijk, niet-commercieel gebruik en het is
  • niet toegestaan om gegevens zoals tekst, afbeeldingen, audio, video of code van de websites of
  • de apps door middel van scraping (of een andere geautomatiseerde werkwijze) te vergaren.
  • Zie ook de Gebruikersvoorwaarden van DPG Media B.V. op www.dpgmedia.nl/gebruiksvoorwaarden
  • All copyrights, neighbouring rights and database rights in the content and layout of the
  • DPG Media websites and DPG Media apps are explicitly reserved by DPG Media BV. The content of
  • the DPG Media websites and DPG Media apps is for personal, non-commercial use only and it is not
  • allowed to collect data such as text, images, audio, video or code from the websites or from the
  • apps by means of scraping (or any other automated method).
  • See also the terms of use of DPG Media B.V. at www.dpgmedia.nl/gebruiksvoorwaarden
  • Tell robots which pages are not very interesting
  • Articles which should not be listed in google search index:
  • tu-e-zet-directeur-op-non-actief~ab9e5892/
  • Tell robots not to crawl redirect urls
Back to top