recklinghaeuser-zeitung.de
robots.txt

Robots Exclusion Standard data for recklinghaeuser-zeitung.de

Resource Scan

Scan Details

Site Domain recklinghaeuser-zeitung.de
Base Domain recklinghaeuser-zeitung.de
Scan Status Ok
Last Scan2024-09-19T09:09:24+00:00
Next Scan 2024-09-26T09:09:24+00:00

Last Scan

Scanned2024-09-19T09:09:24+00:00
URL https://recklinghaeuser-zeitung.de/robots.txt
Domain IPs 37.120.183.250
Response IP 37.120.183.250
Found Yes
Hash 8965389d351c571886718f9ed8678bcf7a33d309041e300f8174b7ed0843dbb4
SimHash 2a344b0a8dec

Groups

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

facebookexternalhit

Rule Path
Allow /*?*utm_source=

twitterbot

Rule Path
Allow /*?*utm_source=

*

Rule Path
Disallow /wp-admin/
Disallow /wp-content/plugins/cyan-statify-addon/hit.php
Disallow *RetransferUrl%3D*
Disallow /login/?
Disallow /User/Modules/LoginModuleHandler.ashx
Disallow /*?s=
Disallow /author/*/feed/
Disallow /schlagwort/*/feed/
Disallow /search/*/feed/
Disallow /page/
Disallow /dpa-evergreen/
Disallow /dpa-infoline-mhb/
Disallow /dpa-infoline-zvr/
Disallow /dpa-infoline/
Disallow /dpa-regionale-nrw-mhb/
Disallow /dpa-regionale-nrw-zvr/
Disallow /dpa-regionale-nrw/
Disallow /dpa-sportsline-mhb/
Disallow /dpa-sportsline-zvr/
Disallow /dpa-sportsline/
Disallow /dpa-starline-mhb/
Disallow /dpa-starline-zvr/
Disallow /dpa-starline/
Disallow /testrn/
Disallow /*?cmpscreencustom
Disallow /*?*utm_source=
Allow /wp-content/uploads/
Allow /wp-admin/admin-ajax.php

Comments

  • BEGIN Block bots that feed AI models
  • END Block bots that feed AI models
  • Legal notice: We expressly reserve the right to use our content for commercial text and data mining (Section 44b UrhG).
  • The use of robots or other automated means to access this site or to collect or mine data without the express permission is strictly prohibited.
  • At our discretion, we may permit certain automated access to certain pages.