Log In Sign Up

tarnkappe.info
robots.txt

Robots Exclusion Standard data for tarnkappe.info

Resource Scan

Scan Details

Site Domain tarnkappe.info
Base Domain tarnkappe.info
Scan Status Ok
Last Scan2024-06-30T20:56:43+00:00
Next Scan 2024-07-07T20:56:43+00:00

Last Scan

Scanned2024-06-30T20:56:43+00:00
URL https://tarnkappe.info/robots.txt
Domain IPs 2600:1901:0:caa2::, 34.120.87.59
Response IP 34.120.87.59
Found Yes
Hash 9cd8960a526fc7ee05b5c62dcac224def75dd78a0aebb5fd39ccb10c4498ea6f
SimHash eabe1b879632

Groups

*

Rule Path
Allow /
Allow /feed
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /search/
Disallow /thema/
Disallow /*.html/feed
Disallow /feed?
Disallow /*/embed
Disallow /lesetipps
Disallow /schlagwort
Disallow /wp-content/languages
Disallow /podcast/feed
Disallow /api
Disallow /forum/auth/
Disallow /forum/assets/browser-update*.js
Disallow /forum/users/
Disallow /forum/u/
Disallow /forum/my/
Disallow /forum/badges/
Disallow /forum/search
Disallow /forum/search/
Disallow /forum/tag
Disallow /forum/g
Disallow /forum/email/
Disallow /forum/session
Disallow /forum/session/
Disallow /forum/admin
Disallow /forum/admin/
Disallow /forum/user-api-key
Disallow /forum/user-api-key/
Disallow /forum/*?api_key*
Disallow /forum/*?*api_key*
Disallow /forum/groups
Disallow /forum/groups/
Disallow /forum/t/*/*.rss
Disallow /forum/tags/*.rss
Disallow /forum/c/*.rss

feedviewer
baiduspider-news
applenewsbot
sogou news spider
feedly
feedfetcher-mojeek
flipboardproxy
googlebot-news

Rule Path
Disallow /forum
Disallow /artikel/advertorial
Disallow /listen
Disallow /lesetipps
Disallow /kommentar
Disallow /glosse
Disallow /intern
Disallow /tutorials
Disallow /artikel/empfehlungen

adsbot-google
mediapartners-google
twitterbot
mozilla/5.0 (compatible; ogdwctxcrawler)

Rule Path
Allow /
Allow /search/
Allow /lesetipps
Allow /schlagwort

googlebot-image
yandeximages

Rule Path
Disallow /embetty
Back to top

Other Records

Field Value
sitemap https://tarnkappe.info/sitemap_index.xml
Back to top

Comments

  • Disallow: /embetty
  • Disallow: /*?feed-stats-url
  • Disallow: /*feed-stats-post-id
  • Disallow: /*?wp-nocache
  • Disallow: /*?ref
  • Disallow: /*?PageSpeed
  • Disallow: /*?flattrss_redirect
  • Disallow: /?s
  • Disallow: /*?s
  • Disallow: /*?q
  • Disallow: /*?lang
  • Disallow: /*?search
  • Disallow: /*?fbclid
  • Disallow: /*?msclkid
  • Disallow: /*?cookie-state-change
  • Disallow: /*?http
  • Disallow: /*?paged
  • Disallow: /*?preview
  • Disallow: /*?utm_source
  • Forum
  • News Crawler
  • Allow ads crawler etc on all pages
  • Disallow some images
Back to top