alexanderjaeger.de
robots.txt

Robots Exclusion Standard data for alexanderjaeger.de

Resource Scan

Scan Details

Site Domain alexanderjaeger.de
Base Domain alexanderjaeger.de
Scan Status Ok
Last Scan2025-05-15T23:19:52+00:00
Next Scan 2025-05-29T23:19:52+00:00

Last Scan

Scanned2025-05-15T23:19:52+00:00
URL https://alexanderjaeger.de/robots.txt
Domain IPs 85.13.157.138
Response IP 85.13.157.138
Found Yes
Hash 670d9a385c793b496f2774e69b56db8be8b58abef9da67a9ff5f7fa42c3dc6bd
SimHash 680cc712f0d1

Groups

*

Rule Path
Disallow /usage/
Disallow /logs/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wpcontent/themes/
Disallow /wp-content/plugins/
Disallow /trackback/
Disallow */trackback/
Disallow */studium/
Disallow /FIRST/

googlebot

Rule Path
Disallow /*.js$
Disallow /*.inc$
Disallow /*.css$
Disallow /*.gz$
Disallow /*.cgi$
Disallow /*.wmv$
Disallow /*.png$
Disallow /*.gif$
Disallow /*.jpg$
Disallow /*.cgi$
Disallow /*.xhtml$
Disallow /*.php*
Disallow */trackback*
Disallow /*?*
Disallow /category/
Disallow /tag/
Disallow /wp-*
Allow /wp-content/uploads/

googlebot-image

Rule Path
Allow /*

mediapartners-google*

Rule Path
Disallow /*?*
Allow /wp-content/
Allow /tag/
Allow /category/
Allow /*.php$
Allow /*.js$
Allow /*.inc$
Allow /*.css$
Allow /*.gz$
Allow /*.cgi$
Allow /*.wmv$
Allow /*.cgi$
Allow /*.xhtml$
Allow /*.php*
Allow /*.gif$
Allow /*.jpg$
Allow /*.png$

ia_archiver

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /

teleport*

Rule Path
Disallow /

webwhacker*

Rule Path
Disallow /

webzip*

Rule Path
Disallow /

net attache*

Rule Path
Disallow /

sitesnagger*

Rule Path
Disallow /

httrack*

Rule Path
Disallow /

webcapture*

Rule Path
Disallow /

websauger*

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.alexanderjaeger.de/sitemap.xml

Comments

  • disallow all files in these directories
  • disallow all files ending with these extensions
  • Disallow: /*.php$
  • Disallow: /archives/
  • Disallow: /feed/
  • allow google image bot to search all images
  • allow adsense bot on entire site
  • disallow archiving site
  • disable duggmirror
  • sonstoge Crawler