Log In Sign Up

about-africa.de
robots.txt

Robots Exclusion Standard data for about-africa.de

Resource Scan

Scan Details

Site Domain about-africa.de
Base Domain about-africa.de
Scan Status Ok
Last Scan2024-11-14T13:40:56+00:00
Next Scan 2024-11-21T13:40:56+00:00

Last Scan

Scanned2024-11-14T13:40:56+00:00
URL https://about-africa.de/robots.txt
Redirect https://www.about-africa.de/robots.txt
Redirect Domain www.about-africa.de
Redirect Base about-africa.de
Domain IPs 85.13.132.126
Redirect IPs 85.13.132.126
Response IP 85.13.132.126
Found Yes
Hash 1a5808968bc4c8fbe9751564d3c54f283ca7c351f45462656b05777817553c94
SimHash e20d8559cae1

Groups

backlinkcrawler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

seoscanners.net/1

Rule Path
Disallow /

spbot

Rule Path
Disallow /

hypercrawl

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

googlebot-video

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

it2media-domain-crawler

Rule Path
Disallow /

*

Rule Path
Disallow /administrator/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /css-dateien/
Disallow /images/
Disallow /galerien/
Disallow /includes/
Disallow /include-dateien/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /modules/
Disallow /plugins/
Disallow /tmp/
Disallow /schau/
Disallow /guapore/
Disallow /bilder/
Disallow /component/content/
Disallow /component/contact/
Disallow /component/mailto/
Disallow /component/mailto
Disallow /component/tags/
Disallow /component/weblinks/
Disallow /templates/wohnmichl/js/robotaway/
Disallow /schlagworte/
Allow /cache/images/
Allow /images/
Allow /plugins/system/lazyloadforjoomla/assets/images/blank.gif
Allow /plugins/system/jcemediabox/themes/standard/css/
Allow /plugins/system/jcemediabox/css/
Allow /plugins/system/jcemediabox/js/
Allow /modules/mod_itpshare/style.css
Allow /plugins/content/jw_allvideos/jw_allvideos/tmpl/Responsive/css/template.css
Allow /plugins/content/jw_allvideos/jw_allvideos/includes/js/
Allow /plugins/system/lazyloadforjoomla/assets/js/lazyloadforjoomla-jquery.js
Allow /plugins/content/jplayer/tmpl/css/style.css
Allow /modules/mod_db8socialmediashare/assets/db8socialmediashare_style-min.css

googlebot-image

Rule Path
Disallow /

googlebot-video

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.about-africa.de/component/osmap/?view=xml&id=1&format=xml
Back to top

Comments

  • http://OpenLinkProfiler.org/bot
  • http://www.seograph.net/bot.html
  • http://www.meanpath.com/meanpathbot.html
  • http://webmeup-crawler.com/
  • http://www.majestic12.co.uk Scannen mailto-Adressen trotz nofollow:
  • http://www.opensiteexplorer.org/dotbot Scannen mailto-Adressen trotz nofollow:
  • Ignoriert wohl doch robots.txt
  • Mittlerweile in .htaccess bad_bot. Ist keine Suchmaschine. Irgend komerzieller Dienst.
  • Auch nur scheiß Analysetool
  • Total wirres Crawlen. Auch in .htaccess
  • If the Joomla site is installed within a folder such as at
  • e.g. www.example.com/joomla/ the robots.txt file MUST be
  • moved to the site root at e.g. www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to the disallowed
  • path, e.g. the Disallow rule for the /administrator/ folder
  • MUST be changed to read Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://www.sxw.org.uk/computing/robots/check.html
  • Disallow: /schwarzes-brett/
  • Disallow: /archiviertes-schwarzes-brett/
  • Disallow: /event/
  • Disallow: /zeugs/
  • Disallow: /drumrum/
  • Disallow: /pinboard/
  • Disallow: /sonstiges-querbeet/
Back to top