Log In Sign Up

bernds-frisurenteam.de
robots.txt

Robots Exclusion Standard data for bernds-frisurenteam.de

Resource Scan

Scan Details

Site Domain bernds-frisurenteam.de
Base Domain bernds-frisurenteam.de
Scan Status Ok
Last Scan2025-07-19T22:10:39+00:00
Next Scan 2025-08-18T22:10:39+00:00

Last Scan

Scanned2025-07-19T22:10:39+00:00
URL https://bernds-frisurenteam.de/robots.txt
Redirect https://www.bernds-frisurenteam.de/robots.txt
Redirect Domain www.bernds-frisurenteam.de
Redirect Base bernds-frisurenteam.de
Domain IPs 2001:8d8:100f:f000::2f0, 217.160.0.3
Redirect IPs 2001:8d8:100f:f000::2f0, 217.160.0.3
Response IP 217.160.0.3
Found Yes
Hash ed244b4e03328c48f628ae27dbf9e17fffbe72b70906758a28b346cd07f6af2d
SimHash b4c4d511cd8a

Groups

*

Rule Path
Disallow /content/form/
Disallow /content/rss/
Disallow /content/tmp/
Disallow /content/pages/
Disallow /filearchive/
Disallow /login.php
Disallow /phpwcms.php
Back to top

Comments

  • http://www.robotstxt.org/
  • https://www.google.com/support/webmasters/bin/answer.py?answer=156449
  • https://support.google.com/webmasters/answer/35769
  • Be careful with these settings related to phpwcms
  • =================================================
  • exclude downloadable files from indexing
  • Disallow: /download.php*
  • The system store several temporary stuff in this dir
  • This will exclude resized images from indexing
  • Disallow: /content/images/
  • And some resizing is done by files here especially cmsimage.php
  • Disallow: /img/
  • phpwcms keeps uploaded files here, better let it as is
  • in general there is no direct access from the frontend to this
  • Better let search engine index everything from this directory
  • Have a look from where your system delivers CSS, JS, and so on
  • Disallow: /template/
  • Block indexing backend files
Back to top