www.world.holzkern.com
robots.txt

Robots Exclusion Standard data for www.world.holzkern.com

Resource Scan

Scan Details

Site Domain www.world.holzkern.com
Base Domain holzkern.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-06-17T02:30:49+00:00
Next Scan 2025-09-15T02:30:49+00:00

Last Successful Scan

Scanned2023-11-02T19:54:42+00:00
URL https://www.world.holzkern.com/robots.txt
Domain IPs 172.66.41.9, 172.66.42.247, 2606:4700:3108::ac42:2909, 2606:4700:3108::ac42:2af7
Response IP 172.66.41.9
Found Yes
Hash d5b556ab752ac5fbb975972c6c6b1bdb9098eb2fcce681b7007f20feb1b43dcd
SimHash 0500b85c4bb0

Groups

*

Rule Path
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /pkginfo/
Disallow /report/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /vendor/
Disallow /admin/
Disallow /skin/
Disallow /directory/
Disallow */review/
Disallow */catalog/product_compare/
Disallow */sendfriend/
Disallow */enable-cookies/
Disallow /*SID%3D
Disallow /*.php$
Disallow /composer.json
Disallow /composer.lock
Disallow /CONTRIBUTING.md
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow /COPYING.txt
Disallow /Gruntfile.js
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /nginx.conf.sample
Disallow /package.json
Disallow /php.ini.sample
Disallow /RELEASE_NOTES.txt

Other Records

Field Value
sitemap https://www.world.holzkern.com/media/sitemaps/sitemap_cust_en_us.xml

Comments

  • Crawlers Setup
  • Directories
  • Paths (clean URLs)
  • Paths (not clean URLs)
  • Files