Log In Sign Up

linsenmax.ch
robots.txt

Robots Exclusion Standard data for linsenmax.ch

Resource Scan

Scan Details

Site Domain linsenmax.ch
Base Domain linsenmax.ch
Scan Status Ok
Last Scan2025-05-20T19:13:01+00:00
Next Scan 2025-06-19T19:13:01+00:00

Last Scan

Scanned2025-05-20T19:13:01+00:00
URL https://linsenmax.ch/robots.txt
Redirect https://www.linsenmax.ch:443/robots.txt
Redirect Domain www.linsenmax.ch
Redirect Base linsenmax.ch
Domain IPs 52.213.166.124, 54.76.204.81
Redirect IPs 104.18.10.207, 104.18.11.207, 2606:4700::6812:acf, 2606:4700::6812:bcf
Response IP 104.18.11.207
Found Yes
Hash dae5595473c79b5de6c362d8b95d6a574e4b4f89ae4fe87512ec1b38fd1dd3f7
SimHash bb08933b60d0

Groups

*

Rule Path
Disallow /*.git$
Disallow /*.svn$
Disallow /*.sql$
Disallow /*.tgz$
Disallow /*.log$
Disallow /*?childSku*
Disallow /*?sort*
Disallow /mobileapp/
Disallow /*?vg_
Disallow /*%26vg_
Disallow /*?price=
Disallow /*%26price%3D

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mj12bot/v*

Rule Path
Disallow /

mj12bot/v1.4.0

Rule Path
Disallow /

mj12bot/v1.4.3

Rule Path
Disallow /

mj12bot/v1.4.5

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.linsenmax.ch/sitemap.xml
Back to top

Comments

  • Crawlers Setup
  • Do not crawl development files and folders (GIT/SVN directories, dump files etc.)
  • Do not crawl sub category pages that are sorted or filtered.
  • Do not crawl mobileapp specific categories
  • Do not crawl sub system directories
  • Disallow: /api/
  • Do not crawl filtered categories
  • Disable agressive crawlers
Back to top