vinduesgrossisten.dk
robots.txt

Robots Exclusion Standard data for vinduesgrossisten.dk

Resource Scan

Scan Details

Site Domain vinduesgrossisten.dk
Base Domain vinduesgrossisten.dk
Scan Status Ok
Last Scan2024-04-11T06:35:30+00:00
Next Scan 2024-05-11T06:35:30+00:00

Last Scan

Scanned2024-04-11T06:35:30+00:00
URL https://vinduesgrossisten.dk/robots.txt
Domain IPs 104.26.14.103, 104.26.15.103, 172.67.72.215, 2606:4700:20::681a:e67, 2606:4700:20::681a:f67, 2606:4700:20::ac43:48d7
Response IP 104.26.15.103
Found Yes
Hash 89b79eefd379b40e72a41f9ad935dbf3578a40ef436fb24f6ee2642d642ff304
SimHash c1707b1645a5

Groups

*

Rule Path Comment
Disallow /pub/sitemap/sitemap_de.xml -
Disallow /pub/sitemap/sitemap_no.xml -
Disallow /pub/sitemap/sitemap_is.xml -
Disallow /pub/sitemap/sitemap_se.xml -
Disallow /lib/ -
Disallow /*.php$ -
Disallow /pkginfo/ -
Disallow /report/ -
Disallow /var/ -
Disallow /customer/ -
Disallow /sendfriend/ -
Disallow /review/ -
Disallow /*SID%3D -
Disallow /*? -
Disallow /checkout/ -
Disallow /onestepcheckout/ -
Disallow /customer/ -
Disallow /customer/account/ -
Disallow /customer/account/login/ -
Disallow /catalogsearch/ -
Disallow /catalog/product_compare/ -
Disallow /catalog/category/view/ -
Disallow /catalog/product/view/ -
Disallow /app/ -
Disallow /bin/ -
Disallow /dev/ -
Disallow /lib/ -
Disallow /phpserver/ -
Disallow /tag/ -
Disallow /review/ -
Disallow /composer.json -
Disallow /composer.lock -
Disallow /CONTRIBUTING.md -
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html -
Disallow /COPYING.txt -
Disallow /Gruntfile.js -
Disallow /LICENSE.txt -
Disallow /LICENSE_AFL.txt -
Disallow /nginx.conf.sample -
Disallow /package.json -
Disallow /php.ini.sample -
Disallow /RELEASE_NOTES.txt -
Disallow /*?*product_list_mode= -
Disallow /*?*product_list_order= -
Disallow /*?*product_list_limit= -
Disallow /*?*product_list_dir= -
Disallow /*.git -
Disallow /*.CVS -
Disallow /*.Zip$ -
Disallow /*.Svn$ -
Disallow /*.Idea$ -
Disallow /*.Sql$ -
Disallow /*.Tgz$ -
Disallow /pdfs/ Block the /pdfs/directory.
Disallow *.pdf Block pdf files. Non-standard but works for major search engines.
Disallow /*?*PageSpeed=noscript -
Disallow /*?*ModPagespeed=noscript -

Other Records

Field Value
sitemap https://vinduesgrossisten.dk/pub/sitemap/sitemap_dk.xml

Comments

  • Disable sitemap.xml
  • Disable checkout & customer account
  • Disable Search pages
  • Disable common folders
  • Disable Tag & Review (Avoid duplicate content)
  • Common files
  • Disable sorting (Avoid duplicate content)
  • Disable version control folders and others
  • https://app.asana.com/0/1118818755815544/1167723184396866/f
  • Fix links for Google PageSpeed optimizer