Log In Sign Up

elsalg.dk
robots.txt

Robots Exclusion Standard data for elsalg.dk

Resource Scan

Scan Details

Site Domain elsalg.dk
Base Domain elsalg.dk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-07-11T09:06:04+00:00
Next Scan 2025-07-18T09:06:04+00:00

Last Successful Scan

Scanned2025-06-10T06:48:56+00:00
URL https://elsalg.dk/robots.txt
Domain IPs 104.26.8.60, 104.26.9.60, 172.67.71.206, 2606:4700:20::681a:83c, 2606:4700:20::681a:93c, 2606:4700:20::ac43:47ce
Response IP 104.26.9.60
Found Yes
Hash a09b2d586d9b37b5a37dc42855a0d610798265f308ce0ed92d67b3176241ff36
SimHash c5705a4e00b4

Groups

*

Rule Path
Disallow /index.php/
Disallow /*?
Disallow /checkout/
Disallow /app/
Disallow /lib/
Disallow /*.php$
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /catalog/
Disallow /customer/
Disallow /sendfriend/
Disallow /review/
Disallow /*SID%3D
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /tag/
Disallow /review/
Disallow /composer.json
Disallow /composer.lock
Disallow /CONTRIBUTING.md
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow /COPYING.txt
Disallow /Gruntfile.js
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /nginx.conf.sample
Disallow /package.json
Disallow /php.ini.sample
Disallow /RELEASE_NOTES.txt
Disallow /*?*product_list_mode=
Disallow /*?*product_list_order=
Disallow /*?*product_list_limit=
Disallow /*?*product_list_dir=
Disallow /*.git
Disallow /*.CVS
Disallow /*.Zip$
Disallow /*.Svn$
Disallow /*.Idea$
Disallow /*.Sql$
Disallow /*.Tgz$

Other Records

Field Value
crawl-delay 30
Back to top

Other Records

Field Value
sitemap https://elsalg.dk/pub/media/sitemap/sitemap.xml
Back to top

Comments

  • All User agents
  • Crawl Delay
  • Disallow Magento 2 defaults.
  • Disable common folders
  • Disable Tag & Review (Avoid duplicate content)
  • Common files
  • Disable sorting (Avoid duplicate content)
  • Disable version control folders and others
Back to top