Log In Sign Up

hunters.lk
robots.txt

Robots Exclusion Standard data for hunters.lk

Resource Scan

Scan Details

Site Domain hunters.lk
Base Domain hunters.lk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-09T22:44:29+00:00
Next Scan 2025-11-08T22:44:29+00:00

Last Successful Scan

Scanned2025-09-16T11:59:04+00:00
URL https://hunters.lk/robots.txt
Domain IPs 104.21.32.106, 172.67.185.143, 2606:4700:3033::6815:206a, 2606:4700:3033::ac43:b98f
Response IP 104.21.32.106
Found Yes
Hash e25a837d146593820b4f4dda4f788c5bd1245cd0afe625a31978a8b0997fc354
SimHash 5806fb8b8515

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

bingbot

Rule Path
Disallow

slurp

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

uptimerobot

Rule Path
Disallow

*

Rule Path
Allow */modules/*.css
Allow */modules/*.js
Allow */modules/*.png
Allow */modules/*.jpg
Allow /js/jquery/*
Disallow /*?order=
Disallow /*?tag=
Disallow /*?id_currency=
Disallow /*?search_query=
Disallow /*?back=
Disallow /*?n=
Disallow /*%26order%3D
Disallow /*%26tag%3D
Disallow /*%26id_currency%3D
Disallow /*%26search_query%3D
Disallow /*%26back%3D
Disallow /*%26n%3D
Disallow /*controller%3Daddresses
Disallow /*controller%3Daddress
Disallow /*controller%3Dauthentication
Disallow /*controller%3Dcart
Disallow /*controller%3Ddiscount
Disallow /*controller%3Dfooter
Disallow /*controller%3Dget-file
Disallow /*controller%3Dheader
Disallow /*controller%3Dhistory
Disallow /*controller%3Didentity
Disallow /*controller%3Dimages.inc
Disallow /*controller%3Dinit
Disallow /*controller%3Dmy-account
Disallow /*controller%3Dorder
Disallow /*controller%3Dorder-slip
Disallow /*controller%3Dorder-detail
Disallow /*controller%3Dorder-follow
Disallow /*controller%3Dorder-return
Disallow /*controller%3Dorder-confirmation
Disallow /*controller%3Dpagination
Disallow /*controller%3Dpassword
Disallow /*controller%3Dpdf-invoice
Disallow /*controller%3Dpdf-order-return
Disallow /*controller%3Dpdf-order-slip
Disallow /*controller%3Dproduct-sort
Disallow /*controller%3Dsearch
Disallow /*controller%3Dstatistics
Disallow /*controller%3Dattachment
Disallow /*controller%3Dguest-tracking
Disallow /app/
Disallow /cache/
Disallow /classes/
Disallow /config/
Disallow /controllers/
Disallow /download/
Disallow /js/
Disallow /localization/
Disallow /log/
Disallow /mails/
Disallow /modules/
Disallow /override/
Disallow /pdf/
Disallow /src/
Disallow /tools/
Disallow /translations/
Disallow /upload/
Disallow /var/
Disallow /vendor/
Disallow /webservice/
Disallow /app/
Disallow /cache/
Disallow /classes/
Disallow /config/
Disallow /controllers/
Disallow /download/
Disallow /js/
Disallow /localization/
Disallow /log/
Disallow /mails/
Disallow /modules/
Disallow /override/
Disallow /pdf/
Disallow /src/
Disallow /tools/
Disallow /translations/
Disallow /upload/
Disallow /var/
Disallow /vendor/
Disallow /webservice/
Disallow /en/app/
Disallow /en/cache/
Disallow /en/classes/
Disallow /en/config/
Disallow /en/controllers/
Disallow /en/download/
Disallow /en/js/
Disallow /en/localization/
Disallow /en/log/
Disallow /en/mails/
Disallow /en/modules/
Disallow /en/override/
Disallow /en/pdf/
Disallow /en/src/
Disallow /en/tools/
Disallow /en/translations/
Disallow /en/upload/
Disallow /en/var/
Disallow /en/vendor/
Disallow /en/webservice/
Disallow /*en/password-recovery
Disallow /*en/address
Disallow /*en/addresses
Disallow /*en/login
Disallow /*en/cart
Disallow /*en/discount
Disallow /*en/order-history
Disallow /*en/identity
Disallow /*en/my-account
Disallow /*en/order-follow
Disallow /*en/credit-slip
Disallow /*en/order
Disallow /*en/search
Disallow /*en/guest-tracking
Disallow /*en/order-confirmation
Back to top

Other Records

Field Value
sitemap https://www.hunters.lk/sitemap.xml
Back to top

Comments

  • robots.txt automatically generated by PrestaShop e-commerce open-source solution
  • https://www.prestashop.com - https://www.prestashop.com/forums
  • This file is to prevent the crawling and indexing of certain parts
  • of your site by web crawlers and spiders run by sites like Yahoo!
  • and Google. By telling these "robots" where not to go on your site,
  • you save bandwidth and server resources.
  • For more information about the robots.txt standard, see:
  • https://www.robotstxt.org/robotstxt.html
  • Block all bots by default
  • Allow Googlebot
  • Allow Googlebot for images
  • Allow Bing
  • Allow Yahoo
  • Allow Google AdsBot
  • Allow Google Mobile
  • Allow UptimeRobot
  • Sitemap (optional, but helps good bots)
  • Allow Directives
  • Private pages
  • Directories for hunters.lk
  • Files
Back to top