vargenthor.se
robots.txt

Robots Exclusion Standard data for vargenthor.se

Resource Scan

Scan Details

Site Domain vargenthor.se
Base Domain vargenthor.se
Scan Status Ok
Last Scan2025-07-06T00:05:52+00:00
Next Scan 2025-08-05T00:05:52+00:00

Last Scan

Scanned2025-07-06T00:05:52+00:00
URL https://vargenthor.se/robots.txt
Domain IPs 13.49.199.225
Response IP 13.49.199.225
Found Yes
Hash 4f43cf468a1a830224daa9d609026e6f7b210cdf0a7f09cdb020d9d89bda3f55
SimHash 6a2038122790

Groups

*

Rule Path
Disallow /index.php/
Disallow /*?SID=
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /customer/
Disallow /customer/account/
Disallow /customer/account/login/
Disallow /catalog/product_compare/
Disallow /catalogsearch/
Disallow /wishlist/
Disallow /review/
Disallow /sendfriend/
Disallow /enable-cookies/
Disallow /newsletter/
Disallow /var/
Disallow /pub/
Disallow /media/captcha/
Disallow /media/customer/
Disallow /media/downloadable/
Disallow /media/import/
Disallow /media/tmp/
Disallow /media/wysiwyg/
Disallow /media/catalog/product/cache/
Disallow /media/catalog/category/cache/
Disallow /static/version
Disallow /*.php$
Disallow /lib/
Disallow /pkginfo/
Disallow /report/
Disallow /setup/
Disallow /update/
Disallow /vendor/
Disallow /api/
Disallow /rest/
Disallow /graphql/
Allow /pub/media/catalog/product/
Allow /pub/static/frontend/
Allow /pub/static/adminhtml/
Allow /media/catalog/product/
Allow /static/frontend/
Allow /static/adminhtml/

Comments

  • Allow CSS, JS, Fonts, and Images to be crawled