mekster.se
robots.txt

Robots Exclusion Standard data for mekster.se

Resource Scan

Scan Details

Site Domain mekster.se
Base Domain mekster.se
Scan Status Ok
Last Scan2025-09-12T14:22:37+00:00
Next Scan 2025-09-19T14:22:37+00:00

Last Scan

Scanned2025-09-12T14:22:37+00:00
URL https://mekster.se/robots.txt
Redirect https://www.mekster.se/robots.txt
Redirect Domain www.mekster.se
Redirect Base mekster.se
Domain IPs 13.50.46.176, 13.61.158.40, 16.170.114.79
Redirect IPs 13.50.46.176, 13.61.158.40, 16.170.114.79
Response IP 13.50.46.176
Found Yes
Hash 3c045419c18be134e6510a084f50b510b0d3b0ac4c1a87519cd4f126c89f88f5
SimHash 61146c2d8790

Groups

*

Rule Path
Disallow */checkout/
Disallow */customer/
Disallow */catalog/product_compare/
Disallow /deliverydate/
Disallow */catalogsearch/
Disallow /*SID%3D

Other Records

Field Value
crawl-delay 2

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

adidxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

seekportbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

Other Records

Field Value
sitemap https://www.mekster.se/media/categories_cms_google_sitemap.xml
sitemap https://www.mekster.se/media/products_google_sitemap_index.xml
sitemap https://www.mekster.se/media/category_car_pages_2.xml