prisjakt.no
robots.txt

Robots Exclusion Standard data for prisjakt.no

Resource Scan

Scan Details

Site Domain prisjakt.no
Base Domain prisjakt.no
Scan Status Ok
Last Scan2024-09-20T20:53:27+00:00
Next Scan 2024-09-27T20:53:27+00:00

Last Scan

Scanned2024-09-20T20:53:27+00:00
URL https://prisjakt.no/robots.txt
Redirect https://www.prisjakt.no/robots.txt
Redirect Domain www.prisjakt.no
Redirect Base prisjakt.no
Domain IPs 104.18.30.223, 104.18.31.223, 2606:4700::6812:1edf, 2606:4700::6812:1fdf
Redirect IPs 104.18.30.223, 104.18.31.223, 2606:4700::6812:1edf, 2606:4700::6812:1fdf
Response IP 104.18.30.223
Found Yes
Hash 2f6ea3e2add763006ea8ad5b55d7823dc2ba930f5bdc9bb3ea3b955e4136ff74
SimHash e6bfc8e46f42

Groups

msiecrawler
httrack
dataforseobot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

*

Rule Path
Disallow /admin*
Disallow /instore/
Disallow /account/*
Disallow /signup
Disallow /login
Disallow /forgot-password
Disallow /redirect.php
Disallow /go-to-shop
Disallow /forum_redirect.php
Disallow /review_redirect.php
Disallow /redirect_privatlan.php
Disallow /prislista/
Disallow /prisliste/
Disallow /hinnasto/
Disallow /pricelist/
Disallow /bilder/
Disallow /*?affiliator_click=*
Disallow /*%26affiliator_click%3D*
Disallow /alerts/unsubscribe
Disallow /expert.php
Disallow /_internal/switch-platform/*
Disallow /c/*?*&*&*
Allow /c/*?*&*&offset=*
Disallow /c/*?*sort=*
Disallow /c/*?*stores=*
Disallow /c/*?*b_in_stock=*
Disallow /c/*?*r_user_reviews_rating=*
Disallow /c/*?*rg_days_since_indexing=*
Disallow /search?*
Disallow /product.php?o=*&offset=*
Disallow /produkt.php?o=*&offset=*
Disallow /product.php?e=*&edit=*
Disallow /produkt.php?e=*&edit=*
Disallow /brand.php?*&view=*
Disallow /tillverkare.php?*&view=*
Disallow /brand.php?*&c=*
Disallow /tillverkare.php?*&c=*
Disallow /shop.php?*&lista=*
Disallow /butiksinfo.php/reviews?*&lista=*
Disallow /s/*?offset=*
Disallow /extern/*
Disallow /amp/*