jotex.no
robots.txt

Robots Exclusion Standard data for jotex.no

Resource Scan

Scan Details

Site Domain jotex.no
Base Domain jotex.no
Scan Status Ok
Last Scan2024-10-22T23:55:33+00:00
Next Scan 2024-11-21T23:55:33+00:00

Last Scan

Scanned2024-10-22T23:55:33+00:00
URL https://jotex.no/robots.txt
Redirect https://www.jotex.no/robots.txt
Redirect Domain www.jotex.no
Redirect Base jotex.no
Domain IPs 91.208.174.141
Redirect IPs 13.226.2.16, 13.226.2.33, 13.226.2.46, 13.226.2.81, 2600:9000:2181:5600:1:cfee:2900:93a1, 2600:9000:2181:5a00:1:cfee:2900:93a1, 2600:9000:2181:8200:1:cfee:2900:93a1, 2600:9000:2181:9600:1:cfee:2900:93a1, 2600:9000:2181:a00:1:cfee:2900:93a1, 2600:9000:2181:ac00:1:cfee:2900:93a1, 2600:9000:2181:c800:1:cfee:2900:93a1, 2600:9000:2181:e000:1:cfee:2900:93a1
Response IP 18.165.140.41
Found Yes
Hash 531910bddc9da63dfabf250c719d59cfa3f6e5db1d15b49670826ff1a481c4eb
SimHash 4012c3c26698

Groups

*

Rule Path
Allow /INTERSHOP/static/WFS/*/*/*/*/css/
Allow /INTERSHOP/static/WFS/*/*/*/*/js/adobe/
Allow /INTERSHOP/static/WFS/*/*/*/*/js/vendor/
Allow /INTERSHOP/static/WFS/*/*/*/*/minifiedJs/
Disallow /INTERSHOP/
Disallow *?write-review
Disallow /search/
Disallow /brand/-/
Disallow /deals/
Disallow /search?
Disallow *?filter
Disallow *_0
Disallow /apm/intake/v2/rum/events

Comments

  • Allowed resources
  • Block crawl on directories
  • Block specific URLs