oem.no
robots.txt

Robots Exclusion Standard data for oem.no

Resource Scan

Scan Details

Site Domain oem.no
Base Domain oem.no
Scan Status Ok
Last Scan2025-10-09T05:46:10+00:00
Next Scan 2025-11-08T05:46:10+00:00

Last Scan

Scanned2025-10-09T05:46:10+00:00
URL https://oem.no/robots.txt
Redirect https://www.oem.no/robots.txt
Redirect Domain www.oem.no
Redirect Base oem.no
Domain IPs 104.21.26.205, 172.67.168.137, 2606:4700:3030::ac43:a889, 2606:4700:3034::6815:1acd
Redirect IPs 104.21.26.205, 172.67.168.137, 2606:4700:3030::ac43:a889, 2606:4700:3034::6815:1acd
Response IP 104.21.26.205
Found Yes
Hash 2dcb7db28095d2a61fc75957ab751f4aba8a8ca99c48a47e69734f5b16fd514a
SimHash e04e40920aa0

Groups

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

oszkbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

*

Rule Path
Disallow /cdn-cgi*
Disallow /logg-in*
Disallow /mine-sider*
Disallow /search*
Disallow /clm10
Disallow /tel%3A*
Disallow /*.zip*
Disallow /*.stp*
Disallow /*.step*
Disallow /*.igs*
Disallow /*.ies*
Disallow /*.pdf*
Disallow /*.dwg*
Disallow /*.dxf*
Disallow /*.dwf*
Disallow /*.ldt*
Disallow /*.xls*
Disallow /*.doc*
Disallow /*.rar*

Other Records

Field Value
sitemap https://www.oem.no/XmlSitemap

Comments

  • Block MJ12Bot
  • Block PetalBot and AspiegelBot
  • Block AhrefsBot
  • Block SEMrushBot
  • Block DotBot
  • Block MauiBot
  • Block OSZKbot
  • Block Bytespider