nifs.no
robots.txt

Robots Exclusion Standard data for nifs.no

Resource Scan

Scan Details

Site Domain nifs.no
Base Domain nifs.no
Scan Status Ok
Last Scan2024-10-11T09:32:46+00:00
Next Scan 2024-11-10T09:32:46+00:00

Last Scan

Scanned2024-10-11T09:32:46+00:00
URL https://nifs.no/robots.txt
Redirect https://www.nifs.no/robots.txt
Redirect Domain www.nifs.no
Redirect Base nifs.no
Domain IPs 2a02:c0:410:1::fe1, 2a02:c0:410:1::fe2, 2a02:c0:410:1::fe3, 2a02:c0:410:1::fe4, 87.238.40.85, 87.238.40.86, 87.238.40.87, 87.238.40.88
Redirect IPs 2a02:c0:410:1::fe1, 2a02:c0:410:1::fe2, 2a02:c0:410:1::fe3, 2a02:c0:410:1::fe4, 87.238.40.85, 87.238.40.86, 87.238.40.87, 87.238.40.88
Response IP 87.238.40.88
Found Yes
Hash f2e0bf6e4bbc72a183bce37e17464ff1272aab5d3e73f9208a624a933e809fca
SimHash 10385a80f828

Groups

*

Rule Path
Disallow /head2head.php
Disallow /lagkamp.php
Disallow /kamper.php
Disallow /kampfakta.php
Disallow /cr
Disallow /codeReplacements
Disallow /widgets
Disallow /nifs

offline explorer

Rule Path
Disallow /

offline explorer/1.4

Rule Path
Disallow /

offline explorer/2.3

Rule Path
Disallow /

mercator

Rule Path
Disallow /

slysearch

Rule Path
Disallow /

teleport pro

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

bytedance

Rule Path
Disallow /

Comments

  • Start legacy stuff
  • End Legacy stuff
  • Start AI crawler block
  • End AI crawler block
  • Start things found on other sites