Log In Sign Up

areena.yle.fi
robots.txt

Robots Exclusion Standard data for areena.yle.fi

Resource Scan

Scan Details

Site Domain areena.yle.fi
Base Domain yle.fi
Scan Status Ok
Last Scan2024-05-12T18:39:23+00:00
Next Scan 2024-05-26T18:39:23+00:00

Last Scan

Scanned2024-05-12T18:39:23+00:00
URL https://areena.yle.fi/robots.txt
Domain IPs 18.164.154.29, 18.164.154.62, 18.164.154.75, 18.164.154.99, 2600:9000:2024:6400:18:f262:e4c0:93a1, 2600:9000:2024:8e00:18:f262:e4c0:93a1, 2600:9000:2024:a200:18:f262:e4c0:93a1, 2600:9000:2024:a400:18:f262:e4c0:93a1, 2600:9000:2024:c800:18:f262:e4c0:93a1, 2600:9000:2024:d800:18:f262:e4c0:93a1, 2600:9000:2024:da00:18:f262:e4c0:93a1, 2600:9000:2024:e00:18:f262:e4c0:93a1
Response IP 3.160.246.110
Found Yes
Hash c72414075ab90e8f9c0ba291ba25386cea8023ae93c3360cef5b711e04731d04
SimHash 32645b504177

Groups

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /
Back to top

Comments

  • Disallow AI crawling
Back to top