Log In Sign Up

aftenposten.no
robots.txt

Robots Exclusion Standard data for aftenposten.no

Resource Scan

Scan Details

Site Domain aftenposten.no
Base Domain aftenposten.no
Scan Status Ok
Last Scan2024-05-31T03:03:48+00:00
Next Scan 2024-06-07T03:03:48+00:00

Last Scan

Scanned2024-05-31T03:03:48+00:00
URL https://aftenposten.no/robots.txt
Redirect https://www.aftenposten.no/robots.txt
Redirect Domain www.aftenposten.no
Redirect Base aftenposten.no
Domain IPs 195.88.54.137, 195.88.55.137, 2001:67c:21e0::ab0
Redirect IPs 195.88.54.137, 195.88.55.137, 2001:67c:21e0::ab0
Response IP 195.88.55.137
Found Yes
Hash 8f331560b0355ab166f006dab57cff01e765db123193f5497f21c8c5d01439b0
SimHash 8e75416bcff6

Groups

*

Rule Path
Disallow /config/
Disallow /template/
Disallow /incoming/
Disallow /WEB-INF/
Disallow /META-INF/
Disallow /feed/externalheader/
Disallow /feed/externalfooter/
Disallow /adtest/
Disallow /henvisninger/div/
Disallow /sok
Disallow /kampanje/
Disallow /widgets/
Disallow /skeleton
Disallow /personalized-frontpage-feed

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /
Back to top

Comments

  • Aftenposten does not permit unlicensed use of our content for training large language models or other
  • artificial intelligence technology. All use, reproductions and extractions of our content for
  • such purposes require specific # approval by Aftenposten. We explicitly disallow text and data mining and
  • all other technical means desig# ned to or with the effect that they enable unlicensed use,
  • reproduction or extraction of o# ur content for training AI models.
Back to top