Log In Sign Up

e24.no
robots.txt

Robots Exclusion Standard data for e24.no

Resource Scan

Scan Details

Site Domain e24.no
Base Domain e24.no
Scan Status Ok
Last Scan2026-01-01T01:00:26+00:00
Next Scan 2026-01-08T01:00:26+00:00

Last Scan

Scanned2026-01-01T01:00:26+00:00
URL https://e24.no/robots.txt
Domain IPs 13.226.2.101, 13.226.2.54, 13.226.2.73, 13.226.2.81, 2600:9000:21f8:2200:18:becf:5dc0:93a1, 2600:9000:21f8:8200:18:becf:5dc0:93a1, 2600:9000:21f8:9a00:18:becf:5dc0:93a1, 2600:9000:21f8:b200:18:becf:5dc0:93a1, 2600:9000:21f8:b400:18:becf:5dc0:93a1, 2600:9000:21f8:be00:18:becf:5dc0:93a1, 2600:9000:21f8:c00:18:becf:5dc0:93a1, 2600:9000:21f8:c400:18:becf:5dc0:93a1
Response IP 13.226.2.81
Found Yes
Hash 3f490092c7f5c3779975229565711a85dd8893813e3ffbf4829d51cbff857301
SimHash 3e5c0991c467

Groups

*
oai-searchbot

Rule Path
Disallow /sok*?*
Allow /

googlebot-news

Rule Path
Disallow /annonsorinnhold/

ai2bot
ai2bot-dolma
amazonbot
anthropic-ai
applebot-extended
bytespider
ccbot
claude-web
claudebot
cohere-ai
cohere-training-data-crawler
crawlspace
diffbot
facebookbot
friendlycrawler
google-extended
iaskspider/2.0
icc-crawler
imagesiftbot
img2dataset
kangaroo bot
meta-externalagent
omgili
omgilibot
pangubot
perplexitybot
petalbot
scrapy
semrushbot-ocob
semrushbot-swa
sidetrade indexer bot
timpibot
velenpublicwebcrawler
webzio-extended
youbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://e24.no/sitemaps/files/articles-48hrs.xml
sitemap https://e24.no/sitemap.xml
Back to top

Comments

  • All crawlers and Open AI Search BOT rules
  • Google Bot News rules
  • Start AI crawler block
  • E24 does not permit unlicensed use of our content for training large language models.
  • All use, reproductions and extractions of our content for such purposes require specific
  • approval by E24. We explicitly disallow text and data mining and other technical means
  • designed to or with the effect that they enable unlicensed use of our content for training
  • AI models.
  • End AI crawler block
  • Start Sitemaps
  • End Sitemaps
Back to top