Log In Sign Up

nordlys.no
robots.txt

Robots Exclusion Standard data for nordlys.no

Resource Scan

Scan Details

Site Domain nordlys.no
Base Domain nordlys.no
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-24T01:15:29+00:00
Next Scan 2025-11-23T01:15:29+00:00

Last Successful Scan

Scanned2025-09-24T15:36:19+00:00
URL https://nordlys.no/robots.txt
Redirect https://www.nordlys.no/robots.txt
Redirect Domain www.nordlys.no
Redirect Base nordlys.no
Domain IPs 2a02:c0:ac::e51:1, 87.238.38.1, 87.238.38.2
Redirect IPs 104.18.22.107, 104.18.23.107, 2606:4700::6812:166b, 2606:4700::6812:176b
Response IP 104.18.23.107
Found Yes
Hash 894856ab67697f82bf1fe3ec877245a88ff19d3b967a687c4eded7aaf6242ce4
SimHash 5097c894e51f

Groups

*

Rule Path
Allow /ads.txt
Disallow /rss/
Disallow /sok/
Disallow /journalister
Disallow /journalister/
Disallow /vis/rubrikk/*localGeo%3Dfalse*
Disallow /vis/rubrikk/*page%3D*
Disallow /vis/rubrikk/*foreignGeo%3D*
Disallow /vis/rubrikk/*priceTo%3D*
Disallow /vis/rubrikk/*priceFrom%3D*
Disallow /vis/rubrikk/*compId%3D*
Disallow /vis/rubrikk/*condition%3D*
Disallow /vis/rubrikk/*saleType%3D*
Disallow /vis/rubrikk/*type%3D*
Disallow /vis/rubrikk/*monthlyRentTo%3D*
Disallow /vis/rubrikk/*pRoomAreaTo%3D*
Disallow /vis/rubrikk/*bedroomsFrom%3D*
Disallow /vis/rubrikk/*mileageTo%3D*
Disallow /vis/rubrikk/*mileageFrom%3D*
Disallow /vis/rubrikk/*modelYearTo%3D*
Disallow /vis/rubrikk/*modelYearFrom%3D*
Disallow /vis/rubrikk/*fuelType%3D*
Disallow /vis/rubrikk/*gearType%3D*
Disallow /vis/rubrikk/*wheelDrive%3D*
Disallow /vis/rubrikk/*engineHpTo%3D*
Disallow /vis/rubrikk/*engineHpFrom%3D*
Disallow /vis/rubrikk/*seats%3D*
Disallow /vis/rubrikk/*workHours%3D*
Disallow /vis/rubrikk/*workSector%3D*
Disallow /vis/rubrikk/*makeModel%3D*

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /
Back to top

Comments

  • ACAP version=1.0
  • Rubrikk
  • Start AI crawler block
  • End AI crawler block
Back to top

Warnings

  • `acap-crawler` is not a known field.
  • `acap-disallow-crawl` is not a known field.
Back to top