bsn.no
robots.txt

Robots Exclusion Standard data for bsn.no

Resource Scan

Scan Details

Site Domain bsn.no
Base Domain bsn.no
Scan Status Ok
Last Scan2026-01-08T14:43:08+00:00
Next Scan 2026-02-07T14:43:08+00:00

Last Scan

Scanned2026-01-08T14:43:08+00:00
URL https://bsn.no/robots.txt
Domain IPs 2a02:4780:c:27ca::1, 62.72.32.63
Response IP 62.72.32.63
Found Yes
Hash 80fbfdafc8476d967b8a3d615d32b3c4ee07ec568ad2874a40cbe71fe738e595
SimHash 716458c4f013

Groups

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 180

*

Rule Path
Disallow

Other Records

Field Value
sitemap https://bsn.no/sitemap_index.xml

Comments

  • OpenAI crawler rate limit - once per 3 minutes
  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK