idrett.sdir.no
robots.txt

Robots Exclusion Standard data for idrett.sdir.no

Resource Scan

Scan Details

Site Domain idrett.sdir.no
Base Domain sdir.no
Scan Status Ok
Last Scan2025-10-22T04:06:17+00:00
Next Scan 2025-11-21T04:06:17+00:00

Last Scan

Scanned2025-10-22T04:06:17+00:00
URL https://idrett.sdir.no/robots.txt
Domain IPs 176.111.200.83
Response IP 176.111.200.83
Found Yes
Hash 2e6feeb817b31647aac15c19822d8c74605be5ba5fcd9af8ffd19ed26f8825ba
SimHash 431dc0d3cf90

Groups

*

Rule Path
Disallow /error/blackhole
Disallow /storage/
Disallow /fileStorage/

dotbot

Rule Path
Disallow /

googlebot

Rule Path
Allow *.js
Allow *.css

Other Records

Field Value
crawl-delay 10

petalbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://idrett.sdir.no/sitemap/events.xml
sitemap https://idrett.sdir.no/sitemap/blog.xml
sitemap https://idrett.sdir.no/sitemap/classifieds.xml
sitemap https://idrett.sdir.no/sitemap/forum.xml
sitemap https://idrett.sdir.no/sitemap/pages.xml