vaara.finna.fi
robots.txt

Robots Exclusion Standard data for vaara.finna.fi

Resource Scan

Scan Details

Site Domain vaara.finna.fi
Base Domain finna.fi
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-08T23:10:19+00:00
Next Scan 2026-02-06T23:10:19+00:00

Last Successful Scan

Scanned2025-04-06T09:49:43+00:00
URL https://vaara.finna.fi/robots.txt
Domain IPs 104.22.30.124, 104.22.31.124, 172.67.39.124, 2606:4700:10::6816:1e7c, 2606:4700:10::6816:1f7c, 2606:4700:10::ac43:277c
Response IP 172.67.39.124
Found Yes
Hash ee4ca4364ad046997aac7f22da090473c21da3fb97b4fbd7a0616e258dd55948
SimHash 7581d8598b91

Groups

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claudebot/1.0

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

omgili

Rule Path
Disallow /

academicbotrtu

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

test-bot

Rule Path
Disallow /

*

Rule Path
Disallow /*/api/
Disallow /*/*/api/
Disallow /MetaLib/
Disallow /*/MetaLib/
Disallow /Primo/
Disallow /*/Primo/
Disallow /PrimoRecord/
Disallow /*/PrimoRecord/
Disallow /Record/*/Holdings
Disallow /*/Record/*/Holdings
Disallow /SearchApi/
Disallow /*/SearchApi/
Disallow /AJAX/
Disallow */AJAX/
Disallow /Blender/
Disallow */Blender/
Disallow /Browse/
Disallow */Browse/
Disallow /EDS/
Disallow */EDS/
Disallow /EDSRecord/
Disallow */EDSRecord/
Disallow /L1/
Disallow */L1/
Disallow /MyResearch/
Disallow */MyResearch/
Disallow /Primo/
Disallow */Primo/
Disallow /PrimoRecord/
Disallow */PrimoRecord/
Disallow /QRCode/
Disallow */QRCode/
Disallow /Search/
Disallow */Search/
Disallow /Search2/
Disallow */Search2/
Disallow /Summon/
Disallow */Summon/
Disallow /SummonRecord/
Disallow */SummonRecord/

Other Records

Field Value
sitemap https://vaara.finna.fi/sitemapIndex.xml.gz

Comments

  • Take into account both the default view and any non-default views
  • (hence two versions of each url)
  • Allow indexing of record pages by commenting out the following two lines:
  • Disallow: /Record/
  • Disallow: /*/Record/