vaski.finna.fi
robots.txt

Robots Exclusion Standard data for vaski.finna.fi

Resource Scan

Scan Details

Site Domain vaski.finna.fi
Base Domain finna.fi
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-06-11T10:40:34+00:00
Next Scan 2025-08-10T10:40:34+00:00

Last Successful Scan

Scanned2025-04-06T10:38:04+00:00
URL https://vaski.finna.fi/robots.txt
Domain IPs 104.22.30.124, 104.22.31.124, 172.67.39.124, 2606:4700:10::6816:1e7c, 2606:4700:10::6816:1f7c, 2606:4700:10::ac43:277c
Response IP 104.22.30.124
Found Yes
Hash f52a3f2c811eaee264654a233bf7d6566852f8a797bb15ba552ac336802aee71
SimHash 7581d8598b93

Groups

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claudebot/1.0

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

omgili

Rule Path
Disallow /

academicbotrtu

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

test-bot

Rule Path
Disallow /

*

Rule Path
Disallow /*/api/
Disallow /*/*/api/
Disallow /MetaLib/
Disallow /*/MetaLib/
Disallow /Primo/
Disallow /*/Primo/
Disallow /PrimoRecord/
Disallow /*/PrimoRecord/
Disallow /Record/*/Holdings
Disallow /*/Record/*/Holdings
Disallow /SearchApi/
Disallow /*/SearchApi/
Disallow /AJAX/
Disallow */AJAX/
Disallow /Blender/
Disallow */Blender/
Disallow /Browse/
Disallow */Browse/
Disallow /EDS/
Disallow */EDS/
Disallow /EDSRecord/
Disallow */EDSRecord/
Disallow /L1/
Disallow */L1/
Disallow /MyResearch/
Disallow */MyResearch/
Disallow /Primo/
Disallow */Primo/
Disallow /PrimoRecord/
Disallow */PrimoRecord/
Disallow /QRCode/
Disallow */QRCode/
Disallow /Search/
Disallow */Search/
Disallow /Search2/
Disallow */Search2/
Disallow /Summon/
Disallow */Summon/
Disallow /SummonRecord/
Disallow */SummonRecord/

Other Records

Field Value
sitemap https://vaski.finna.fi/sitemapIndex.xml.gz

Comments

  • Take into account both the default view and any non-default views
  • (hence two versions of each url)
  • Allow indexing of record pages by commenting out the following two lines:
  • Disallow: /Record/
  • Disallow: /*/Record/