Log In Sign Up

mikropc.fi
robots.txt

Robots Exclusion Standard data for mikropc.fi

Resource Scan

Scan Details

Site Domain mikropc.fi
Base Domain mikropc.fi
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-08-24T05:24:14+00:00
Next Scan 2025-09-07T05:24:14+00:00

Last Successful Scan

Scanned2025-08-15T01:37:27+00:00
URL http://mikropc.fi/robots.txt
Redirect https://www.mikrobitti.fi/robots.txt
Redirect Domain www.mikrobitti.fi
Redirect Base mikrobitti.fi
Domain IPs 3.5.65.173, 52.218.109.52, 52.218.37.140, 52.218.40.116, 52.218.90.212, 52.92.0.44, 52.92.1.164, 52.92.2.196
Redirect IPs 3.164.182.125, 3.164.182.70, 3.164.182.78, 3.164.182.96
Response IP 18.244.87.49
Found Yes
Hash 8fdfdd4963f198e8efd630d1984a0cd05086779158d4c568de4b3850df512537
SimHash 668ccb01c5f6

Groups

googlebot

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

googlebot-image

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

googlebot-video

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

googlebot-news

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

bingbot

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

Other Records

Field Value
crawl-delay 10

msnbot

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

Other Records

Field Value
crawl-delay 10

msnbot-media

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

Other Records

Field Value
crawl-delay 10

duckduckbot

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

yandex

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

Other Records

Field Value
crawl-delay 10

applebot

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

Other Records

Field Value
crawl-delay 10

alexabot

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

facebookbot

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

twitterbot

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

linkedinbot

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

slackbot

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

discordbot

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

whatsapp

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

sogoubot

Rule Path
Disallow /

sogouwebspider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

baiduspider-mobile

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

perplexity-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

ai2bot-dolma

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

brightbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

cohere-training-data-crawler

Rule Path
Disallow /

cotoyogi

Rule Path
Disallow /

crawlspace

Rule Path
Disallow /

deepseek

Rule Path
Disallow /

deepseekbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

factset_spyderbot

Rule Path
Disallow /

firecrawlagent

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

googleother

Rule Path
Disallow /

googleother-image

Rule Path
Disallow /

googleother-video

Rule Path
Disallow /

iaskspider

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

img2dataset

Rule Path
Disallow /

imgproxy

Rule Path
Disallow /

isscyberriskcrawler

Rule Path
Disallow /

kangaroobot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

novaact

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

operator

Rule Path
Disallow /

pangubot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

semrushbot-ocob

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

sidetradeindexerbot

Rule Path
Disallow /

tiktokspider

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /

youbot

Rule Path
Disallow /

*

Rule Path
Allow /
Disallow /haku
Disallow /api/pages/v2/listingpage/search

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.mikrobitti.fi/sitemaps/sitemapindex.xml
Back to top

Comments

  • Google bots
  • Bing bots
  • Other major search engines
  • Social media and messaging bots
  • Chinese search engines - blocked everywhere
  • Other search engines - blocked everywhere
  • AI user agents - blocked everywhere
  • Main AI training bots - blocked everywhere
  • Other AI bots - blocked everywhere
  • All other bots
  • Sitemaps
Back to top