Log In Sign Up

maistolyga.lt
robots.txt

Robots Exclusion Standard data for maistolyga.lt

Resource Scan

Scan Details

Site Domain maistolyga.lt
Base Domain maistolyga.lt
Scan Status Ok
Last Scan2024-09-18T03:29:33+00:00
Next Scan 2024-09-25T03:29:33+00:00

Last Scan

Scanned2024-09-18T03:29:33+00:00
URL https://maistolyga.lt/robots.txt
Redirect https://www.maistolyga.lt/robots.txt
Redirect Domain www.maistolyga.lt
Redirect Base maistolyga.lt
Domain IPs 104.26.6.139, 104.26.7.139, 172.67.70.134, 2606:4700:20::681a:68b, 2606:4700:20::681a:78b, 2606:4700:20::ac43:4686
Redirect IPs 104.26.6.139, 104.26.7.139, 172.67.70.134, 2606:4700:20::681a:68b, 2606:4700:20::681a:78b, 2606:4700:20::ac43:4686
Response IP 104.26.7.139
Found Yes
Hash d3c0dd1fbdd85d8573d65445f1b9ba9e2c5e887c7dad5c0c1f9303bff9ebbd9d
SimHash 7b14d848e2e6

Groups

*

Rule Path
Allow /
Allow /index/
Allow /paskyra/prisijungti/
Allow /paskyra/registracija/
Allow /paskyra/pamirsau/
Allow /products/
Allow /products/search/
Allow /produktai/paieska/
Allow /produktai/ziureti/*
Allow /products/view/*
Allow /receptai/viesi/
Allow /receptai/perziura/*
Allow /bendrinama/kmi-skaiciuokle/
Allow /bendrinama/skaiciuokle/
Allow /blogas/
Allow /knowledge/
Allow /knowledge/*
Allow /knowledge/view/siuolaikine-mityba/
Allow /programele/
Allow /instrukcijos/programele/
Allow /instrukcijos/android/
Allow /instrukcijos/ios/
Allow /instrukcijos/windows/
Allow /instrukcijos/macintosh/
Allow /instrukcijos/facebook-android/
Allow /instrukcijos/facebook-ios/
Allow /naryste/
Allow /parduotuve/pirkti/premium/
Disallow /sistema/atnaujinimas17/
Disallow /paskyra/patvirtinti/
Disallow /paskyra/patvirtinti/*
Disallow /paskyra/pamirsau/
Disallow /paskyra/atstatymas/
Disallow /paskyra/atstatymas/*
Disallow /paskyra/atsijungti/
Disallow /paskyra/sugrazinimas/
Disallow /paskyra/sugrazinimas/*
Disallow /receptai/viesi/produktas/*
Disallow /receptai/viesi/vartotojas/*
Disallow /mitybos-planas/planas123/
Disallow /parduotuve/pirkti/pro/
Disallow /mitybos-planas/ai-info/
Disallow bendrinama/paieska/
Disallow bendrinama/paieska/*
Disallow /bendrinama/paieska/*
Disallow /public/search/*
Disallow /api.php
Disallow /node.php
Disallow /cron.php
Disallow /produktai/grupes/
Disallow /produktai/grupe/
Disallow /produktai/grupe/*
Disallow /pagalba/duk/
Disallow /pagalba/naudojimosi-taisykles/
Disallow /informacija/naudojimosi-taisykles/
Disallow /bendrinama/vartotojas/
Disallow /bendrinama/vartotojas/*
Disallow /bendrinama/narys/
Disallow /bendrinama/narys/*
Disallow /bendrinama/nariu-top-vakar/
Disallow /bendrinama/nariu-top-vakar/*
Disallow /bendrinama/nariu-top-15-vakar/
Disallow /bendrinama/nariu-top-15-vakar/*
Disallow /mityba/valyti/
Disallow /mityba/valyti/*
Disallow /kalba/keisti/*
Disallow /support/agreement/
Disallow /support/privacy-policy/
Disallow /informacija/privatumo-politika/
Disallow /export/agreement-print/
Disallow /products/group/
Disallow /products/group/*
Disallow /public/time-test/
Disallow /languages/change/
Disallow /admin/*
Disallow /tungumal/breyting/
Disallow /programele/gauti/
Disallow /app/get/
Disallow /instrukcijos/index/
Disallow /manual/index/
Disallow /parduotuve/apmoketi/
Disallow /parduotuve/uzsakymas-apdorojamas/
Disallow /parduotuve/uzsakymo-perziura/
Disallow /parduotuve/sisteminiai-taskai/
Disallow /parduotuve/pro-paskyra/
Disallow /shop/buy-membership-process/
Disallow /shop/processing-order/
Disallow /shop/order-view/
Disallow /shop/credits-info/
Disallow /shop/stripe-payment-confirm/
Disallow /account/confirm/
Disallow /account/confirm/*
Disallow /account/logout/
Disallow /account/reset/
Disallow /account/psw-reset/
Disallow /account/psw-reset/*
Disallow /account/recovery/
Disallow /account/recovery/*
Disallow /account/google-sign-in/

Other Records

Field Value
crawl-delay 10

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

repolookoutbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

buck

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

timpibot/0.8

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

senutobot

Rule Path
Disallow /

dataprovider

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /
Back to top

Comments

  • BLEXBot Crawler:
  • MJ12bot (Majesty bot)
  • AdsBot
  • SERPSTATBOT
  • DataForSEO Link Bot:
  • RepoLookoutBot
  • VelenPublicWebCrawler
  • dotbot
  • https://app.hypefactors.com/media-monitoring/about.html
  • SemrushBot
  • GrapeShot
  • CCBot
  • AhrefsBot
  • AwarioBot
  • SeekportBot
  • Amazon bot
  • Zoominfobot
  • Timpi bot
  • GPTBot
  • GeedoBot
  • SenutoBot
  • Dataprovider.com
  • Babbar bot (crawler)
Back to top