Log In Sign Up

lastminute.ee
robots.txt

Robots Exclusion Standard data for lastminute.ee

Resource Scan

Scan Details

Site Domain lastminute.ee
Base Domain lastminute.ee
Scan Status Ok
Last Scan2025-11-21T18:45:39+00:00
Next Scan 2025-12-21T18:45:39+00:00

Last Scan

Scanned2025-11-21T18:45:39+00:00
URL https://lastminute.ee/robots.txt
Domain IPs 104.26.10.231, 104.26.11.231, 172.67.75.143, 2606:4700:20::681a:ae7, 2606:4700:20::681a:be7, 2606:4700:20::ac43:4b8f
Response IP 172.67.75.143
Found Yes
Hash 1eb58896bba95b61cf61a0226cdd561bc7fb604ce9bac73d1c9042b3569eecdc
SimHash eb1ed940cfd8

Groups

*

Rule Path
Disallow /stats/

*

Rule Path
Disallow /glib/*

http://www.almaden.ibm.com/cs/crawler

Rule Path
Disallow /

irlbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

teemer

Rule Path
Disallow /

msrbot

Rule Path
Disallow /

speedy

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

yeti

Rule Path
Disallow /

my-heritrix-crawler

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

xenu's

Rule Path
Disallow /

charlotte

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

exabot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

gigablastopensource

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yandex
yandexbot
semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

ideelaborplagiaat

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

spbot

Rule Path
Disallow /

abonti

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.lastminute.ee/sitemap.xml.gz
Back to top

Comments

  • igas veebis ka ei saa uurimist��d toetada
  • 02.08.2007
  • 26.11.2007
  • 26.11.2007
  • 26.11.2007
  • 26.11.2007
  • 26.11.2007
  • 26.11.2007
  • 26.11.2007
  • hullult palju vigaseid p�ringuid - 06.12.2007
  • http://www.searchme.com/support/ - 10.12.2007
  • 07.05.08
  • 2011-01-31
  • 2011-01-31
  • 2011-05-12
  • 2011-11-29
  • 2013-06-26
  • 2013-11-04
  • 2016-02-26
  • Crawl-delay: 15
  • Crawl-delay: 15
  • 2021-03-20
  • 2021-03-22
  • 2021-10-27
Back to top

Warnings

  • 2 invalid lines.
Back to top