4fstore.lt
robots.txt

Robots Exclusion Standard data for 4fstore.lt

Resource Scan

Scan Details

Site Domain 4fstore.lt
Base Domain 4fstore.lt
Scan Status Ok
Last Scan2024-06-19T19:02:07+00:00
Next Scan 2024-07-03T19:02:07+00:00

Last Scan

Scanned2024-06-19T19:02:07+00:00
URL https://4fstore.lt/robots.txt
Domain IPs 23.32.29.107, 23.32.29.89, 2600:1413:b000:1a::17d7:74d, 2600:1413:b000:1a::17d7:753
Response IP 184.50.85.131
Found Yes
Hash f9ffb6c93f0de54a62fab80df77b8eeb48895124febe41ec2a15ad9dfca5a947
SimHash a01d7103ce52

Groups

*

Rule Path
Disallow /customer/*
Disallow /taisykles
Disallow /informacija-apie-taisykliu-pakeitima
Disallow /privatumo-politika
Disallow /nauda-prisiregistravus
Disallow /nauda-apsiperkant
Disallow /akciju-taisykles
Disallow /atitikties-deklaracijos
Disallow /informacija-apie-sankcijas
Disallow /cms/page/view/page_id/2082
Disallow /wishlist/
Disallow /admin/
Disallow /checkout*
Disallow /home
Disallow /index.php
Disallow /catalog/category/*
Disallow */catalogsearch*
Disallow */search*
Disallow *.html/1000
Disallow */kaina-*-*.html
Disallow */spalvos-*-*.html
Disallow */dydis-*-*.html
Disallow */stilius-*-*.html
Disallow */disciplina*.html
Disallow */*savybes*.html
Disallow */atsparumas_vandeniui*.html
Disallow */pralaidumas_orui*.html
Disallow */lytis-*-*.html
Disallow */aulu_aukstis*.html
Disallow */gobtuvas*.html
Disallow */izoliacines_medziagos*.html
Disallow */prisiutas_pamusalas*.html
Disallow */talpa*.html
Disallow */ilgis-*-*.html
Disallow */parama*.html
Disallow */technologija-*.html
Disallow */vyraujanti_medziaga*.html
Disallow */rastas*.html
Disallow */tvari_pletra*.html
Disallow */*stilius*.html
Disallow */kamsalas*.html
Disallow */apsauga_nuo_salcio*.html
Disallow */apsauga_nuo_lietaus*.html
Disallow */kisenes*.html
Disallow */uzsegimas*.html
Disallow */product_style*.html
Disallow */iskirpte*.html
Disallow */oro_salygos*.html
Disallow */filtras*.html
Disallow */klesnes*.html
Disallow */irangos_rusis*.html
Disallow */galines_dalies_rusis*.html
Disallow */juosmuo*.html
Disallow */pakeiciamu_stiklu_sistema*.html
Disallow */akiu_apsauga*.html
Disallow */produkto_tipas*.html
Disallow */special_price_*.html
Disallow */strona-*.html
Disallow */puslapis-*.html
Disallow */cena_specjalna*.html
Disallow */pamusalas*.html
Disallow */categories*
Disallow */newsletters*
Disallow */newsletterowe*
Disallow */newsleterowe*

screaming frog

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandeximages

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

duckduckbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

Other Records

Field Value
sitemap https://4fstore.lt/sitemap_lt_index.xml

Comments

  • filtry
  • kat. mrkt i stare
  • boty, którym opóźniamy zapytania
  • boty blokowane