Log In Sign Up

biosklep.com.pl
robots.txt

Robots Exclusion Standard data for biosklep.com.pl

Resource Scan

Scan Details

Site Domain biosklep.com.pl
Base Domain biosklep.com.pl
Scan Status Ok
Last Scan2025-08-10T22:16:53+00:00
Next Scan 2025-09-09T22:16:53+00:00

Last Scan

Scanned2025-08-10T22:16:53+00:00
URL https://biosklep.com.pl/robots.txt
Domain IPs 77.79.221.136, 77.79.221.168
Response IP 77.79.221.136
Found Yes
Hash 4c335a3e715e23d7ac3b04858fa52a6c555894012be777ba1eb035a3cb3dbbb3
SimHash d33db3728f4f

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

*

Rule Path
Disallow /application
Disallow /environment
Allow /environment/cache/images
Disallow /libraries
Disallow /*/fav/add
Disallow /*/p/comment/add
Disallow /*/p/mail/recommend
Disallow /*/p/q
Disallow /*/reg
Disallow /*/login
Disallow /*/basket
Disallow /*/webapi

rogerbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blackwidow

Rule Path
Disallow /

chinaclaw

Rule Path
Disallow /

custo

Rule Path
Disallow /

disco

Rule Path
Disallow /

download\ demon

Rule Path
Disallow /

ecatch

Rule Path
Disallow /

eirgrabber

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

express\ webpictures

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

eyenetie

Rule Path
Disallow /

flashget

Rule Path
Disallow /

getright

Rule Path
Disallow /

getweb!

Rule Path
Disallow /

go!zilla

Rule Path
Disallow /

go-ahead-got-it

Rule Path
Disallow /

grabnet

Rule Path
Disallow /

grafula

Rule Path
Disallow /

hmview

Rule Path
Disallow /

httrack

Rule Path
Disallow /

image\ stripper

Rule Path
Disallow /

image\ sucker

Rule Path
Disallow /

indy\ library

Rule Path
Disallow /

interget

Rule Path
Disallow /

internet\ ninja

Rule Path
Disallow /

jetcar

Rule Path
Disallow /

joc\ web\ spider

Rule Path
Disallow /

larbin

Rule Path
Disallow /

leechftp

Rule Path
Disallow /

mass\ downloader

Rule Path
Disallow /

midown\ tool

Rule Path
Disallow /

mister\pix

Rule Path
Disallow /

navroad

Rule Path
Disallow /

nearsite

Rule Path
Disallow /

netants

Rule Path
Disallow /

netspider

Rule Path
Disallow /

net\vampire

Rule Path
Disallow /

netzip

Rule Path
Disallow /

octopus

Rule Path
Disallow /

offline\explorer

Rule Path
Disallow /

offline\navigator

Rule Path
Disallow /

pagegrabber

Rule Path
Disallow /

papa\foto

Rule Path
Disallow /

pavuk

Rule Path
Disallow /

pcbrowser

Rule Path
Disallow /

realdownload

Rule Path
Disallow /

reget

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

smartdownload

Rule Path
Disallow /

superbot

Rule Path
Disallow /

superhttp

Rule Path
Disallow /

surfbot

Rule Path
Disallow /

takeout

Rule Path
Disallow /

teleport\pro

Rule Path
Disallow /

voideye

Rule Path
Disallow /

web\image\collector

Rule Path
Disallow /

web\sucker

Rule Path
Disallow /

webauto

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webfetch

Rule Path
Disallow /

webgo\is

Rule Path
Disallow /

webleacher

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

websauger

Rule Path
Disallow /

website\extractor

Rule Path
Disallow /

website\quester

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webwhacker

Rule Path
Disallow /

webzip

Rule Path
Disallow /

wget

Rule Path
Disallow /

widow

Rule Path
Disallow /

wwwoffle

Rule Path
Disallow /

xaldon\ webspider

Rule Path
Disallow /

zeus

Rule Path
Disallow /
Back to top

Warnings

  • `request-rate` is not a known field.
Back to top