Log In Sign Up

bpcuk.com
robots.txt

Robots Exclusion Standard data for bpcuk.com

Resource Scan

Scan Details

Site Domain bpcuk.com
Base Domain bpcuk.com
Scan Status Ok
Last Scan2024-09-26T12:16:10+00:00
Next Scan 2024-10-26T12:16:10+00:00

Last Scan

Scanned2024-09-26T12:16:10+00:00
URL https://bpcuk.com/robots.txt
Domain IPs 104.21.12.70, 172.67.193.188, 2606:4700:3032::6815:c46, 2606:4700:3036::ac43:c1bc
Response IP 104.21.12.70
Found Yes
Hash f599cdf57798305fdb41cdce4708291ad962b7a84cf84c277976430b713b4b2d
SimHash 2b8ec6b221e7

Groups

googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

apis-google

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

adsbot-google-mobile

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

adsbot-google-mobile

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

adsbot-google

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

mediapartners-google

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

googlebot-image

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

googlebot-news

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

googlebot-video

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

storebot-googlebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

google-site-verification

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

adidxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

bingpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

microsoftpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

duckduckbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

exabot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

facebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

siteauditbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

semrushbot-ba

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

semrushbot-si

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

semrushbot-swa

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

semrushbot-ct

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

splitsignalbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

semrushbot-coub

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

screaming frog seo spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

admantx

Rule Path
Disallow /

aibot

Rule Path
Disallow /

alittle client

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

abonti

Rule Path
Disallow /

aboundex

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

acunetix

Rule Path
Disallow /

afd-verbotsverfahren

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

aipbot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

allsubmitter

Rule Path
Disallow /

alligator

Rule Path
Disallow /

alphabot

Rule Path
Disallow /

anarchie

Rule Path
Disallow /

anarchy

Rule Path
Disallow /

anarchy99

Rule Path
Disallow /

ankit

Rule Path
Disallow /

anthill

Rule Path
Disallow /

apexoo

Rule Path
Disallow /

aspiegel

Rule Path
Disallow /

asterias

Rule Path
Disallow /

atomseobot

Rule Path
Disallow /

attach

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

bbbike

Rule Path
Disallow /

bdcbot

Rule Path
Disallow /

bdfetch

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

backdoorbot

Rule Path
Disallow /

backstreet

Rule Path
Disallow /

backweb

Rule Path
Disallow /

backlink-ceck

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

badass

Rule Path
Disallow /

bandit

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

batchftp

Rule Path
Disallow /

battleztar bazinga

Rule Path
Disallow /

betabot

Rule Path
Disallow /

bigfoot

Rule Path
Disallow /

bitacle

Rule Path
Disallow /

black hole

Rule Path
Disallow /

blackwidow

Rule Path
Disallow /

blackboard

Rule Path
Disallow /

blow

Rule Path
Disallow /

blowfish

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

bolt

Rule Path
Disallow /

botalot

Rule Path
Disallow /

brandprotect

Rule Path
Disallow /

brandwatch

Rule Path
Disallow /

buck

Rule Path
Disallow /

buddy

Rule Path
Disallow /

builtbottough

Rule Path
Disallow /

builtwith

Rule Path
Disallow /

bullseye

Rule Path
Disallow /

bunnyslippers

Rule Path
Disallow /

buzzsumo

Rule Path
Disallow /

catexplorador

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

code87

Rule Path
Disallow /

cshttp

Rule Path
Disallow /

calculon

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

cegbfeieh

Rule Path
Disallow /

censysinspect

Rule Path
Disallow /

cheteam

Rule Path
Disallow /

cheesebot

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

chinaclaw

Rule Path
Disallow /

chlooe

Rule Path
Disallow /

citoid

Rule Path
Disallow /

claritybot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

cloud mapping

Rule Path
Disallow /

cocolyzebot

Rule Path
Disallow /

cogentbot

Rule Path
Disallow /

collector

Rule Path
Disallow /

copier

Rule Path
Disallow /

copyrightcheck

Rule Path
Disallow /

copyscape

Rule Path
Disallow /

cosmos

Rule Path
Disallow /

craftbot

Rule Path
Disallow /

crawling at home project

Rule Path
Disallow /

crazywebcrawler

Rule Path
Disallow /

crescent

Rule Path
Disallow /

crunchbot

Rule Path
Disallow /

curious

Rule Path
Disallow /

custo

Rule Path
Disallow /

cyotekwebcopy

Rule Path
Disallow /

dblbot

Rule Path
Disallow /

diibot

Rule Path
Disallow /

dsearch

Rule Path
Disallow /

dts agent

Rule Path
Disallow /

datacha0s

Rule Path
Disallow /

databasedrivermysqli

Rule Path
Disallow /

demon

Rule Path
Disallow /

deusu

Rule Path
Disallow /

devil

Rule Path
Disallow /

digincore

Rule Path
Disallow /

digitalpebble

Rule Path
Disallow /

dirbuster

Rule Path
Disallow /

disco

Rule Path
Disallow /

discobot

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

dispatch

Rule Path
Disallow /

dittospyder

Rule Path
Disallow /

dnbcrawler-analytics

Rule Path
Disallow /

dnyzbot

Rule Path
Disallow /

domcopbot

Rule Path
Disallow /

domainappender

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

domainsigmacrawler

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

domains project

Rule Path
Disallow /

download wonder

Rule Path
Disallow /

dragonfly

Rule Path
Disallow /

drip

Rule Path
Disallow /

eccp/1.0

Rule Path
Disallow /

email siphon

Rule Path
Disallow /

email wolf

Rule Path
Disallow /

easydl

Rule Path
Disallow /

ebingbong

Rule Path
Disallow /

ecxi

Rule Path
Disallow /

eirgrabber

Rule Path
Disallow /

erocrawler

Rule Path
Disallow /

evil

Rule Path
Disallow /

express webpictures

Rule Path
Disallow /

extlinksbot

Rule Path
Disallow /

extractor

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

extreme picture finder

Rule Path
Disallow /

eyenetie

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

fdm

Rule Path
Disallow /

fhscan

Rule Path
Disallow /

femtosearchbot

Rule Path
Disallow /

fimap

Rule Path
Disallow /

firefox/7.0

Rule Path
Disallow /

flashget

Rule Path
Disallow /

flunky

Rule Path
Disallow /

foobot

Rule Path
Disallow /

freeuploader

Rule Path
Disallow /

frontpage

Rule Path
Disallow /

fuzz

Rule Path
Disallow /

fyberspider

Rule Path
Disallow /

fyrebot

Rule Path
Disallow /

g-i-g-a-b-o-t

Rule Path
Disallow /

gt::www

Rule Path
Disallow /

galaxybot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

germcrawler

Rule Path
Disallow /

getright

Rule Path
Disallow /

getweb

Rule Path
Disallow /

getintent

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

go!zilla

Rule Path
Disallow /

go-ahead-got-it

Rule Path
Disallow /

gozilla

Rule Path
Disallow /

gotit

Rule Path
Disallow /

grabnet

Rule Path
Disallow /

grabber

Rule Path
Disallow /

grafula

Rule Path
Disallow /

grapefx

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

gridbot

Rule Path
Disallow /

headmasterseo

Rule Path
Disallow /

hmview

Rule Path
Disallow /

htmlparser

Rule Path
Disallow /

http::lite

Rule Path
Disallow /

httrack

Rule Path
Disallow /

haansoft

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

harvest

Rule Path
Disallow /

havij

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

hloader

Rule Path
Disallow /

honolulubot

Rule Path
Disallow /

humanlinks

Rule Path
Disallow /

hybridbot

Rule Path
Disallow /

idbte4m

Rule Path
Disallow /

idbot

Rule Path
Disallow /

irlbot

Rule Path
Disallow /

iblog

Rule Path
Disallow /

id-search

Rule Path
Disallow /

ilsebot

Rule Path
Disallow /

image fetch

Rule Path
Disallow /

image sucker

Rule Path
Disallow /

indeedbot

Rule Path
Disallow /

indy library

Rule Path
Disallow /

infonavirobot

Rule Path
Disallow /

infotekies

Rule Path
Disallow /

intelliseek

Rule Path
Disallow /

interget

Rule Path
Disallow /

internet ninja

Rule Path
Disallow /

internetseer

Rule Path
Disallow /

iria

Rule Path
Disallow /

iskanie

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

joc web spider

Rule Path
Disallow /

jamesbot

Rule Path
Disallow /

jbrofuzz

Rule Path
Disallow /

jennybot

Rule Path
Disallow /

jetcar

Rule Path
Disallow /

jetty

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

joomla

Rule Path
Disallow /

jorgee

Rule Path
Disallow /

justview

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

kenjin spider

Rule Path
Disallow /

keybot translation-search-machine

Rule Path
Disallow /

keyword density

Rule Path
Disallow /

kinza

Rule Path
Disallow /

kozmosbot

Rule Path
Disallow /

lnspiderguy

Rule Path
Disallow /

lwp::simple

Rule Path
Disallow /

lanshanbot

Rule Path
Disallow /

larbin

Rule Path
Disallow /

leap

Rule Path
Disallow /

leechftp

Rule Path
Disallow /

leechget

Rule Path
Disallow /

lexibot

Rule Path
Disallow /

lftp

Rule Path
Disallow /

libweb

Rule Path
Disallow /

libwhisker

Rule Path
Disallow /

liebaofast

Rule Path
Disallow /

lightspeedsystems

Rule Path
Disallow /

likse

Rule Path
Disallow /

linkscan

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

linkbot

Rule Path
Disallow /

linkextractorpro

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

linksmanager

Rule Path
Disallow /

linqiametadatadownloaderbot

Rule Path
Disallow /

linqiarssbot

Rule Path
Disallow /

linqiascrapebot

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

lipperhey spider

Rule Path
Disallow /

litemage_walker

Rule Path
Disallow /

lmspider

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

mfc_tear_sample

Rule Path
Disallow /

midown tool

Rule Path
Disallow /

miixpc

Rule Path
Disallow /

mqqbrowser

Rule Path
Disallow /

msfrontpage

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

mtrobot

Rule Path
Disallow /

mag-net

Rule Path
Disallow /

magnet

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

majestic seo

Rule Path
Disallow /

majestic-seo

Rule Path
Disallow /

majestic12

Rule Path
Disallow /

markmonitor

Rule Path
Disallow /

markwatch

Rule Path
Disallow /

mass downloader

Rule Path
Disallow /

masscan

Rule Path
Disallow /

mata hari

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mb2345browser

Rule Path
Disallow /

meanpath bot

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

metauri

Rule Path
Disallow /

micromessenger

Rule Path
Disallow /

microsoft data access

Rule Path
Disallow /

microsoft url control

Rule Path
Disallow /

minefield

Rule Path
Disallow /
Back to top

Comments

  • Allowed Bots
  • Updated: Thurs 26 Jan 2023
  • Google
  • End Google
  • Bing
  • END Bing
  • Yahoo
  • END Yahoo
  • DuckDuckGo
  • END DuckDuckGo
  • Baidu
  • END Baidu
  • Yandex
  • END Yandex
  • ExaLead
  • END Exalead
  • Facebook
  • END Facebook
  • Applebot
  • END Applebot
  • MOZ
  • END MOZ
  • SEMrush
  • END SEMrush
  • ahrefs
  • END ahrefs
  • Majestic Bot
  • END Majestic Bot
  • Screaming Frog
  • END Screaming Frog
  • END Allowed Bots
  • Bad Bot Blocker
  • Updated: Thurs 26 Jan 2023
Back to top

Warnings

  • 11 invalid lines.
Back to top