Log In Sign Up

bestcountrysingers.com
robots.txt

Robots Exclusion Standard data for bestcountrysingers.com

Resource Scan

Scan Details

Site Domain bestcountrysingers.com
Base Domain bestcountrysingers.com
Scan Status Ok
Last Scan2024-06-14T02:36:00+00:00
Next Scan 2024-06-21T02:36:00+00:00

Last Scan

Scanned2024-06-14T02:36:00+00:00
URL https://bestcountrysingers.com/robots.txt
Domain IPs 2001:41d0:1:1b00:87:98:231:18, 87.98.231.18
Response IP 87.98.231.18
Found Yes
Hash a86568420375aa160128db80ece56c65ff7a52374813aeefef539df00ff65e9a
SimHash 9a35eb6028d8

Groups

abonti

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

accelobot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

amznkassocbot

Rule Path
Disallow /

antbot

Rule Path
Disallow /

archive.org

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

betabot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

contextad bot

Rule Path
Disallow /

copubbot

Rule Path
Disallow /

dapper-host-ip

Rule Path
Disallow /

domainreanimator

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

hubspot

Rule Path
Disallow /

integromedb

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

izsearch

Rule Path
Disallow /

linkapedia

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

lwnutch

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

orangebot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

panscient*

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

semantic-visions

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

siteexplorer.info

Rule Path
Disallow /

sitesucker

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

solomonobot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

squider

Rule Path
Disallow /

stq_bot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

twisted pagegetter

Rule Path
Disallow /

typhoeus

Rule Path
Disallow /

uk_lddc_bot

Rule Path
Disallow /

unister*

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

winhttprequest

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

xenu link sleuth

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

yyspider

Rule Path
Disallow /

zend_http_client

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

*

Rule Path
Disallow /rss/
Disallow /indices/
Disallow /data/
Disallow /plugins/

Other Records

Field Value
crawl-delay 5
Back to top

Warnings

  • 4 invalid lines.
Back to top