vi.yellowpages.net
robots.txt

Robots Exclusion Standard data for vi.yellowpages.net

Resource Scan

Scan Details

Site Domain vi.yellowpages.net
Base Domain yellowpages.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2024-08-28T16:00:27+00:00
Next Scan 2024-09-27T16:00:27+00:00

Last Successful Scan

Scanned2024-07-07T15:58:48+00:00
URL https://vi.yellowpages.net/robots.txt
Domain IPs 23.88.10.82
Response IP 23.88.10.82
Found Yes
Hash 18ee545a4a08e8ed6351a45d61e6d1e737d0ca04fdb15e3129da1ba9b5a5a518
SimHash 749c6621005a

Groups

*

Rule Path
Disallow
Disallow /business/
Disallow */listing/*
Disallow */accounts/*
Disallow /offers/
Disallow */en/*
Disallow */pl/*
Disallow */ar/*
Disallow */be/*
Disallow */bg/*
Disallow */cs/*
Disallow */da/*
Disallow */de/*
Disallow */el/*
Disallow */es/*
Disallow */et/*
Disallow */fi/*
Disallow */fil/*
Disallow */fr/*
Disallow */hi/*
Disallow */hu/*
Disallow */it/*
Disallow */ja/*
Disallow */ko/*
Disallow */lt/*
Disallow */lv/*
Disallow */mk/*
Disallow */ms/*
Disallow */nl/*
Disallow */no/*
Disallow */pt/*
Disallow */ro/*
Disallow */ru/*
Disallow */sk/*
Disallow */sl/*
Disallow */sr/*
Disallow */sv/*
Disallow */th/*
Disallow */tr/*
Disallow */uk/*
Disallow */vi/*
Disallow */zh/*
Disallow */en/amp/*
Disallow */pl/amp/*
Disallow */ar/amp/*
Disallow */be/amp/*
Disallow */bg/amp/*
Disallow */cs/amp/*
Disallow */da/amp/*
Disallow */de/amp/*
Disallow */el/amp/*
Disallow */es/amp/*
Disallow */et/amp/*
Disallow */fi/amp/*
Disallow */fil/amp/*
Disallow */fr/amp/*
Disallow */hi/amp/*
Disallow */hu/amp/*
Disallow */it/amp/*
Disallow */ja/amp/*
Disallow */ko/amp/*
Disallow */lt/amp/*
Disallow */lv/amp/*
Disallow */mk/amp/*
Disallow */ms/amp/*
Disallow */nl/amp/*
Disallow */no/amp/*
Disallow */pt/amp/*
Disallow */ro/amp/*
Disallow */ru/amp/*
Disallow */sk/amp/*
Disallow */sl/amp/*
Disallow */sr/amp/*
Disallow */sv/amp/*
Disallow */th/amp/*
Disallow */tr/amp/*
Disallow */uk/amp/*
Disallow */vi/amp/*
Disallow */zh/amp/*
Disallow */pages/*
Disallow */r/*
Disallow */citymap/*
Disallow */citymap*

rogerbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

abonti

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

buddhabot

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

cms crawler

Rule Path
Disallow /

cometrics-bot

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

dataprovider

Rule Path
Disallow /

dataprovider site explorer

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

dnyzbot

Rule Path
Disallow /

dubaiindex

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

freewebmonitoring sitechecker

Rule Path
Disallow /

gigablast

Rule Path
Disallow /

gigablastopensource

Rule Path
Disallow /

gluten free crawler

Rule Path
Disallow /

gobyus

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

iccrawler - icjobs

Rule Path
Disallow /

icjobs

Rule Path
Disallow /

implisensebot

Rule Path
Disallow /

ips-agent

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

komodiabot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

linkstats bot

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

lipperhey spider

Rule Path
Disallow /

lipperhey-kaus-australis

Rule Path
Disallow /

loadtimebot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

mindupbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

netseer

Rule Path
Disallow /

nutch

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

pixray

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

proximic

Rule Path
Disallow /

psbot

Rule Path
Disallow /

qcrawl

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

screenerbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seodiver

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seolytics

Rule Path
Disallow /

seoscanners

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

seoscanners.net/1

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

sitecheck-sitecrawl

Rule Path
Disallow /

siteimprove

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

spiderbot

Rule Path
Disallow /

spiderbot/nutch

Rule Path
Disallow /

spiderlytics

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

unister

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

vebidoobot

Rule Path
Disallow /

webintegration jobroboter webspider

Rule Path
Disallow /

webthumbnail

Rule Path
Disallow /

wi job roboter spider

Rule Path
Disallow /

wonderbot

Rule Path
Disallow /

wonderbot/js

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

xovi

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yet-another-spider

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

psbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

findxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

cliqzbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

coccoc

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

istellabot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

Warnings

  • 4 invalid lines.