vi.yellowpages.net
robots.txt

Robots Exclusion Standard data for vi.yellowpages.net

Resource Scan

Scan Details

Site Domain vi.yellowpages.net
Base Domain yellowpages.net
Scan Status Ok
Last Scan2024-06-07T15:58:45+00:00
Next Scan 2024-07-07T15:58:45+00:00

Last Scan

Scanned2024-06-07T15:58:45+00:00
URL https://vi.yellowpages.net/robots.txt
Domain IPs 23.88.10.82
Response IP 23.88.10.82
Found Yes
Hash 18ee545a4a08e8ed6351a45d61e6d1e737d0ca04fdb15e3129da1ba9b5a5a518
SimHash 749c6621005a

Groups

*

Rule Path
Disallow
Disallow /business/
Disallow */listing/*
Disallow */accounts/*
Disallow /offers/
Disallow */en/*
Disallow */pl/*
Disallow */ar/*
Disallow */be/*
Disallow */bg/*
Disallow */cs/*
Disallow */da/*
Disallow */de/*
Disallow */el/*
Disallow */es/*
Disallow */et/*
Disallow */fi/*
Disallow */fil/*
Disallow */fr/*
Disallow */hi/*
Disallow */hu/*
Disallow */it/*
Disallow */ja/*
Disallow */ko/*
Disallow */lt/*
Disallow */lv/*
Disallow */mk/*
Disallow */ms/*
Disallow */nl/*
Disallow */no/*
Disallow */pt/*
Disallow */ro/*
Disallow */ru/*
Disallow */sk/*
Disallow */sl/*
Disallow */sr/*
Disallow */sv/*
Disallow */th/*
Disallow */tr/*
Disallow */uk/*
Disallow */vi/*
Disallow */zh/*
Disallow */en/amp/*
Disallow */pl/amp/*
Disallow */ar/amp/*
Disallow */be/amp/*
Disallow */bg/amp/*
Disallow */cs/amp/*
Disallow */da/amp/*
Disallow */de/amp/*
Disallow */el/amp/*
Disallow */es/amp/*
Disallow */et/amp/*
Disallow */fi/amp/*
Disallow */fil/amp/*
Disallow */fr/amp/*
Disallow */hi/amp/*
Disallow */hu/amp/*
Disallow */it/amp/*
Disallow */ja/amp/*
Disallow */ko/amp/*
Disallow */lt/amp/*
Disallow */lv/amp/*
Disallow */mk/amp/*
Disallow */ms/amp/*
Disallow */nl/amp/*
Disallow */no/amp/*
Disallow */pt/amp/*
Disallow */ro/amp/*
Disallow */ru/amp/*
Disallow */sk/amp/*
Disallow */sl/amp/*
Disallow */sr/amp/*
Disallow */sv/amp/*
Disallow */th/amp/*
Disallow */tr/amp/*
Disallow */uk/amp/*
Disallow */vi/amp/*
Disallow */zh/amp/*
Disallow */pages/*
Disallow */r/*
Disallow */citymap/*
Disallow */citymap*

rogerbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

abonti

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

buddhabot

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

cms crawler

Rule Path
Disallow /

cometrics-bot

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

dataprovider

Rule Path
Disallow /

dataprovider site explorer

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

dnyzbot

Rule Path
Disallow /

dubaiindex

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

freewebmonitoring sitechecker

Rule Path
Disallow /

gigablast

Rule Path
Disallow /

gigablastopensource

Rule Path
Disallow /

gluten free crawler

Rule Path
Disallow /

gobyus

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

iccrawler - icjobs

Rule Path
Disallow /

icjobs

Rule Path
Disallow /

implisensebot

Rule Path
Disallow /

ips-agent

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

komodiabot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

linkstats bot

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

lipperhey spider

Rule Path
Disallow /

lipperhey-kaus-australis

Rule Path
Disallow /

loadtimebot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

mindupbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

netseer

Rule Path
Disallow /

nutch

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

pixray

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

proximic

Rule Path
Disallow /

psbot

Rule Path
Disallow /

qcrawl

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

screenerbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seodiver

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

seolytics

Rule Path
Disallow /

seoscanners

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

seoscanners.net/1

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

sitecheck-sitecrawl

Rule Path
Disallow /

siteimprove

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

spiderbot

Rule Path
Disallow /

spiderbot/nutch

Rule Path
Disallow /

spiderlytics

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

unister

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

vebidoobot

Rule Path
Disallow /

webintegration jobroboter webspider

Rule Path
Disallow /

webthumbnail

Rule Path
Disallow /

wi job roboter spider

Rule Path
Disallow /

wonderbot

Rule Path
Disallow /

wonderbot/js

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

xovi

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yet-another-spider

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

psbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

findxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

cliqzbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

coccoc

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

istellabot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

Warnings

  • 4 invalid lines.