Log In Sign Up

bencom.nl
robots.txt

Robots Exclusion Standard data for bencom.nl

Resource Scan

Scan Details

Site Domain bencom.nl
Base Domain bencom.nl
Scan Status Ok
Last Scan2024-09-18T11:02:23+00:00
Next Scan 2024-10-02T11:02:23+00:00

Last Scan

Scanned2024-09-18T11:02:23+00:00
URL https://bencom.nl/robots.txt
Redirect https://www.bencom.nl/robots.txt
Redirect Domain www.bencom.nl
Redirect Base bencom.nl
Domain IPs 52.157.220.210
Redirect IPs 35.189.118.249
Response IP 35.189.118.249
Found Yes
Hash db508e2e4367d4bb566527fd3663267e5a3cb7b07ee25009c2c95ebeae8d76ee
SimHash 131c65c1ceb0

Groups

*

Rule Path
Disallow /beheerbellen
Disallow /inc
Disallow /medewerkers

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

webcrawler

Rule Path
Disallow /

bubing

Rule Path
Disallow /

proximic

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

vagabondo

Rule Path
Disallow /

bilbo

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yandexvideo

Rule Path
Disallow /

yandexmedia

Rule Path
Disallow /

yandexfavicons

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

niki-bot

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

beetlebot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

melodibot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

queryseekerspider

Rule Path
Disallow /

buddhabot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

imrbot

Rule Path
Disallow /

acoonbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

gufybot

Rule Path
Disallow /

nesotebot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

solomonobot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

noyabot

Rule Path
Disallow /

flappybot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

proximic

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

nutch

Rule Path
Disallow /

berlin-fu-cow

Rule Path
Disallow /

riddler

Rule Path
Disallow /

butterfly

Rule Path
Disallow /
Back to top

Comments

  • robots.txt file for http://www.bencom.nl
  • User-Agent: bingbot
  • Disallow: /
Back to top

Warnings

  • 4 invalid lines.
Back to top