Log In Sign Up

businesscheck.co.nz
robots.txt

Robots Exclusion Standard data for businesscheck.co.nz

Resource Scan

Scan Details

Site Domain businesscheck.co.nz
Base Domain businesscheck.co.nz
Scan Status Ok
Last Scan2024-09-23T14:01:32+00:00
Next Scan 2024-09-30T14:01:32+00:00

Last Scan

Scanned2024-09-23T14:01:32+00:00
URL https://businesscheck.co.nz/robots.txt
Domain IPs 139.99.247.212
Response IP 139.99.247.212
Found Yes
Hash 30d948f14bbfed69143c5f7bf7b5abf61ff5cc1c13e2f0bbf765f1f69159c93f
SimHash e374d2704631

Groups

*

Rule Path
Allow /

dotbot

Rule Path
Disallow /
Disallow /filter/

admantx

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

exabot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

wikido

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

wesee:search

Rule Path
Disallow /

bingbot
msnbot
msnbot-media
adidxbot
bingpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

lexxebot

Rule Path
Disallow /
Back to top