Log In Sign Up

finance-review.com
robots.txt

Robots Exclusion Standard data for finance-review.com

Resource Scan

Scan Details

Site Domain finance-review.com
Base Domain finance-review.com
Scan Status Ok
Last Scan2024-09-08T13:58:10+00:00
Next Scan 2024-10-08T13:58:10+00:00

Last Scan

Scanned2024-09-08T13:58:10+00:00
URL https://finance-review.com/robots.txt
Domain IPs 173.254.106.233
Response IP 173.254.106.233
Found Yes
Hash 66b32a009eabea6aaba998f9ee022269d96d479cdbed2c8745c59e5cea925401
SimHash 1034e1226df0

Groups

ccbot

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

accelobot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

orangebot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandex/1.01.001

Rule Path
Disallow /

yandeximages/3.0

Rule Path
Disallow /

yandex.com

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider/2.0

Rule Path
Disallow /

mail.ru_bot/2.0

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

exabot

Rule Path
Disallow /

obot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

amazonbot

Rule Path
Disallow /

daum

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

dnyzbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

*

Rule Path
Disallow /author/
Back to top