Log In Sign Up

myfidevs.io
robots.txt

Robots Exclusion Standard data for myfidevs.io

Resource Scan

Scan Details

Site Domain myfidevs.io
Base Domain myfidevs.io
Scan Status Ok
Last Scan2024-05-25T13:24:23+00:00
Next Scan 2024-06-24T13:24:23+00:00

Last Scan

Scanned2024-05-25T13:24:23+00:00
URL https://www.myfidevs.io/robots.txt
Domain IPs 3.215.8.216, 34.193.95.20, 34.195.115.224, 34.199.176.145, 34.207.47.205, 52.5.52.207
Response IP 52.5.52.207
Found Yes
Hash 855424bf4245091ae0c3a29045ef53107b7575d4909feb1d8f6b7e3d8280756f
SimHash a3107402cdb3

Groups

facebot

Rule Path
Disallow /

twikle/1.0

Rule Path
Disallow /

leikibot

Rule Path
Disallow /

wesee

Rule Path
Disallow /

coralwebprx

Rule Path
Disallow /

admantx

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

wikido

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

flamingo_searchengine

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

wocbot

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

alexabot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

apexoo spider 1.x

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

yacy

Rule Path
Disallow /

backlink crawlers

Rule Path
Disallow /

attrakt

Rule Path
Disallow /

lucidworks

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

seoengbot

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

accelobot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

swebot

Rule Path
Disallow /

webcrawler

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

adbot

Rule Path
Disallow /

agbot

Rule Path
Disallow /

suggybot

Rule Path
Disallow /

treezy

Rule Path
Disallow /

aboutusbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

netseer

Rule Path
Disallow /

proximic

Rule Path
Disallow /

covario-ids

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

speedy

Rule Path
Disallow /

psbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

nutch

Rule Path
Disallow /

viglink/nutch-1.0

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

gosospider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yrspider

Rule Path
Disallow /

camontspider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

purebot

Rule Path
Disallow /

pikimal

Rule Path
Disallow /

pik-a-part

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

israbot

Rule Path
Disallow /

orthogaffe

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

zao

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

xenu

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

wget

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

bender

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

mediapartners-google

Rule Path
Allow /

*

Rule Path
Disallow /landers/*
Disallow /landers/
Disallow /landers
Disallow /partner-preview/*
Disallow /partner-preview/
Disallow /partner-preview
Disallow /reporting/*
Disallow /reporting/
Disallow /reporting
Back to top

Other Records

Field Value
sitemap https://www.myfinance.com/sitemap.xml
Back to top

Comments

  • Wikipedia work bots:
  • Landers
  • Partner-Preview
  • Reporting links
Back to top

Warnings

  • 2 invalid lines.
Back to top