Log In Sign Up

guidetolenders.com
robots.txt

Robots Exclusion Standard data for guidetolenders.com

Resource Scan

Scan Details

Site Domain guidetolenders.com
Base Domain guidetolenders.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-07-09T13:29:34+00:00
Next Scan 2024-10-07T13:29:34+00:00

Last Successful Scan

Scanned2023-11-20T09:54:08+00:00
URL https://www.guidetolenders.com/robots.txt
Domain IPs 23.203.72.4
Response IP 104.69.172.188
Found Yes
Hash 20fe8cbce77bd72a5c0956b1fdd74f20f1f8d3e4ccd8f880698743f78beb02a5
SimHash 7896e885e0fa

Groups

mj12bot

Rule Path
Disallow /

mojobot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

snapbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

semanticdiscovery

Rule Path
Disallow /

msrbot

Rule Path
Disallow /

yesup-robot

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

sedo_parking_robot

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

omniexplorer_bot

Rule Path
Disallow /

gurujibot

Rule Path
Disallow /

visbot

Rule Path
Disallow /

hmse_robot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

grbot

Rule Path
Disallow /

lmqueuebot

Rule Path
Disallow /

owsbot

Rule Path
Disallow /

phpdig

Rule Path
Disallow /

nrsbot

Rule Path
Disallow /

vadixbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

nutchcvs

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

clicktagrobot

Rule Path
Disallow /

iccrawler

Rule Path
Disallow /

s4jcrawl

Rule Path
Disallow /

redcarpet

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

charlotte

Rule Path
Disallow /
Back to top