kiva.org
robots.txt

Robots Exclusion Standard data for kiva.org

Resource Scan

Scan Details

Site Domain kiva.org
Base Domain kiva.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-25T07:14:26+00:00
Next Scan 2024-10-24T07:14:26+00:00

Last Successful Scan

Scanned2024-06-20T07:13:36+00:00
URL https://kiva.org/robots.txt
Redirect https://www.kiva.org/robots.txt
Redirect Domain www.kiva.org
Redirect Base kiva.org
Domain IPs 44.231.60.173, 54.200.201.132
Redirect IPs 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91
Response IP 199.232.45.91
Found Yes
Hash a05158efbb7591e7515cca7a6d0dc837f491ee4c627ffa8cd4c3632860c5e3a5
SimHash bcb99830b933

Groups

*

Rule Path
Disallow /authenticate
Disallow /error
Disallow /last_login
Disallow /login
Disallow /logout
Disallow /oauth
Disallow /register
Disallow /ui-login
Disallow /ui-logout
Disallow /basket
Disallow /checkout
Disallow /thanks
Disallow /account
Disallow /admin
Disallow /ajax
Disallow /myLenderId
Disallow /portfolio
Disallow /settings
Disallow /updates/myloans
Disallow /withdraw
Disallow /app.php
Disallow /i/
Disallow /*?*pageID=
Disallow /teams/*/loans
Disallow /teams/join/*

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

petalbot

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.kiva.org/sitemaps/index.xml

Comments

  • Login pages
  • Checkout pages
  • Common pages requiring login
  • Other