Log In Sign Up

sunlife.co.id
robots.txt

Robots Exclusion Standard data for sunlife.co.id

Resource Scan

Scan Details

Site Domain sunlife.co.id
Base Domain sunlife.co.id
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-29T21:21:58+00:00
Next Scan 2026-02-27T21:21:58+00:00

Last Successful Scan

Scanned2025-03-20T16:37:47+00:00
URL https://sunlife.co.id/robots.txt
Redirect https://www.sunlife.co.id/robots.txt
Redirect Domain www.sunlife.co.id
Redirect Base sunlife.co.id
Domain IPs 23.203.72.93
Redirect IPs 23.203.72.93
Response IP 23.203.72.93
Found Yes
Hash 6945cf513034ca4c162f2785bdbcea954b880fb869f350fb8f33a2625703817b
SimHash 2541df1d4b96

Groups

*

Rule Path
Disallow /plan/v/index.jsp?vgnextoid=ec073dbaf83e4210VgnVCM100000abd2d09fRCRD
Disallow /brandmanager*
Disallow /Brandmanager*
Disallow /static/brandmanager*
Disallow /slfglobal*
Disallow /slf/mssp*
Disallow /static/slf/mssp*
Disallow /static/MarCom/Welcome%20to%20Sun%20Life/PDFs/*
Disallow /Canada/Welcome%2Bto%2BSun%2BLife%2BFinancial*
Disallow /Global/Investors/ch.Presentations%2Bfor%2Binvestors.mobile/*
Disallow /us/mercer/
Disallow /us/why-buy/
Disallow /us/nsm/
Disallow /us/Tools/Find%2Ba%2Bdentist
Disallow /us/Tools/Find%20a%20dentist
Disallow /static/slfeb/formembers/DentistSearch.html
Disallow /Canada/MMFL*
Disallow */Site%2BAdministration/*
Disallow /Carolyn.Dicks
Disallow /international/fp*
Disallow /International/fp*
Disallow /Canada/sunlifeCA/SEM%2BFinancial%2Bplanning*
Disallow /Canada/sunlifeCA/SEM%2BFinancial%2Bplanning%2Bproducts*
Disallow /Canada/sunlifeCA/SEM%2BLife%2Binsurance*
Disallow /Canada/sunlifeCA/SEM%2BMortgage*
Disallow /Canada/sunlifeCA/SEM%2BAnnuity*
Disallow /Canada/sunlifeCA/SEM%2BHealth%2BInsurance*
Disallow /Canada/sunlifeCA/Financial%2Breview%2Bdirect*
Disallow /Canada/sunlifeCA/Financial%2Breview*
Disallow /Canada/sunlifeCA/Health%2Breview%2Bsem*
Disallow /Canada/sunlifeCA/Health%2Breview%2Bsocial*
Disallow /Canada/sunlifeCA/Insurance%2Breview*
Disallow /newglobal
Disallow /slf/styleguide*
Disallow /files/advisorabc*
Disallow /ca/Tools%2Band%2BResources/Health%2Band%2BWellness/Health-care%2Bprovider%2Bprofile*
Disallow /ca/Learn%2Band%2BPlan/*pageNo%3D
Disallow /ca/Learn%20and%20Plan/*pageNo%3D

adsbot-google

Rule Path
Disallow /newglobal
Disallow /slf/styleguide*
Disallow

linkedinbot

Rule Path
Disallow /inv/About%2Bus/Newsroom/News%2Breleases/Announcement/Sun%2BLife%2BAnnounces%2BEstablishment%2Bof%2BSLC%2BManagement

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap /en/sitemap.xml
sitemap /id/sitemap.xml
Back to top

Comments

  • Disallow the Learn and Plan paginated pages.
  • Allow google adbot
  • Delay LinkedInBot
Back to top

Warnings

  • 1 invalid line.
Back to top