fundlater.com.au
robots.txt

Robots Exclusion Standard data for fundlater.com.au

Resource Scan

Scan Details

Site Domain fundlater.com.au
Base Domain fundlater.com.au
Scan Status Ok
Last Scan2024-11-12T02:38:29+00:00
Next Scan 2024-11-19T02:38:29+00:00

Last Scan

Scanned2024-11-12T02:38:29+00:00
URL https://fundlater.com.au/robots.txt
Redirect https://www.fundlater.com.au/robots.txt
Redirect Domain www.fundlater.com.au
Redirect Base fundlater.com.au
Domain IPs 20.53.115.182
Redirect IPs 18.155.68.127, 18.155.68.24, 18.155.68.60, 18.155.68.90, 2600:9000:23d2:1600:1a:303a:78c0:93a1, 2600:9000:23d2:4000:1a:303a:78c0:93a1, 2600:9000:23d2:6200:1a:303a:78c0:93a1, 2600:9000:23d2:8c00:1a:303a:78c0:93a1, 2600:9000:23d2:ac00:1a:303a:78c0:93a1, 2600:9000:23d2:c600:1a:303a:78c0:93a1, 2600:9000:23d2:d800:1a:303a:78c0:93a1, 2600:9000:23d2:ea00:1a:303a:78c0:93a1
Response IP 18.155.68.60
Found Yes
Hash 551f0ad955a9d17dd6ce02e0a0b621c783adc938c41689ffb67b1b9fb2b22ef5
SimHash 1a7018558fb5

Groups

*

Rule Path
Disallow /identity/logon?returnUrl*
Disallow /home/_investment-news-articles
Disallow /home/_investment-ideas
Disallow /home/_asx-announcements
Disallow /home/_smh-guide
Disallow /home/_my-investsmart-widget
Disallow /search/_shares
Disallow /search/_Shares
Disallow /search/_announcements
Disallow /search/_Announcements
Disallow /search/_dividends
Disallow /search/_Dividends
Disallow /search/_funds
Disallow /search/_Funds
Disallow /search/_news
Disallow /search/_News
Disallow /investment-news/newsletter/
Disallow /*.asp$
Disallow /*_price-chart
Disallow /portfolio-manager/diversified-portfolios/_getdetails
Disallow /portfolio-manager/diversified-portfolios/_getholdings
Disallow /portfolio-manager/diversified-portfolios/_getdepositswithdrawls
Disallow /portfolio-manager/diversified-portfolios/_gettransactions
Disallow /portfolio-manager/diversified-portfolios/_getdividendsinterest
Disallow /portfolio-manager/diversified-portfolios/_getfees
Disallow /test/throwerror
Disallow /test/version
Disallow /*%26size%3D*
Disallow /resources/pdf/*
Disallow /account/*
Disallow /events/registermodel
Disallow /widgets/*
Disallow /*_related-articles*
Disallow /callback/*
Disallow /oauth/*

Other Records

Field Value
crawl-delay 20

yandex
baiduspider
baidu
mail.ru_bot/2.0
mail.ru_bot
mail.ru
megaindex.ru/2.0
megaindex.ru
megaindex
vagabondo
exabot
bspider
screaming frog seo spider

Rule Path
Disallow /

twitterbot

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.fundlater.com.au/sitemap-fl.xml
sitemap https://www.fundlater.com.au/sitemap-insights-fl.xml

Comments

  • Certain social media sites are whitelisted to allow crawlers to access page markup